Theme Title: Deep learning model compiler
Technical Area: Device AI, Compiler
Mobile AI is a kind of technique that doing inference on mobile devices based on deep learning neural network algorithm along with its pre-trained models. There are lots of applications having been adopting this technique, including face detection, liveness detection and PaiLiTao in Taobao App.
This can be easily implemented by loading models with deep learning neural network frameworks and then do the following inference. However, there are several problems with this proposal. First of all, the risk of model leakage is greatly increased due to the separation from framework and models. Secondly, much more efforts have to be put on optimizing code and improving performance for different platforms. Last but not least, there are some redundant nodes in the model that leads to the performance loss.
As a result, a potential solution for problems mentioned above is to build a model compiler to compile the neural network model into binary code as well as do graph merging and op code optimization during the process of compilation in order to improve performance. In addition, we can easily adapt a variety of platforms with the help of the backend of compiler.
- Build model compiler to compile the neural network model into binary code. The isolation between the frontend of compiler and its backend makes us easily to adapt our framework for various platforms and devices.
- Enhance the security of model.
- Improve performance.
Related Research Topics
Not much research has been done on model compiler.
- Tianqi Chen and his team released TVM last year, which is a compiler stack for deep learning systems.
- The LLVM, which is a collection of modular and reusable compiler and toolchain technologies. The design and architecture of LLVM are great reference for us.