Theme Title: Deep learning model compiler

 

Technical Area: Device AI, Compiler

 

Background

Mobile AI is a kind of technique that doing inference on mobile devices based on deep learning neural network algorithm along with its pre-trained models. There are lots of applications having been adopting this technique, including face detection, liveness detection and PaiLiTao in Taobao App.

 

This can be easily implemented by loading models with deep learning neural network frameworks and then do the following inference. However, there are several problems with this proposal. First of all, the risk of model leakage is greatly increased due to the separation from framework and models. Secondly, much more efforts have to be put on optimizing code and improving performance for different platforms. Last but not least, there are some redundant nodes in the model that leads to the performance loss.

 

As a result, a potential solution for problems mentioned above is to build a model compiler to compile the neural network model into binary code as well as do graph merging and op code optimization during the process of compilation in order to improve performance. In addition, we can easily adapt a variety of platforms with the help of the backend of compiler.

 

Target

 

Related Research Topics

Not much research has been done on model compiler.