Topic Title: Memory Based Model for User Intention Prediction


Technical Area: Memory Network, Recommender System



Building models to characterize long term dependencies in sequential data is a monumental challenge in machine learning research. These models can be critical to tasks such as predicting user’s intention from their historical behaviors and answering questions.


Encoder-Decoder models play well in most situations, but their performance is limited by the size of their ‘memory’. When it comes to very long sequences of data, this limitation becomes specifically apparent. Unfortunately, long sequence data are very common in Taobao, because most of the users have diverse interests and huge heterogeneous actions. Memory network takes advantage of both attention and explicit storage and offers an approach to alleviate this limitation.


Users’ historical behaviors in Taobao are essentially sequential data, which contain rich information about the user. We can use memory network to model users’ behaviors and better understand users’ intentions. This technique can be widely used in recommender systems, display advertising systems and estimation of gross merchandise volume, etc.


We invite researchers who are either experts or are keenly aware of the challenges and opportunities in user sequential behavior modeling. This research would be deployed in our real-world systems. We hope that both sellers and buyers on our platform could benefit from this work.



Although models based on memory networks perform well in QA problems, there is still a grand challenge adopting them in e-commerce applications such as in Taobao.


Most models based on memory networks calculate memory for each instance independently. In other words, each instance has a private memory in the traditional memory networks. However, directly modeling a private memory for each user is extremely difficult. On the one hand, the sequence of user’s historical behaviors may contain multiple concurrent interests. Rapid jumping and sudden ending over these interests make the sequence data of a user’s behaviors appear to be noisy. On the other hand, the length of a user’s behavior sequence can be extremely long. It is also a big challenge to build a model to handle very long sequences.


We propose a principled way to investigate a new framework of memory network design for user behavior and intention modeling. For instance, a potential framework can employ a public memory for similar users, and each user may have several public memories.


Related Research Topics