Topic Title: Wearable TmallGenie



Technical Area:

Deep Learning

Multi-Agent Coordination



TmallGenie consists of a cylindrical body with omni-directional speakers and an LED light ring at the bottom of the device. As with other smart speakers, the Genie supports web searches, music streaming, control home automation devices, and order products from Tmall. Voice interaction with the device is currently only available in Mandarin. However, the current TmallGenie products are all designed for home scenarios with no battery. Making TmallGenie a wearable device can result in TMallGenie being able to operate in a wider range of application scenarios as a mobile personal assistant. The biggest challenge is that many of the current algorithms, that are running on the cloud, may need to be ported to the device so that TmallGenie can still be operational under bad network condition. As the wearable TmallGenie is just a mobile client, it could be an extension to the desktop TmallGenie which may have different skill set that are complementary to the mobile one. In home scenarios, the different agents, desktop and mobile, are supposed to work together on decision making and (sensor) data sharing so that they can serve the users better as a team.



  1. We are going to design light-weight deep learning neural networks for porting algorithms from cloud to device. Algorithm optimization focuses on better energy consumption and minimum performance drawback.
  2. We are going to research on the multi-agent coordination strategies for making different devices working together. In order to achieve effective interactions with the users ,AI capabilities need to be coordinated to be deployed for optimal effect under different situations.



Related Research Topics

  1. Light-weight Deep Learning . Deep learning is a useful technique for natural language processing tasks which are central to the operation of TmallGenie. However, when miniaturized, we need to ensure that deep learning algorithms can be executed in devices with limited computational resources. In this research, we focus on designing light-weight deep learning.
  2. Multi-agent Coordination. In order to serve better interactions with the users, a multi-agent coordination algorithm are required for choosing proper skills from different devices to balance the use of novelty, uncertainty and complexity to achieve high level of user adherence.