Title: Innovative Design of Data Structures and Storage Mechanisms for In-Memory Databases
Technical Area: Database
In-memory databases, such as SAP HANA, can provide high performance and throughput and have been widely deployed in such environments. Typically, deployment cost is driven by local memory size, which becomes a limiting factor.
With more local memory becoming available and emerging hardware such as 3DXPoint and AEP, servers with super large memory capacity (more than 1TB) are commonplace. Therefore, the utility of in-memory database is about to undergo rapid growth.
Currently, most databases are designed for traditional hardware such as SSD and HDD. For instance, their sub-modules such as logging, index and cache have not been optimized according to the access features of memory, such as how to exploit ten-nanosecond level access speed or how to take advantage of byte- addressable capability.
For in-memory databases such as Redis, MemSQL, VoltDB, and so on, there are also many challenges such as how to efficiently support data compression, data indexing and loading, and data persistence in the scenario of large memory capacity or hybrid memory systems. To tackle these issues, innovative design of data structures and storage mechanisms is needed.
The main target is to exploit data structures and storage mechanisms suitable for high performance in-memory database, especially for the hybrid memory environment. The measurable outputs of the research may include but are not limited to:
- Developed techniques and methodologies and related system prototypes for hybrid in-memory database (more than 1TB).
- Storage engine that supports high throughput (ten million QPS or more).
- Research works will be published in top conferences (CCF-A Level preferred).
Related Research Topics
- DRAM/NVM/SSD hybrid memory oriented optimization mechanisms for key modules of in-memory database, such as logging and index structures.
- High performance storage engine for very large in-memory database providing hybrid storage allocation and memory-type-aware data layout management.
- New compression algorithms that improve memory space utilization as well as support efficient database indexing.