Developed by the operating systems group of the department of computer science of the Heinrich-Heine-University Düsseldorf, DXRAM is a distributed in-memory key-value store for low-latency cloud applications, e.g. social networks, search engines and I/O-bound long running scientific computations running in a data center. It is designed to efficiently manage billions of small data objects which are typical for graph based applications.
For example, Facebook runs approx. 1,000 memcached servers keeping around 75% of its data always in RAM because back-end database accesses are too slow. We argue that it is possible to go one step further by keeping all data always in RAM relieving developers from cache management and synchronization of caches with secondary storage.
DXRAM is a research project that's under development. We do not recommend using the system in a production environment or with production data without having an external backup. Expect to encounter bugs. However, we are looking forward to bug reports and code contributions.
- Distributed object lookup and scalable meta-data management with super-peer overlay
- Optimized for up to 10 GBit/s Ethernet network and up to 56 GBit/s InfiniBand
- All objects always in RAM
- Custom, highly efficient and performant memory management for billions of small objects (< 64 bytes) per node
- Smart asynchronous logging to disk (optimized for SSD) for persistency
- Replication of logs to remote nodes for fault tolerance
- Crash-recovery failure-model
- Object migrations for storage nodes to handle hot spots
- Run computations on storage nodes using lightweight jobs or tasks submitted to coordinated compute groups
- Quick start guide: Setup and run DXRAM instances using our deploy scripts
- Using the DXRAM terminal: Your first steps to get familiar with the environment
- Run applications on DXRAM: Run existing applications or develop your own
- Development: DXRAM and applications: How to develop for the DXRAM core and applications running on DXRAM
- Troubleshooting
- DXRAM Architecture
- Publications involving DXRAM
- Development setup: Setup environment for DXRAM development
Copyright (C) 2018 Heinrich-Heine-Universitaet Duesseldorf, Institute of Computer Science, Department Operating Systems. Licensed under the GNU General Public License.