Skip to content

Commit

Permalink
feat: add README
Browse files Browse the repository at this point in the history
  • Loading branch information
ethe committed Jul 30, 2024
1 parent a57e97d commit 35c587a
Showing 1 changed file with 117 additions and 0 deletions.
117 changes: 117 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# MorselDB (WIP)

MorselDB is an embedded persistent column-family database. Extreme performance, safety, and scalability — choose any three.

## Features

- [x] Fully asynchronous API support.
- [x] Zero-copy rusty API ensuring safety with compile-time type and lifetime checks.
- [x] Vendor-agnostic:
- [ ] Supports various usage methods, async runtimes, and file systems:
- [x] Rust library:
- [x] Customizable async runtime and file system support.
- [x] Tokio and Tokio fs.
- [ ] Async-std.
- [ ] Python library (via PyO3):
- [ ] asyncio (via [pyo3-asyncio](https://github.com/awestlake87/pyo3-asyncio)).
- [ ] JavaScript library:
- [ ] WASM and OPFS.
- [ ] Dynamic library with a C interface.
- [x] Implements a lightweight and a standard approach to Arrow / Parquet LSM Trees:
- [x] Define column-family using just Arrow schema and store data in Parquet files.
- [x] (Optimistic) Transactions.
- [x] Leveled compaction strategy.
- [x] Push down filter, limit and projection.
- [ ] Runtime schema definition (*in next release*).
- [ ] Support SQL (via [Apache DataFusion](https://datafusion.apache.org/)).
- [ ] Fusion storage across RAM, flash, SSD, and remote Object Storage Service (OSS) for each column-family, balancing performance and cost efficiency per data block:
- [ ] Supports remote storage (via [Arrow object_store](https://github.com/apache/arrow-rs/tree/master/object_store) or [Apache OpenDAL](https://github.com/apache/opendal)).
- [ ] Supports distributed query and compaction.
- [ ] Blob storage (like [BlobDB in RocksDB](https://github.com/facebook/rocksdb/wiki/BlobDB)).

## Example

```rust
use std::ops::Bound;

use futures_util::stream::StreamExt;
use morseldb::{executor::tokio::TokioExecutor, record, Projection, DB};

// use macro to define schema of column family just like ORM
// it provides type safety read & write API
#[record]
pub struct User {
#[primary_key]
name: String,
email: Option<String>,
age: u8,
}

#[tokio::main]
async fn main() {
// pluggable async runtime and I/O
let db = DB::new("./db_path/users".into(), TokioExecutor::default())
.await
.unwrap();

// insert with owned value
db.insert(User {
name: "Alice".into(),
email: Some("[email protected]".into()),
age: 22,
});

{
// morseldb supports transaction
let mut txn = db.transaction().await;

// get from primary key
let name = "Alice".into();

// get the zero-copy reference of record without any allocations.
let user = txn.get(
&name,
// morseldb supports pushing down projection
Projection::All
).await.unwrap();
assert!(user.is_some());
assert_eq!(user.unwrap().get().age, Some(22));

{
let upper = "Blob".into();
// range scan of
let mut scan = txn
.scan((Bound::Included(&name), Bound::Excluded(&upper)))
.await
// morseldb supports pushing down projection
.projection(vec![1])
.take()
.await
.unwrap();
loop {
let user = scan.next().await.transpose().unwrap();
match user {
Some(entry) => {
assert_eq!(
entry.value(),
Some(UserRef {
name: "Alice",
email: Some("[email protected]")
age: Some(22),
})
);
}
None => break,
}
}
}

// commit transaction
txn.commit().await.unwrap();
}
}

```

## Contributing to MorselDB
Please feel free to ask any question or contact us on Github Discussions.

0 comments on commit 35c587a

Please sign in to comment.