Tonbo release 0.2.0

Tonbo Now Supports S3 as Remote Storage

With the integration of Fusio, Tonbo now supports storing data on S3, making cloud storage integration easier and reducing costs. Users can store either parts of stale data or the full dataset on S3. We believe object storage is key for the next generation of data systems, and supporting S3 is crucial for Tonbo's goal of providing a unified storage solution for both local and cloud environments, enabling databases like SQLite and PostgreSQL to act as stateless query engines on top of it. Configuration example:

let options = DbOption::from(Path::from_filesystem_path("./db_path/users").unwrap())
    // tonbo only stores 3 and beyond level stale data of lsm tree on S3
    // 1 and 2 level fresh data are on local disk as default
    .level_path(
        3,
        "/remote-storage".into(),
        FsOptions::S3 {
            bucket: "bucket".into(),
            credential: None,
            region: None,
            sign_payload: None,
            checksum: None,
        },
    )
    .unwrap();
let db = DB::new(options, TokioExecutor::default()).await.unwrap();

Python Binding Now Supported

Starting from version 0.2.0, Tonbo can be used in Python with minimal setup. Given Python's wide adoption in scientific computing, AI, and data analysis, this support helps developers easily build data-intensive applications for local/edge-first environments.

The Python binding offers an ORM-like experience, simplifying data modeling and reducing boilerplate code, allowing developers to focus on building features. Quick preview:

from tonbo import DbOption, Column, DataType, Record, TonboDB, Bound
import asyncio
tempfile

@Record
class User:
    id = Column(DataType.Int64, name="id", primary_key=True)
    age = Column(DataType.Int16, name="age", nullable=True)
    name = Column(DataType.String, name="name", nullable=False)
    email = Column(DataType.String, name="email", nullable=True)
    data = Column(DataType.Bytes, name="data", nullable=True)


async def main():
    temp_dir = tempfile.TemporaryDirectory()

    db = TonboDB(DbOption(temp_dir.name), User())
    await db.insert(User(id=18, age=175, name="Alice"))

    record = await db.get(18)
    assert record == {
        "id": 18,
        "age": 175,
        "name": "Alice",
        "email": None,
        "data": None,
    }

    txn = await db.transaction()

    txn.insert(
        User(
            id=19,
            age=195,
            name="Bob",
            data=b"Hello Tonbo!",
            email="[email protected]",
        )
    )

    await txn.commit()
    txn = await db.transaction()
    scan = await txn.scan(
        Bound.Excluded(18),
        None,
        limit=100,
        projection=["id", "email", "data"],
    )
    async for record in scan:
        assert record["age"] is None
        print(record)
    await txn.commit()


asyncio.run(main())