Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

redb support #4

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

redb support #4

wants to merge 1 commit into from

Conversation

cryptoquick
Copy link

@casey gave me the advice that redb performs better than sled. In the hopes that it's more stable and user-friendly than sled, I investigated adding redb support to stored, and things initially went pretty well for a bit, but then I ran into a few lifetimes errors I couldn't think of a good way to resolve.

@dr-orlovsky, would you be interested in something like this? If so, I'm not sure if there's a good way to resolve the lifetimes issues, but maybe @casey might have some ideas. If we can get redb support, maybe it'll resolve the issue I reported in #3, by, uh, supporting something other than sled.

@casey
Copy link

casey commented Sep 14, 2022

Can you post some of the errors? Definitely note that redb is not yet stable. @cberner (the author) is working on stabilizing the file format, but you should expect at least a few breaking changes until then.

@cryptoquick
Copy link
Author

@casey There were so many errors that I just got overwhelmed. They'd also make more sense if you checked out the repo, since they'd be in context. I did reference some of your code in ord as an example of how to use redb, that was helpful. Although the implementation and use-cases are similar, there are definitely some major differences between the sled and redb interfaces that make switching between one and the other more difficult. Being explicit about when writes are to occur, the different datatypes supported, and the <k', v'> database lifetimes make the transition more difficult.

Also, I think stored is best relied on as a cache and not a more permanent contract storage solution, so the file format isn't a big deal-breaker for us, but it certainly would be for those already using stored. I don't think it should default to redb, I only did that for convenience in development. It would be very helpful for my use-case, however, since sled just won't quit.

@casey
Copy link

casey commented Sep 14, 2022

I'm a little backed up, so I probably won't have time to check out the repo to try out for a bit. Maybe start with code that produces a single error, post that, and then work from there?

@cryptoquick
Copy link
Author

Totally fine, I'm also investigating alternative approaches

@dr-orlovsky
Copy link
Member

I think these experiments are cool and should be continued. Unfortunately due to a big time problems on my side I can't really participate them - but would be happy to merge once a better solution than sled will be found and working.

@cryptoquick
Copy link
Author

@dr-orlovsky Awesome! I'm also working on a new, similar project, if you find it valuable. It came to mind because changing database formats could be a problem; it might be nice to have a flat file backup format for replication and checkpointing. I'm still working out a few details, but overall it mostly seems to work. Check it out:
https://github.com/diba-io/carbonado

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants