Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(katana-db): database transaction abstractions #2171

Merged
merged 4 commits into from
Jul 11, 2024

Conversation

kariy
Copy link
Member

@kariy kariy commented Jul 10, 2024

Description

Make the katana database generic over the database tx implementations. Doesnt change any database behaviour.

Right now we only define the traits and implement them on the appropriate structs, replacing the current implementations that are just associated methods.

Later, we will use this trait on the provider level so that we can actually be generic over katana persistent database implementation.

Related issue

Tests

  • Yes
  • No, because they aren't needed
  • No, because I need help

Added to documentation?

  • README.md
  • Dojo Book
  • No documentation needed

Checklist

  • I've formatted my code (scripts/prettier.sh, scripts/rust_fmt.sh, scripts/cairo_fmt.sh)
  • I've linted my code (scripts/clippy.sh, scripts/docs.sh)
  • I've commented my code
  • I've requested a review after addressing the comments

Summary by CodeRabbit

  • New Features

    • Introduced traits for advanced database cursor operations, enabling navigation, retrieval, insertion, deletion, and iteration over tables.
    • Added traits for read-only and read-write transactions, enhancing database manipulation capabilities.
  • Refactor

    • Consolidated cursor and transaction implementations using new abstraction traits for improved modularity and reusability.
  • Chores

    • Updated imports in several modules to use the newly introduced database abstraction traits.

Copy link

coderabbitai bot commented Jul 10, 2024

Walkthrough

The updates introduce traits for database cursor operations and transactions across multiple files in the katana project. These traits include read-only and read-write cursors, DUPSORT cursors, and transaction operations like insertion and deletion. The changes also involve refactoring existing code to implement these new traits, enhancing modularity and abstraction in database interactions.

Changes

File/Path Change Summary
crates/katana/storage/db/src/abstraction/cursor.rs Introduced traits for cursor operations including navigation, retrieval, insertion, deletion, and iteration.
crates/katana/storage/db/src/abstraction/mod.rs Added modules for cursor and transaction, re-exporting their functionality.
crates/katana/storage/db/src/abstraction/transaction.rs Introduced traits for read-only and read-write transactions, with methods for cursor creation and data manipulation.
crates/katana/storage/db/src/lib.rs Added the abstraction module to the storage/db crate.
crates/katana/storage/db/src/mdbx/cursor.rs Refactored cursor implementation to conform to new traits, updating method signatures and implementations.
crates/katana/storage/db/src/mdbx/mod.rs Added and updated imports to use the new database abstraction traits.
crates/katana/storage/provider/src/providers/db/mod.rs Introduced new imports for database abstractions.
crates/katana/storage/provider/src/providers/db/state.rs Added imports for database abstraction traits from katana_db.

Poem

In the heart of Katana's code,
New cursor traits now boldly strode.
They seek, they set, they navigate,
In tables full of data's weight.
Transactions too, with deftly hand,
Insert, delete, at their command.
Code refined, we now extol,
The rabbit's work to make it whole.

Tip

CodeRabbit can approve the review once CodeRabbit's comments are resolved

Enable the reviews.request_changes_workflow setting in your project's settings in CodeRabbit to approve the review once CodeRabbit's comments are resolved.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@kariy kariy force-pushed the refactor/db-abstraction branch 2 times, most recently from 3f645f6 to 9b8449d Compare July 10, 2024 18:59
@kariy kariy force-pushed the refactor/db-abstraction branch from 9b8449d to adb3af4 Compare July 10, 2024 19:00
Copy link

codecov bot commented Jul 10, 2024

Codecov Report

Attention: Patch coverage is 55.71429% with 31 lines in your changes missing coverage. Please review.

Project coverage is 68.06%. Comparing base (bbb355e) to head (4fdec56).
Report is 4 commits behind head on main.

Files Patch % Lines
crates/katana/storage/db/src/mdbx/tx.rs 36.66% 19 Missing ⚠️
crates/katana/storage/db/src/abstraction/cursor.rs 70.00% 6 Missing ⚠️
crates/katana/storage/db/src/mdbx/cursor.rs 70.00% 6 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2171      +/-   ##
==========================================
+ Coverage   68.00%   68.06%   +0.06%     
==========================================
  Files         331      332       +1     
  Lines       42697    42687      -10     
==========================================
+ Hits        29035    29057      +22     
+ Misses      13662    13630      -32     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 4b69971 and 4fdec56.

Files selected for processing (9)
  • crates/katana/storage/db/src/abstraction/cursor.rs (1 hunks)
  • crates/katana/storage/db/src/abstraction/mod.rs (1 hunks)
  • crates/katana/storage/db/src/abstraction/transaction.rs (1 hunks)
  • crates/katana/storage/db/src/lib.rs (1 hunks)
  • crates/katana/storage/db/src/mdbx/cursor.rs (9 hunks)
  • crates/katana/storage/db/src/mdbx/mod.rs (2 hunks)
  • crates/katana/storage/db/src/mdbx/tx.rs (5 hunks)
  • crates/katana/storage/provider/src/providers/db/mod.rs (1 hunks)
  • crates/katana/storage/provider/src/providers/db/state.rs (1 hunks)
Files skipped from review due to trivial changes (1)
  • crates/katana/storage/db/src/abstraction/mod.rs
Additional comments not posted (33)
crates/katana/storage/db/src/abstraction/transaction.rs (2)

6-33: DbTx Trait: Ensure Consistent Error Handling.

The DbTx trait defines methods for managing read-only transactions. Ensure that all methods handle errors consistently and consider adding comments to describe the purpose of each method.


35-68: DbTxMut Trait: Ensure Consistent Error Handling

The DbTxMut trait defines methods for managing read-write transactions. Ensure that all methods handle errors consistently and consider adding comments to describe the purpose of each method.

crates/katana/storage/db/src/lib.rs (1)

8-8: Verify Integration of Abstraction Module

The new abstraction module is integrated into the storage/db crate. Ensure that the integration follows the project's modular design and verify that all necessary components are included.

crates/katana/storage/db/src/mdbx/tx.rs (2)

Line range hint 52-92:
DbTx Implementation: Ensure Correctness of Database Operations

The DbTx trait is implemented for the Tx struct. Ensure the correctness of the database operations and verify that all error handling is consistent and appropriate.


Line range hint 95-127:
DbTxMut Implementation: Ensure Correctness of Database Operations

The DbTxMut trait is implemented for the Tx struct. Ensure the correctness of the database operations and verify that all error handling is consistent and appropriate.

crates/katana/storage/db/src/mdbx/cursor.rs (4)

38-71: DbCursor Implementation: Ensure Correctness of Cursor Operations

The DbCursor trait is implemented for the Cursor struct. Ensure the correctness of cursor operations and verify that all error handling is consistent and appropriate.


Line range hint 85-124:
DbDupSortCursor Implementation: Ensure Correctness of Cursor Operations

The DbDupSortCursor trait is implemented for the Cursor struct. Ensure the correctness of cursor operations and verify that all error handling is consistent and appropriate.


Line range hint 169-214:
DbCursorMut Implementation: Ensure Correctness of Cursor Operations

The DbCursorMut trait is implemented for the Cursor struct. Ensure the correctness of cursor operations and verify that all error handling is consistent and appropriate.


219-228: DbDupSortCursorMut Implementation: Ensure Correctness of Cursor Operations

The DbDupSortCursorMut trait is implemented for the Cursor struct. Ensure the correctness of cursor operations and verify that all error handling is consistent and appropriate.

crates/katana/storage/db/src/abstraction/cursor.rs (6)

6-39: Ensure consistent error handling in cursor methods.

While the DbCursor trait methods' signatures look correct, ensure that the implementations of these methods consistently handle errors and edge cases.


42-57: Verify transaction integrity in cursor mutation methods.

The methods in DbCursorMut involve mutations in the database. Ensure that the implementations of these methods maintain transaction integrity and handle rollbacks correctly in case of errors.


60-91: Check for consistent handling of duplicates in DUPSORT cursor methods.

The DbDupSortCursor trait methods should ensure consistent handling of duplicate keys and values. Verify that the implementations handle edge cases and maintain cursor state correctly.


94-100: Ensure atomicity in DUPSORT cursor mutation methods.

The methods in DbDupSortCursorMut involve complex operations on DUPSORT tables. Ensure that the implementations maintain atomicity and handle rollbacks correctly in case of errors.


110-147: Confirm iterator correctness in Walker.

The Walker struct methods should ensure correct iteration over table items, especially handling the initial cursor position and edge cases.


154-195: Verify duplicate handling in DupWalker.

The DupWalker struct methods should ensure correct iteration over DUPSORT table items, especially handling duplicates and edge cases.

crates/katana/storage/provider/src/providers/db/state.rs (4)

Line range hint 15-55: Ensure proper error handling in state update methods.

The methods in the StateWriter trait implementation should properly handle errors and ensure database consistency.


Line range hint 56-85: Ensure proper error handling in contract class update methods.

The methods in the ContractClassWriter trait implementation should properly handle errors and ensure database consistency.


Line range hint 86-111: Ensure proper error handling in contract class retrieval methods.

The methods in the ContractClassProvider trait implementation should properly handle errors and ensure correct retrieval of contract class information.


Line range hint 112-251: Ensure proper error handling in historical state retrieval methods.

The methods in the StateProvider trait implementation should properly handle errors and ensure correct retrieval of historical state information.

crates/katana/storage/db/src/mdbx/mod.rs (4)

Line range hint 17-37: Ensure correct database environment setup.

The DbEnv::open function should correctly set up the database environment, handling errors and edge cases.


Line range hint 39-53: Ensure correct table creation.

The DbEnv::create_tables function should correctly create all defined tables, handling errors and ensuring consistency.


Line range hint 69-77: Ensure transaction integrity in DbEnv::update.

The DbEnv::update function should maintain transaction integrity, handling rollbacks correctly in case of errors.


Line range hint 147-389: Ensure comprehensive test coverage.

The tests should comprehensively cover all database operations, including edge cases and error handling.

crates/katana/storage/provider/src/providers/db/mod.rs (10)

Line range hint 22-40: Ensure proper error handling in state factory methods.

The methods in the StateFactoryProvider trait implementation should properly handle errors and ensure correct state information retrieval.


Line range hint 42-56: Ensure proper error handling in block number methods.

The methods in the BlockNumberProvider trait implementation should properly handle errors and ensure correct block number retrieval.


Line range hint 58-72: Ensure proper error handling in block hash methods.

The methods in the BlockHashProvider trait implementation should properly handle errors and ensure correct block hash retrieval.


Line range hint 74-93: Ensure proper error handling in header methods.

The methods in the HeaderProvider trait implementation should properly handle errors and ensure correct block header retrieval.


Line range hint 95-140: Ensure proper error handling in block methods.

The methods in the BlockProvider trait implementation should properly handle errors and ensure correct block information retrieval.


Line range hint 142-161: Ensure proper error handling in block status methods.

The methods in the BlockStatusProvider trait implementation should properly handle errors and ensure correct block status retrieval.


Line range hint 163-180: Ensure proper error handling in state root methods.

The methods in the StateRootProvider trait implementation should properly handle errors and ensure correct state root retrieval.


Line range hint 182-247: Ensure proper error handling in state update methods.

The methods in the StateUpdateProvider trait implementation should properly handle errors and ensure correct state update retrieval.


Line range hint 249-319: Ensure proper error handling in transaction methods.

The methods in the TransactionProvider trait implementation should properly handle errors and ensure correct transaction information retrieval.


Line range hint 387-493: Ensure proper error handling in block writing methods.

The methods in the BlockWriter trait implementation should properly handle errors and ensure correct block information writing.

@kariy kariy merged commit f8776ba into main Jul 11, 2024
14 of 15 checks passed
@kariy kariy deleted the refactor/db-abstraction branch July 11, 2024 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant