Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(mito): Add writable flag to region #2349

Merged
merged 17 commits into from
Sep 14, 2023

Conversation

evenyag
Copy link
Contributor

@evenyag evenyag commented Sep 9, 2023

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

This PR adds a writable flag to each mito region and a set_writable() method to RegionEngine trait

  • The engine opens regions in readonly mode
  • To write a region, users need to call set_writable() to mark the region as writable

Note that the writable flag is a simple atomic flag, so it can't guarantee that the engine immediately cancels all running write operations.

Refactor

  • It wraps Sender<Result<Output>> and Option<Sender<Result<Output>>> into new types OutputTx and OptionOutputtx.
  • The region map provides two methods get_region_or() and writable_region_or(). They invoke the OnFailure callback if the region is unavailable.

The we can refactor the following code:

let region = match self.regions.get_writable_region(region_id) {
    Ok(v) => v,
    Err(e) => {
        request.on_failure(e);
        return;
    }
};

With writable_region_or()

let Some(region) = self.regions.writable_region_or(region_id, &mut request) else {
    return;
};

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.

Refer to a related PR or issue link (optional)

@evenyag evenyag force-pushed the feat/mito2-readonly branch 3 times, most recently from 931d4cb to 0cecae3 Compare September 12, 2023 09:50
@waynexia waynexia force-pushed the migrate-region-server branch from 50cea29 to f1f7824 Compare September 12, 2023 12:23
@evenyag evenyag force-pushed the feat/mito2-readonly branch from 0cecae3 to 94d1355 Compare September 12, 2023 13:14
@evenyag evenyag changed the base branch from migrate-region-server to develop September 12, 2023 13:14
@evenyag evenyag marked this pull request as ready for review September 13, 2023 03:07
@codecov
Copy link

codecov bot commented Sep 13, 2023

Codecov Report

Merging #2349 (c892de7) into develop (93f3048) will decrease coverage by 0.02%.
The diff coverage is 81.42%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #2349      +/-   ##
===========================================
- Coverage    77.49%   77.48%   -0.02%     
===========================================
  Files          716      716              
  Lines       113735   113827      +92     
===========================================
+ Hits         88141    88197      +56     
- Misses       25594    25630      +36     

@fengjiachun
Copy link
Collaborator

Is handle_alter also need to get writeable_region?

Copy link
Contributor

@killme2008 killme2008 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@fengjiachun fengjiachun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@fengjiachun fengjiachun added this pull request to the merge queue Sep 14, 2023
Merged via the queue into GreptimeTeam:develop with commit d4ae8a6 Sep 14, 2023
paomian pushed a commit to paomian/greptimedb that referenced this pull request Oct 19, 2023
* feat: add writable flag to region.

* refactor: rename MitoEngine to MitoEngine::scanner

* feat: add set_writable() to RegionEngine

* feat: check whether region is writable

* feat: make set_writable sync

* test: test set_writable

* docs: update comments

* feat: send result on compaction failure

* refactor: wrap output sender in new type

* feat: on failure

* refactor: use get_region_or/writable_region_or

* refactor: remove send_result

* feat: notify waiters on flush scheduler drop

* test: fix tests

* fix: only alter writable region
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants