Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store columns Fate needs to find work in a locality group #4977

Open
keith-turner opened this issue Oct 12, 2024 · 1 comment · May be fixed by #5031
Open

Store columns Fate needs to find work in a locality group #4977

keith-turner opened this issue Oct 12, 2024 · 1 comment · May be fixed by #5031
Assignees
Labels
enhancement This issue describes a new feature, improvement, or optimization.
Milestone

Comments

@keith-turner
Copy link
Contributor

keith-turner commented Oct 12, 2024

Is your feature request related to a problem? Please describe.

Fate continually scans the accumulo.fate table looking for new work. This scan only reads a small amount of data per fate transaction, but needs to filter out all of the other data. If the data needed to find work was stored in its own locality group then this would avoid filtering out data. Using a locality group would also help with cache utilization in the tablet server as only the small amount of data is read when fate does a full table scan looking for work.

Describe the solution you'd like

These two columns are read when fate looks for work. Need to refactor these columns to be in their own column family and then configure the accumulo initialization code to setup the locality group config when creating the table.

@keith-turner keith-turner added the enhancement This issue describes a new feature, improvement, or optimization. label Oct 12, 2024
@keith-turner keith-turner added this to the 4.0.0 milestone Oct 12, 2024
@ArbaazKhan1
Copy link
Contributor

I can take a look at this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement This issue describes a new feature, improvement, or optimization.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants