Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataframe v2: "constant-time" pagination #7665

Merged
merged 4 commits into from
Oct 10, 2024

Conversation

teh-cmc
Copy link
Member

@teh-cmc teh-cmc commented Oct 9, 2024

This implements support for "constant-time" pagination.
Obviously it's not constant-time, but it scales in a sane fashion.

This PR is still not about general performance optimizations. It is the last step before those can start though.


Test: check the performance while scrolled down at the bottom of a large dataframe.

Previous branch:
image

This branch:
image

Checklist

  • I have read and agree to Contributor Guide and the Code of Conduct
  • I've included a screenshot or gif (if applicable)
  • I have tested the web demo (if applicable):
  • The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG
  • If applicable, add a new check to the release checklist!
  • If have noted any breaking changes to the log API in CHANGELOG.md and the migration guide

To run all checks from main, comment on the PR with @rerun-bot full-check.

@teh-cmc teh-cmc added 🔍 re_query affects re_query itself 🚀 performance Optimization, memory use, etc do-not-merge Do not merge this PR exclude from changelog PRs with this won't show up in CHANGELOG.md feat-dataframe-view Everything related to the dataframe view feat-dataframe-api Everything related to the dataframe API labels Oct 9, 2024
@teh-cmc teh-cmc marked this pull request as ready for review October 9, 2024 16:16
@teh-cmc teh-cmc force-pushed the cmc/dataframev2_clears branch 2 times, most recently from 5f92358 to 96dbdac Compare October 10, 2024 07:21
Base automatically changed from cmc/dataframev2_clears to main October 10, 2024 07:22
@teh-cmc teh-cmc force-pushed the cmc/dataframev2_constant_time_paging branch from fa88da2 to ac1bf1a Compare October 10, 2024 07:23
@teh-cmc teh-cmc removed the do-not-merge Do not merge this PR label Oct 10, 2024
@teh-cmc teh-cmc merged commit bb58101 into main Oct 10, 2024
8 of 13 checks passed
@teh-cmc teh-cmc deleted the cmc/dataframev2_constant_time_paging branch October 10, 2024 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
exclude from changelog PRs with this won't show up in CHANGELOG.md feat-dataframe-api Everything related to the dataframe API feat-dataframe-view Everything related to the dataframe view 🚀 performance Optimization, memory use, etc 🔍 re_query affects re_query itself
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Dataframe v2: constant-time pagination
2 participants