Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lazy loading partial index? #511

Open
chrisspen opened this issue Apr 1, 2022 · 0 comments
Open

Lazy loading partial index? #511

chrisspen opened this issue Apr 1, 2022 · 0 comments

Comments

@chrisspen
Copy link

How difficult would it be to modify lunr.js to lazy load a partial index?

For a large index covering hundreds of thousands of large documents, I've found it's impractical, and usually unnecessary, to load the entire index on the client side.

Inspecting the code, it looks like the two main data-structures in the index are fieldVectors and invertedIndex, which are both keyed by search term and document ID respectively.

Instead of requiring those entire data structures to be loaded, would it be possible to shard them, based on term, and then load the shards as needed during a query? This could drastically reduce index loading time as well as help lunr scale over larger collections of documents.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant