-
Notifications
You must be signed in to change notification settings - Fork 135
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add expand_nested_docs Parameter support to NMSLIB engine #2331
Conversation
Can you explain a bit more on this? |
For nmslib, searches are performed at the nested document level without deduplicating per parent documents. As a result, multiple inner hits can occur as long as the nested documents belong to the top-K results. In this context, the demand for this functionality may be lower compared to other engines. For instance, consider an extreme scenario where there are two documents, each containing 10 nested documents. If a query is executed with k=5, it is possible that all five closest vectors come from just one of the two documents. In this case, expanding nested documents would include five additional nested documents with scores lower than the minimum score of the already retrieved nested documents. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add/update the IT of expand nested docs variable for nmslib engine too.
Added. |
8868d00
to
f298037
Compare
0e8326d
to
1b8e8fb
Compare
Signed-off-by: Heemin Kim <[email protected]>
The backport to
To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-2331-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 a5fb171b065747a39e7aaae3de330c0fda0800ca
# Push it to GitHub
git push --set-upstream origin backport/backport-2331-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x Then, create a pull request where the |
Signed-off-by: Heemin Kim <[email protected]>
Signed-off-by: Heemin Kim <[email protected]>
…oject#2331) Signed-off-by: Heemin Kim <[email protected]>
Description
Add support for the
expand_nested_docs
parameter in the nmslib engine. As nmslib does not support multi-vector functionality, this feature may have limited usefulness but poses no downside to including it.Related Issues
N/A
Check List
--signoff
.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.