Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: updating document in a vector store #194

Open
mhordynski opened this issue Nov 18, 2024 · 0 comments
Open

feat: updating document in a vector store #194

mhordynski opened this issue Nov 18, 2024 · 0 comments
Assignees
Labels
feature New feature or request
Milestone

Comments

@mhordynski
Copy link
Member

Feature description

VectorStore implementations in ragbits should allow for the selective removal of records (based on criteria like metadata) and then enable the injection of new records in their place.

To ensure that this feature is working correctly, an integration test should be created with the following scenario:

  1. Run document search ingestion with mocked providers on two documents, A and B.
  2. Check if the vector store can return elements from both documents.
  3. Introduce a new version of document B with changed content and rerun the ingestion.
  4. Check if document A is still in the store.
  5. Verify that the old content from document B is not returned from the store.
  6. Confirm that the new content from document B is returned from the store.

Motivation

An example scenario when updating a VectorStore is required is in ragbits-document-search. It may happen that document XYZ, which was already ingested into a store, is revised, resulting in a new version being introduced to the system. In this case, the system should look for all records that have the same document_meta.source.id and replace them with new ones from the revised document.

Additional context

No response

@mhordynski mhordynski added the feature New feature or request label Nov 18, 2024
@mhordynski mhordynski self-assigned this Nov 18, 2024
@mhordynski mhordynski moved this to Backlog in ragbits Nov 18, 2024
@mhordynski mhordynski removed their assignment Nov 18, 2024
@mhordynski mhordynski moved this from Backlog to Ready in ragbits Nov 18, 2024
@mhordynski mhordynski modified the milestones: Ragbits 0.6, Ragbits 0.5 Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
Status: In Progress
Development

No branches or pull requests

2 participants