You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
VectorStore implementations in ragbits should allow for the selective removal of records (based on criteria like metadata) and then enable the injection of new records in their place.
To ensure that this feature is working correctly, an integration test should be created with the following scenario:
Run document search ingestion with mocked providers on two documents, A and B.
Check if the vector store can return elements from both documents.
Introduce a new version of document B with changed content and rerun the ingestion.
Check if document A is still in the store.
Verify that the old content from document B is not returned from the store.
Confirm that the new content from document B is returned from the store.
Motivation
An example scenario when updating a VectorStore is required is in ragbits-document-search. It may happen that document XYZ, which was already ingested into a store, is revised, resulting in a new version being introduced to the system. In this case, the system should look for all records that have the same document_meta.source.id and replace them with new ones from the revised document.
Additional context
No response
The text was updated successfully, but these errors were encountered:
Feature description
VectorStore
implementations in ragbits should allow for the selective removal of records (based on criteria like metadata) and then enable the injection of new records in their place.To ensure that this feature is working correctly, an integration test should be created with the following scenario:
A
andB
.B
with changed content and rerun the ingestion.A
is still in the store.B
is not returned from the store.B
is returned from the store.Motivation
An example scenario when updating a VectorStore is required is in
ragbits-document-search
. It may happen that documentXYZ
, which was already ingested into a store, is revised, resulting in a new version being introduced to the system. In this case, the system should look for all records that have the samedocument_meta.source.id
and replace them with new ones from the revised document.Additional context
No response
The text was updated successfully, but these errors were encountered: