You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
We are finding that the SentenceWindowRetriever is a powerful RAG tool for some client projects. We are especially testing it out in with some early prospects. While using it @JasperLS and I identified some aspects that could make it easier to use.
Describe the solution you'd like
Ensure that the documents in context_documents are sorted by split_idx_start. We noticed that the documents are only sorted when merging them into one text blob in the merge_documents_text function.
Would it be possible to also export a list of merged documents. So have an output with type List[Document]. This would make it easier to use in a downstream prompt builder since we typically want to also use the metadata of the resulting merged Document.
I realize for the second request it is possible to use a nested loop to overcome this in a Prompt so I wouldn't say it's a hard requirement, but it would be more convenient than working with a List[List[Documents]]
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
We are finding that the
SentenceWindowRetriever
is a powerful RAG tool for some client projects. We are especially testing it out in with some early prospects. While using it @JasperLS and I identified some aspects that could make it easier to use.Describe the solution you'd like
context_documents
are sorted bysplit_idx_start
. We noticed that the documents are only sorted when merging them into one text blob in themerge_documents_text
function.I realize for the second request it is possible to use a nested loop to overcome this in a Prompt so I wouldn't say it's a hard requirement, but it would be more convenient than working with a List[List[Documents]]
The text was updated successfully, but these errors were encountered: