I am not able to get answers when i use open AI model to retrieve the documents

[VikRathod]

This is the m the code , when i run the then i ma getting error . I have inserted at below

#from sentence_transformers import SentenceTransformer
if os.path.exists("faiss_document_store.db"):
os.remove("faiss_document_store.db")

model = SentenceTransformer('sentence-transformers/msmarco-distilbert-base-tas-b')
#print(model)
max_seq_length = model.max_seq_length
embedding_dim = model.get_sentence_embedding_dimension()

#Init document store
document_store = FAISSDocumentStore(embedding_dim=768,
faiss_index_factory_str="Flat",
similarity="cosine")

Update document store with contents having more than 20 words

document_store.write_documents([{"content": t, } for t in df.QA_pair.tolist()])

document_store.write_documents([{
"content": r['QA_pair'],
"metadata": {"Category": r['Category']}}
for i, r in df.iterrows()])

from haystack.nodes import EmbeddingRetriever

OpenAI EmbeddingRetriever

retriever = EmbeddingRetriever(
document_store=document_store,
batch_size=8,
embedding_model="text-embedding-ada-002",
api_key="FAKE-API-KEY",
max_seq_len=1536
)
from haystack.pipelines import DocumentSearchPipeline
pipeline = DocumentSearchPipeline(retriever=retriever)
pipeline
questions = ["What registration documents are required for French citizens?",
"I do not have an account with Bank of Mauritius. Can I still use the app?",
"Can I pay utility bills with the app?",
"Which plan should I choose if I need minimum 1GB data per day?"]

for i in questions:
result = pipeline.run(query=i)
print(">>>>> ",result)

ERROR:

AssertionError Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/haystack/pipelines/base.py in run(self, query, file_paths, labels, documents, meta, params, debug)
555 start = time()
--> 556 node_output, stream_id = self._run_node(node_id, node_input)
557 if "_debug" in node_output and node_id in node_output["_debug"]:

11 frames
AssertionError:

The above exception was the direct cause of the following exception:

Exception Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/haystack/pipelines/base.py in run(self, query, file_paths, labels, documents, meta, params, debug)
561 # If you really want to see it, raise the log level.
562 logger.debug("Exception while running node '%s' with input %s", node_id, node_input)
--> 563 raise Exception(
564 f"Exception while running node '{node_id}': {e}\nEnable debug logging to see the data that was passed when the pipeline failed."
565 ) from e

Exception: Exception while running node 'Retriever':
Enable debug logging to see the data that was passed when the pipeline failed.

[bogdankostic]

Hi @VikRathod! It's quite hard for me to understand your code snippet due to formatting, could you send another comment with it making sure that it's formatted correctly? Also, could please share the full error traceback?

[VikRathod]

This is the m the code , when i run the then i ma getting error . I have inserted at below

from sentence_transformers import SentenceTransformer
if os.path.exists("faiss_document_store.db"):
os.remove("faiss_document_store.db")

model = SentenceTransformer('sentence-transformers/msmarco-distilbert-base-tas-b')
print(model)
max_seq_length = model.max_seq_length
embedding_dim = model.get_sentence_embedding_dimension()

document_store = FAISSDocumentStore(embedding_dim=768,
faiss_index_factory_str="Flat",
similarity="cosine")
document_store.write_documents([{
"content": r['QA_pair'],
"metadata": {"Category": r['Category']}}
for i, r in df.iterrows()])

from haystack.nodes import EmbeddingRetriever

retriever = EmbeddingRetriever(
document_store=document_store,
batch_size=8,
embedding_model="text-embedding-ada-002",
api_key="FAKE-API-KEY",
max_seq_len=1536
)
from haystack.pipelines import DocumentSearchPipeline
pipeline = DocumentSearchPipeline(retriever=retriever)
pipeline
questions = ["What registration documents are required for French citizens?",
"I do not have an account with Bank of Mauritius. Can I still use the app?",
"Can I pay utility bills with the app?",
"Which plan should I choose if I need minimum 1GB data per day?"]

for i in questions:
result = pipeline.run(query=i)
print(">>>>> ",result)

ERROR is :

AssertionError Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/haystack/pipelines/base.py in run(self, query, file_paths, labels, documents, meta, params, debug)
555 start = time()
--> 556 node_output, stream_id = self._run_node(node_id, node_input)
557 if "_debug" in node_output and node_id in node_output["_debug"]:

11 frames
AssertionError:

The above exception was the direct cause of the following exception:

Exception Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/haystack/pipelines/base.py in run(self, query, file_paths, labels, documents, meta, params, debug)
561 # If you really want to see it, raise the log level.
562 logger.debug("Exception while running node '%s' with input %s", node_id, node_input)
--> 563 raise Exception(
564 f"Exception while running node '{node_id}': {e}\nEnable debug logging to see the data that was passed when the pipeline failed."
565 ) from e

Exception: Exception while running node 'Retriever':
Enable debug logging to see the data that was passed when the pipeline failed.