running QA Agent with local LLm

[SggTio]

Hi.

I'm having some troubles using an implementation for a QA Agent in spanish with a local model.
Due to enterprise reasons, I should not use huggingface but a local instance.
I downloaded my models but have been unable to use them.

My concern goes as follows:
I divided my process in 3 stages:

1. Cleaning my files and uploading them (To a InMemoryStore because this is a PoC).
2. Using the Pipeline method to create the QA Agent in spanish. I can create the document store with the files and parse them using the bm25 that does not use embeddings. When using embeddings, I have problems: The LocalHuggingFaceGenerator seems restrictive and does not support spanish and also, the query embedder and the documents embedder seems to be acting strange. Can you point out to an example of this?
3. Finally, when asking questions and using the filters, the answer seems quite off ocmpared to the documents (this might be related to me being unable to use a spanish model or to the problem regarding the embeddings).

if you can point out any directions, comments or recommendations, I would really appreciate it.

[Amnah199]

Hi @SggTio, in queries like yours its helpful if you could share your pipeline code, along with details such as your Haystack version, operating system, and any error messages you're encountering. This would give more context for troubleshooting.

Regarding your second point, if you're not passing a specific model to the LocalHuggingFaceGenerator, it defaults to using flan-t5-base, which supports multiple languages, including Spanish, so ideally this shouldn't be the root of the issue.
Could you clarify the second and third points a bit more? They are currently quite vague.

In the meantime, here are some resources on using embedders in QA pipelines that might be helpful:

Tutorial: First RAG Pipeline
Tutorial: Extractive QA Pipeline