OpenAI 429 error even when using HuggingFace Model

[medeirosle]

Hello,

I'm trying to use HuggingFace model, and I keep getting 429 error from OpenAPI My code:

`export class EmbedService implements OnModuleDestroy, OnModuleInit {
private ragApplication: RAGApplication;

public async query(query: string) {
    return this.ragApplication.query(query);
}

async onModuleInit() {

    this.ragApplication = await new RAGApplicationBuilder()
        .setModel(new HuggingFace({ modelName: "meta-llama/Llama-2-7b" }))
        .setTemperature(0.1)
        .addLoader(new YoutubeLoader({ videoIdOrUrl: 'https://www.youtube.com/watch?v=w2KbwC-s7pY' }))
        .addLoader(new WebLoader({ url: 'https://adhityan.com/' }))
        .setVectorDb(new ChromaDb({ url: 'http://localhost:8000' }))
        .build();
}

onModuleDestroy() {}

}

The error:

`InsufficientQuotaError: 429 You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.
at Object.defaultFailedAttemptHandler [as onFailedAttempt] (file:///home/leandro/code/CDEBot/node_modules/.pnpm/@langchain[email protected]/node_modules/@langchain/core/dist/utils/async_caller.js:33:21)
at RetryOperation._fn (/home/leandro/code/CDEBot/node_modules/.pnpm/[email protected]/node_modules/p-retry/index.js:67:20)
at processTicksAndRejections (node:internal/process/task_queues:95:5)

Important: I just can reach this point if Open AI env API key variable is set. If I don˜t set it,I got the error:

Error: OpenAI or Azure OpenAI API key not found

Thanks! :)

PS: Congratulations for the project, awesome work :)

[adhityan]

Hello @neomedeiros - this happens because there are two models involved -

1. The LLM - which you have set to meta-llama/Llama-2-7b
2. The embedding model - which you have not explicitly set and defaults to OpenAI's 3-small model

The embedding model is used to convert a text into a vector and is different from the LLM model. The library allows you to change this and even use a non OpenAI embedding model. You can do so using the setEmbeddingModel method in the builder. Look at the documentation here. The library today does not support using embedding models available in HuggingFace at the moment, but this is coming very soon.

Hope this is helpful.

[medeirosle]

Understood.

Just tested now (setting de EmbeddingModel) and it worked perfectly!

Many thanks @adhityan :)