Support for Azure OpenAI and GCP Vertex AI

[jwgo]

I would like to propose adding support for Azure OpenAI and Google Vertex AI within this framework. These integrations would provide significant advantages by incorporating powerful language processing and machine learning capabilities. Leveraging these services could enhance the framework's performance, scalability, and flexibility for a wide range of applications.

Is there a possibility for this integration to be considered in future updates?

I strongly agree with the direction of this framework and have high expectations for its future. I will actively work on creating use cases to explore its full potential.

[int29]

We also desire the addition of this feature.

[mmurad2]

I just opened to these tickets, we'll work on prioritizing for the coming week 😊
#140
#141

[mmurad2]

Do you need these inference provider in the framework only or in the rest of the stack as well (API/UI)?

[jwgo]

At the moment, it would be sufficient to have only the inference provider added. However, it would be greatly appreciated if updates could also be extended to the API and UI.😊😊

[Tomas2D]

Related PR: #164

[Tomas2D]

Temporary workaround with use of LangChain.

import { BaseMessage } from "bee-agent-framework/llms/primitives/message";
import { LangChainChatLLM } from "bee-agent-framework/adapters/langchain/llms/chat";

// you need to install this package first @langchain/google-genai
import { ChatGoogleGenerativeAI } from "@langchain/google-genai";

console.info("===CHAT===");
const llm = new LangChainChatLLM(
    new ChatGoogleGenerativeAI({
    model: "gemini-pro",
    maxOutputTokens: 2048,
  })
);

const response = await llm.generate([
  BaseMessage.of({
    role: "user",
    text: "Hello world!",
  }),
]);

For Azure use this instead

import { AzureChatOpenAI } from "@langchain/openai";

const model = new AzureChatOpenAI({
  temperature: 0.9,
  azureOpenAIApiKey: "<your_key>", // In Node.js defaults to process.env.AZURE_OPENAI_API_KEY
  azureOpenAIApiInstanceName: "<your_instance_name>", // In Node.js defaults to process.env.AZURE_OPENAI_API_INSTANCE_NAME
  azureOpenAIApiDeploymentName: "<your_deployment_name>", // In Node.js defaults to process.env.AZURE_OPENAI_API_DEPLOYMENT_NAME
  azureOpenAIApiVersion: "<api_version>", // In Node.js defaults to process.env.AZURE_OPENAI_API_VERSION
});

Related code: https://github.com/i-am-bee/bee-agent-framework/blob/main/examples/llms/providers/langchain.ts

[mmurad2]

@jwgo @int29 we just shipped a GCP Vertex AI Adaptor https://github.com/i-am-bee/bee-agent-framework/blob/main/examples/llms/providers/vertexai.ts

Azure is coming soon!

[Tomas2D]

Released in v0.0.47