Is it possible to have openrouter as the LLM API source?

[chenyuz3]

I am a openrouter heavy user and I really hope this can come true..

[cephalization]

Open router appears to support the OpenAI client https://openrouter.ai/docs/quick-start

this means that you should be able to enter the openrouter base url and api key within the OpenAI section of cannoli settings and have it work

let me know how it goes!

[chenyuz3]

Open router appears to support the OpenAI client https://openrouter.ai/docs/quick-start

this means that you should be able to enter the openrouter base url and api key within the OpenAI section of cannoli settings and have it work

let me know how it goes!

Well, it turns out if I use OpenAI Models cannoli will be fine in most cases, but if I use other models (i.e. Gemini 1.5 flash as my favorite model for its long output window + low price) small issues just keep occurring. I assume this is because I am using openrouter via the "OpenAI config" in the settings but gemini models have different API input/output configs...? The color thing (for both nodes and arrows never really works as expected from the college) is especially frustrating.

[cephalization]

Open router appears to support the OpenAI client https://openrouter.ai/docs/quick-start

this means that you should be able to enter the openrouter base url and api key within the OpenAI section of cannoli settings and have it work

let me know how it goes!

Well, it turns out if I use OpenAI Models cannoli will be fine in most cases, but if I use other models (i.e. Gemini 1.5 flash as my favorite model for its long output window + low price) small issues just keep occurring. I assume this is because I am using openrouter via the "OpenAI config" in the settings but gemini models have different API input/output configs...? The color thing (for both nodes and arrows never really works as expected from the college) is especially frustrating.

Can you give more details on the small issues you are experiencing? An openai compatible client means that open router should be handling all conversions from an incoming openai configuration to the outgoing model that you choose...

[chenyuz3]

Open router appears to support the OpenAI client https://openrouter.ai/docs/quick-start
this means that you should be able to enter the openrouter base url and api key within the OpenAI section of cannoli settings and have it work
let me know how it goes!

Well, it turns out if I use OpenAI Models cannoli will be fine in most cases, but if I use other models (i.e. Gemini 1.5 flash as my favorite model for its long output window + low price) small issues just keep occurring. I assume this is because I am using openrouter via the "OpenAI config" in the settings but gemini models have different API input/output configs...? The color thing (for both nodes and arrows never really works as expected from the college) is especially frustrating.

Can you give more details on the small issues you are experiencing? An openai compatible client means that open router should be handling all conversions from an incoming openai configuration to the outgoing model that you choose...

i.e. note node can not be directly referenced:

and all reference have to be in {{[[note]]}} format (unlike what is shown in college):

, and sometimes it is color dependent... which I have never really figured out.

functions like choice arrows are not working too:

[blindmansion]

Thanks for those examples, I'll try to replicate these.

You said that these are only happening when using non-openai providers through OpenRouter? Or are these errors happening in other cases too?

[cephalization]

Choice arrows may be unreliable depending on the model chosen in open router but everything else should be unrelated to LLMs at all and may just be real bugs.