Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support local llm using ollama, lm-studio or any other OpenAPI-compatible local servers #5

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

b3nab
Copy link

@b3nab b3nab commented Aug 6, 2024

hardcoded local server URL is: http//localhost:1234 - default from LM Studio

  • imported partial working code from Yhozen:feat/ai-sdk
  • add new model-context
  • extension-model ui component
  • use model context inside chat and summary
  • add new background port to auto-retrieve available models from local server
  • improve select ui adding tabs for Online and Local models

Briefly I took the work you were already doing with @Yhozen from the Yhozen:feat/ai-sdk branch and extended it to support a local server. I didn't went down the path to fully create an interface to manage settings like the url of the server (or other things like editing the prompts with proper {variables} for transcript, video title etc..) so I hardcoded it to the default server url from LM Studio.

I wanted to share this, feel free to do whatever you want with this code. 😆

Great work done with this extension @PaoloJN , this local llm feature is just the cherry on top 🍒 .

…ible local servers

hardcoded local server URL is: http//localhost:1234  - default from LM Studio

- imported partial working code from Yhozen:feat/ai-sdk
- add new model-context
- extension-model ui component
- use model context inside chat and summary
- add new background port to auto-retrieve available models from local server
- improve select ui adding tabs for Online and Local models
@PaoloJN
Copy link
Owner

PaoloJN commented Aug 9, 2024

Thank you for your contribution @b3nab! The updates you've made, the integration of the local server support, look great. I really appreciate the way you've extended the work from the @Yhozen Yhozen:feat/ai-sdk branch to enhance the model management capabilities.

I'll take a closer look at the changes soon and plan to merge them afterward. You've done a fantastic job with this great work! 🍒.

Thanks @Yhozen & @b3nab : )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants