Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DevServices support for Ollama model #182

Merged
merged 1 commit into from
Dec 22, 2023

Conversation

lordofthejars
Copy link
Contributor

@lordofthejars lordofthejars commented Dec 21, 2023

This PR adds dev services to Ollama, so in dev mode, you don't need to have Ollama running locally; it starts an Ollama container and installs the first time the model you want to use. After that, every run against the same model will reuse the created image.

One of the things in this implementation that is different from other devservices is that it is developed in its module, so you need to add the module to enable it.

The reason it is this way is that the Ollama container takes a lot of minutes to start the first time, and also when running it and querying locally might take some seconds, so maybe it's not always the best to have devservices running. I know there is the enable flag, so here we can decide if we want to set the flag enabled to false by default and let the user explicitly set to true (which means that developers might not notice there is a devservice by default) or we explicitly make them add as a dependency enabled to true by default

@lordofthejars lordofthejars requested a review from a team as a code owner December 21, 2023 09:27
@geoand
Copy link
Collaborator

geoand commented Dec 21, 2023

Thanks for this!

@cescoffier WDYT?

@lordofthejars
Copy link
Contributor Author

Tests are not the best, I followed what I've found. I tested locally

Copy link
Collaborator

@cescoffier cescoffier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Thanks!

@lordofthejars
Copy link
Contributor Author

The build is failing because of modules not related with my change.

@geoand
Copy link
Collaborator

geoand commented Dec 22, 2023

Indeed, I restarted CI

@lordofthejars
Copy link
Contributor Author

When the PR is merged, I'll update the documentation with DevServices section.

@geoand geoand merged commit 88f2dd4 into quarkiverse:main Dec 22, 2023
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create a DevService for ollama
3 participants