Re-write the Ollama dev service

[edeandrea]

The current implementation of the Ollama dev service is a hot mess. It doesn't work properly. Therefore I have done the following things:

1. Merged the quarkus-langchain4j-ollama-devservices artifact into the quarkus-langchain4j-ollama-deployment artifact.
2. The dev service is aware of whether or not there is a local Ollama service running on port 11434. If so, it doesn't do anything.
3. If there isn't a local Ollama service running on port 11434 then it will start an ollama/ollama:latest container on a random port. It will also expose a few configuration properties with values of the host & port the container is running on, which can be used by applications to manually configure things if need be.
4. The container shares the filesystem location with the Ollama client (user.home/.ollama), so any models downloaded by the client or the container are shared with each other, and therefore only need to be downloaded once by one or the other.
5. The pulling of the required models hasn't changed. The main dev service still uses the Ollama rest api to pull the required models. It is simply passed a URL, which could be the local Ollama client, or it could be a url to a container. It doesn't matter at that point.

The documentation has also been updated to reflect everything.

[geoand]

Thanks a lot for this work @edeandrea.

Just to have a public record what I also said in public to you: I don't plan to spend any time on this whatsoever, so if you believe this solves a use case you have and you are willing to assume the full maintainance of all this Ollama in a container stuff, I'm willing to accept it.

[edeandrea]

Thanks a lot for this work @edeandrea.

Just to have a public record what I also said in public to you: I don't plan to spend any time on this whatsoever, so if you believe this solves a use case you have and you are willing to assume the full maintainance of all this Ollama in a container stuff, I'm willing to accept it.

Sounds good!

[geoand]

👌

[edeandrea]

@geoand one thought/question I had while working on this - the devservice stuff that you wrote in core/deployment/src/main/java/io/quarkiverse/langchain4j/deployment/devservice (DevServicesOllamaProcessor, OllamaClient, JdkOllamaClient, etc)...

Should those really belong in the core deployment module? Or should those belong in the ollama deployment module?

Whats weird is

quarkus-langchain4j/core/deployment/src/main/java/io/quarkiverse/langchain4j/deployment/config/LangChain4jBuildConfig.java

Line 37 in cefbc8c

DevServicesConfig devservices();

- it seems there is devservice stuff thats specific to ollama located in the quarkus.langchain4j.devservices config item? Doesn't this belong nested under .ollama. as well?

I can do this additional cleanup if you think it makes sense. Otherwise I'll leave it as is.

[geoand]

Should those really belong in the core deployment module?

Yes, because the hope is that at some point Ollama will be able to provide all OpenAI APIs as well

[edeandrea]

Yes, because the hope is that at some point Ollama will be able to provide all OpenAI APIs as well

Got it. I'll leave things as is then.

[geoand]

👌

[quarkus-bot]

Status for workflow Build (on pull request)

This is the status report for running Build (on pull request) on commit d94639b.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

[geoand]

Very nice work!