ollama integration

[jasonmehlhoff]

Hello, I'm not sure I'm in the right area, this is my first post. I have a self-hosted ollama LLM running and would like to access it with this app, but I am having trouble. Can you reach out to me?
Thanks!
Jason

[Taewan-P]

Hi. What kind of issue are you facing?

[jasonmehlhoff]

Thanks for getting back to me so quick! Well, I have a self-hosted LLama3.1:latest running on a computer in my apartment. It's in a Ubuntu 22.04 server. Right now I access from a FQDN (I have my own domain I play around with). I did get my API from Ollama's website. I'd like to use your app to access my LLama3.1:latest but I'm not sure how to connect it. When I put in my domain address in the app I get a 404 error. I feel like I'm doing something wrong or missing a step! Haha. I don't mean to bother you and I can continue to play around with it, I just thought you might have some thoughts to point me in the right direction. :-) Thanks again, Jason Dec 17, 2024 12:57:51 AM Taewan Park ***@***.***>:

…

Hi. What kind of issue are you facing? — Reply to this email directly, view it on GitHub[#134 (comment)], or unsubscribe[https://github.com/notifications/unsubscribe-auth/BNYNLFTZRS2HYPS2J52FE7T2F7DO5AVCNFSM6AAAAABTXSXXRSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNJYHEZTONA]. You are receiving this because you authored the thread. [Tracking image][https://github.com/notifications/beacon/BNYNLFXRIFCRLT4GTBFW3FL2F7DO5A5CNFSM6AAAAABTXSXXRSWGG33NNVSW45C7OR4XAZNRIRUXGY3VONZWS33OINXW23LFNZ2KUY3PNVWWK3TUL5UWJTQAWDLP4.gif]

[Taewan-P]

The custom url address should be set to http://your_ollama_address:11434/, so the app can access http://your_ollama_address:11434/v1/chat/completions API.

Is this correctly set for your case? Tried maybe accessing the url via your phone browser?

[jasonmehlhoff]

This is where I am noticing the issue too. I actually access via the 8080 port set in my Nginx. It doesn't seem to work for me with 11434 way locally or from web. It's ok, I can keep playing around with it. I don't want to take up all your time! ;-) Thank you, Jason Dec 17, 2024 2:34:36 AM Taewan Park ***@***.***>:

…

The custom url address should be set to *http://your_ollama_address:11434/*, so the app can access *http://your_ollama_address:11434/v1/chat/completions* API. Is this correctly set for your case? Tried maybe accessing the url via your phone browser? — Reply to this email directly, view it on GitHub[#134 (reply in thread)], or unsubscribe[https://github.com/notifications/unsubscribe-auth/BNYNLFXY4J52T7EO4SUDZU32F7OZVAVCNFSM6AAAAABTXSXXRSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNJZGAZDKNY]. You are receiving this because you authored the thread. [Tracking image][https://github.com/notifications/beacon/BNYNLFWWCKNDATT4EPABNP32F7OZVA5CNFSM6AAAAABTXSXXRSWGG33NNVSW45C7OR4XAZNRIRUXGY3VONZWS33OINXW23LFNZ2KUY3PNVWWK3TUL5UWJTQAWDNHC.gif]

[jasonmehlhoff]

I'm starting to figure this out I think. I believe I have my domain address pointed to the Open WebUI server IP. I have to figure out how to point my domain to the internal local address of the ollama instead. I'm just not sure how I do this. Thanks, Jason Dec 17, 2024 9:45:12 AM Jason Mehlhoff ***@***.***>:

…

This is where I am noticing the issue too. I actually access via the 8080 port set in my Nginx. It doesn't seem to work for me with 11434 way locally or from web. It's ok, I can keep playing around with it. I don't want to take up all your time! ;-) Thank you, Jason Dec 17, 2024 2:34:36 AM Taewan Park ***@***.***>: > > The custom url address should be set to *http://your_ollama_address:11434/*, so the app can access *http://your_ollama_address:11434/v1/chat/completions* API. > > Is this correctly set for your case? Tried maybe accessing the url via your phone browser? > > — > Reply to this email directly, view it on GitHub[#134 (reply in thread)], or unsubscribe[https://github.com/notifications/unsubscribe-auth/BNYNLFXY4J52T7EO4SUDZU32F7OZVAVCNFSM6AAAAABTXSXXRSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNJZGAZDKNY]. > You are receiving this because you authored the thread. > [Tracking image][https://github.com/notifications/beacon/BNYNLFWWCKNDATT4EPABNP32F7OZVA5CNFSM6AAAAABTXSXXRSWGG33NNVSW45C7OR4XAZNRIRUXGY3VONZWS33OINXW23LFNZ2KUY3PNVWWK3TUL5UWJTQAWDNHC.gif] >

[Taewan-P]

Point your domain DNS record to your server's external IP address, and port forward the required ports in your router.