Thank you for making this, how can I use this for embeddings

[thistleknot]

I tried to copy settings.yml in place

this is my work around for running this on oracle linux 8 as I don't have glibc, your container bypassed my limitation and I can load llama.cpp (gguf) models once more.

[Atinoda]

Hi @thistleknot, I'm glad to hear that you find the container useful! If you'd like to generate embeddings you can use the API endpoint, however, most open source models are not capable of creating vectors that are drop-in compatible with OpenAI frameworks. Please consult the upstream project for additional information.