Vault Chat isn't working with Ollama

[apsjohn]

Hello, when selecting Vault chat it seems to reference some of my files while repeating some of them and i see this after the referencing "Understood! I will only include `

tags for existing markdown blocks you provide withstartLineandendLine` attributes when referencing them. For newly generated markdown content, line numbers won't be included.

I'm ready to assist with your Obsidian editing and organization questions using markdown format as specified. Please ask away! "
After that whatever i prompt i get that 'I don't have access to your obsidian vault'...

I am using gemma 2-9b from ollama with bge-m3 for embedding...
Any thoughts why this is happening?

Also the button vault chat is doing the above in a started conversation, in a new conversation in the input box if i press it, it doesnt do anything...

[kevin-on]

Thank you for reporting this issue. This behavior is likely related to the limitations of smaller models when handling complex contexts and longer prompts. I will add this to our tracking issue #146.

Also the button vault chat is doing the above in a started conversation, in a new conversation in the input box if i press it, it doesnt do anything...

About this, could you provide more details about what you're experiencing?

[apsjohn]

https://imgur.com/a/w7rAPi7
Actually if i write a prompt and press it works. But the first problem mentioned that it doesnt actually do anything with the referenced files is shown here...

Does the ai have context for file structure, tags, links etc? What i am after is to be able to talk and query our assistant more generally about our vault as a whole...
I tried making my own solution with training models on my own data, but it is more complex to get right...

FYI: Today i will fork your plugin to add a amazon bedrock integration, which has most models with 0 data retention...
Edit: It would also be good if you can expose some API methods, for people who want to make their own implementation for connecting a local model to your plugin. That way someone, for example, could use unsloth library for inferencing which is much more optimized than ollama for example, or other custom valueable uses ...

[kevin-on]

The plugin does include retrieved context from your vault in the requests. Small models just struggle with longer prompts and complex contextual queries, leading to those repetitive responses.

You can see how we structure the prompts and context in promptGenerator.ts.

It would also be good if you can expose some API methods

Could you elaborate on which specific methods you'd like to see exposed? I'm not familiar with how other local models work.

[apsjohn]

Could you elaborate on which specific methods you'd like to see exposed? I'm not familiar with how other local models work.

I haven't taken a closer look to what exactly you are passing to each model call you make and where. But generally the idea would be to make an interface that has all the logic of the actions the plugin does in relation to ai calling, where its internal model API implementations could also pass through, and this way we could make external model implementations more easily as well...
Edit: If the plugin isn't built like this though, then this should go to advanced later feature request...

[apsjohn]

I didn't manage to make it work with AWS Bedrock... Hope you implement it some day so will be able to use full functionality privacy free!

[glowingjade]

If AWS Bedrock supports OpenAI-compatible APIs, you should be able to use it by selecting 'Custom (OpenAI Compatible)' as your Chat model.
For more details on setting this up, you can check our documentation:
https://github.com/glowingjade/obsidian-smart-composer/wiki/1.4-Advanced-Configuration#custom-openai-compatible-models

[apsjohn]

Thanks for following up! If i do come up with a working solution i will share it but Amazon Bedrock uses a different API structure and authentication system (AWS credentials) that isn't compatible with that structure.

[kevin-on]

Thanks for sharing. While we don't have immediate plans to implement Amazon Bedrock integration, we're keeping track of user requests. If more users express interest in this support, we'll definitely consider prioritizing it in our roadmap. Would it be okay to close this issue?

[apsjohn]

Yes please do