Feature request: Chat history persisted to Cosmos DB

[hfaouaz-rcg]

Please provide us with the following information:

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [X] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Any log messages given by the failure

Expected/desired behavior

To Capture and maintain question/answers in cosmos. I was able to refactor current codebase and chatGPT code base to capture questions in cosmos. still some work needs to be done, but I am wondering if this feature is on the roadmap. Once I have a complete working solution, I will create a feature enhancement PR.

Build cosmos biceps <- done
provide api for history capture/generate/list <- done
capture question <-done
capture answer
session base/conversation base

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

azd version?

run azd version and copy paste here.

Versions

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

[DuboisABB]

This seems like a duplicate of #489. There is also a stale PR that implemented this (although probably not based on sample-app-aoai-chatGPT): #259
I'm also interested in this functionality. Do you have a public fork where I could follow your development? I might be able to contribute too, as I had also planned on doing this.

[hfaouaz-rcg]

This seems like a duplicate of #489. There is also a stale PR that implemented this (although probably not based on sample-app-aoai-chatGPT): #259 I'm also interested in this functionality. Do you have a public fork where I could follow your development? I might be able to contribute too, as I had also planned on doing this.

Hi @DuboisABB . I am currently developing it for a client. Let me see if I can create a fork minus clients changes. would love the help.

[cloudhunnter]

I also have a client asking for this kind of feature. @hfaouaz-rcg did you made any steps towards a working solution? What kind of services do you implement for this? i was thinking about cosmos db, but im fairly new to databases tbh.

[DuboisABB]

This request is pretty much the same: #1822

[idrisfl]

I'm also interested in this kind of feature for one of my clients. We are looking to implement a feedback loop on the responses, which would be stored in a Cosmos DB.

[cloudhunnter]

Yesterday, I was able to successfully store a full chat history in my CosmosDB for the first time. This was done from a local environment using my deployment user. I haven’t had the chance to further test and refine the implementation yet. Some adjustments and optimizations are definitely still needed. Additionally, I need to consider the exact timing of when the data should be stored. This has already been implemented within the scope of a client project, so it’s no longer very "general."

I largely based my work on this reference (#259) and adapted the components necessary for my use case. My goal is simply to enable the evaluation of the chat histories in the background to determine whether the source data needs to be optimized. So, i almost skipped the whole frontend part for the user and only focused on the backend app.py, api and adjusted the bicep, so the cosmosdb is already been created with everything else.
I also ran into a role problem, which took me a while to solve, because for querying a cosmosDB you need a role thats not been visible in the azure roles UI 😄

[hfaouaz-rcg]

Hello @idrisfl , @cloudhunnter and @DuboisABB .
This is my fork that I created for a client. It will capture the question and answer json into cosmos. I also included the bicep to create the cosmos.

You may need to add the following enviornments

AZURE_OPENAI_MODEL="gpt-4o"
AZURE_OPENAI_ENDPOINT="https://xxxxxxxx.openai.azure.com/"
AZURE_COSMOSDB_ACCOUNT="cosmos-accountname"
AZURE_COSMOSDB_DATABASE="db_conversation_history"
AZURE_COSMOSDB_CONVERSATIONS_CONTAINER="conversations"
AZURE_COSMOSDB_ACCOUNT_KEY=""
APPLICATIONINSIGHTS_CONNECTION_STRING=""

@pamelafox please look into this to see if we can integrate it. I moved some code from the https://github.com/microsoft/sample-app-aoai-chatGPT/tree/main project and made it work within the Chat flow. I can help with integration if needed.

Fork repo:
https://github.com/hfaouaz-rcg/azure-search-openai-demo

Note: two new api added. history/generate history/list

Hope this help.

[ibrahimnasir0]

Hello @hfaouaz-rcg,

I tried deploying the application using the environment variables and setup mentioned in your commit, but I am encountering an application error during deployment.

Would you be able to guide me on what might be causing the issue? I have double-checked the environment variables but still can't seem to resolve the error. Any help or pointers on what could be wrong and how to fix it would be greatly appreciated!

Thank you in advance!

[hfaouaz-rcg]

hi @ibrahimnasir0, what kind of error you are getting. if you can show me the stack trace, perhaps I can guide you.

[cforce]

@hfaouaz-rcg This is great!
Can you please create a pull request with this feature using a switch for bicep to create cosmos db and what else is needed .
Also use the switch to offer the option in ui and api in python code to switch it on or off. That would be great. Btw - you would need to rebase as well : tx for contributing this

[ibrahimnasir0]

Hello @hfaouaz-rcq,

Thank you for your response! I am getting an application error during deployment. I’ve double-checked the environment variables, but the error persists. Could you share your LinkedIn profile or any other contact information so I can connect with you directly for more guidance?

Here’s my LinkedIn profile for your reference: https://www.linkedin.com/in/ibrahimnasir0/

Thanks again for your help!

[MayankSingh173]

@ibrahimnasir0 @hfaouaz-rcg Could you please share the rebased version of your code. I also want to implement in my app

[pamelafox]

This repo now has support for browser-based chat history:
https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/docs/deploy_features.md#enabling-chat-history

I have re-titled this issue to focus on adding Cosmos DB persistence, and will keep this issue open.

For those who want Cosmos DB persistence, are you also using the user login feature? Or would you want persistence when users aren't logged in?

[cforce]

authorization is mandatory for me -always 😄 I still don‘t get how this and the other chat history Mr play together. Is it enhancing or duplication? Pamela Fox ***@***.***> schrieb am Fr. 4. Okt. 2024 um 18:44:

…

This repo now has support for browser-based chat history: https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/docs/deploy_features.md#enabling-chat-history I have re-titled this issue to focus on adding Cosmos DB persistence, and will keep this issue open. For those who want Cosmos DB persistence, are you also using the user login feature? Or would you want persistence when users aren't logged in? — Reply to this email directly, view it on GitHub <#1863 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADAARXHE3SB4QGE3S3EHRLZZ3AXFAVCNFSM6AAAAABLQRTP7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJUGA4TKMRVHE> . You are receiving this because you commented.Message ID: ***@***.***>

[pamelafox]

When we add CosmosDB support, then you would enable one or the other, I think. (We'd need to warn if you enabled both since that wouldn't make sense).

[hfaouaz]

This repo now has support for browser-based chat history: https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/docs/deploy_features.md#enabling-chat-history

I have re-titled this issue to focus on adding Cosmos DB persistence, and will keep this issue open.

For those who want Cosmos DB persistence, are you also using the user login feature? Or would you want persistence when users aren't logged in?

Hi Pamela, currently our client as fine with anonymous user id, to encourage feedbacks :-) but I believe the cosmos code, handle capturing used id if they are logged in.

P.S. sorry, I've been swamped with work to do a PR with code changes that I've done.

[cforce]

I was more interested in exploring the feature gaps between the two solutions rather than non-functional aspects like scalability, cost, and operational complexity. As you mentioned, why should we use both databases for persistence if there is no noticeable difference in user experience?

…

When we add CosmosDB support, then you would enable one or the other, I think. (We'd need to warn if you enabled both since that wouldn't make sense). — Reply to this email directly, view it on GitHub <#1863 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADAARRWLB5LTVRFBCLILALZZ3CB3AVCNFSM6AAAAABLQRTP7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJUGEYTGMBXHA> . You are receiving this because you commented.Message ID: ***@***.***>

[pamelafox]

IndexedDB is only persisted in the browser, which means that if the user switches browsers/computers, they no longer have the history.
CosmosDB would be persisted in the server, and if the user is logged in, it'd be tied to their user ID, so that it would be available across computers.
We'd have to decide whether we'd create guest user IDs for CosmosDB persistence for folks who want CosmosDB persistence but don't want to enforce user login. Those guest user IDs likely would not follow the users around, so the main benefit there is just that the developers would have access to chat history, which might be helpful for app improvement (but you should be wary of looking at your user's chats / PII and secrecy and all that).

[cforce]

Understood. While it's important to protect PII and secrets, storing data without associating it with a user doesn't make much sense. Otherwise, how would we restore the data later? It can't be anonymous since that would involve PII.

I believe it would be beneficial to store the data based on user consent, perhaps like a bookmark feature. Encrypting the data with browser-managed keys could be another option, although the user would need to enter the key for each intial persistent session, and we'd have to store it locally. This approach is similar to how browsers handle storing secrets.

[pamelafox]

@fujita-h has added support for persistence using CosmosDB, tied to the user's entra ID:
https://github.com/Azure-Samples/azure-search-openai-demo/releases
There's no user consent there, but I think that will be fine given that the user has already consented to logging in.

I'll close this issue, but please open new ones with ideas for how that feature can be improved. Thank you all!

[hfaouaz]

@pamelafox , thank you!!!