Skip to content
This repository has been archived by the owner on Jan 5, 2025. It is now read-only.

Make socket io sessions scale #625

Open
gharbat opened this issue Feb 14, 2024 · 1 comment · May be fixed by #643
Open

Make socket io sessions scale #625

gharbat opened this issue Feb 14, 2024 · 1 comment · May be fixed by #643
Assignees
@itssubhodiproy
Labels
🌠 Enhancement Enhance a current feature

Comments

@gharbat
Copy link
Member

gharbat commented Feb 14, 2024

Currently, the socket connections work fine when there is a single LLM server instance. Once we have multiple instances, the socket connections get lost because they are not sticky, and the load balancer routes requests to a server where the session ID is not defined.
@gharbat gharbat added the 🌠 Enhancement Enhance a current feature label Feb 14, 2024
@itssubhodiproy
Copy link

Hey, I want to work on this. Can you assign this to me?

@gharbat gharbat assigned gharbat and itssubhodiproy and unassigned gharbat Feb 14, 2024
@itssubhodiproy itssubhodiproy linked a pull request Feb 19, 2024 that will close this issue
Open
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@itssubhodiproy itssubhodiproy

Labels
🌠 Enhancement Enhance a current feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Feat] Socket scale using ip-hash algo itssubhodiproy/OpenCopilot
2 participants
@gharbat @itssubhodiproy