New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a choice of how to end streaming from callback: STOP or CANCEL #1476

Draft

sbalandi wants to merge 1 commit into openvinotoolkit:master from sbalandi:callback

Contributor

sbalandi commented Jan 3, 2025

No description provided.

github-actions bot added category: visual language category: continuous batching category: LLM category: speculative decoding category: GenAI C++ API no-match-files category: prompt lookup labels

Contributor Author

sbalandi commented Jan 3, 2025

TODO: add CANCEL for ContinuousBatching

sbalandi force-pushed the callback branch from 62439bf to 3800085 Compare

January 3, 2025 22:10


          Add a choice of how to end streaming from callback: STOP or CANCEL

ilya-lavrenov added this to the 2025.0 milestone

ilya-lavrenov self-assigned this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: continuous batching category: GenAI C++ API category: LLM category: prompt lookup category: speculative decoding category: visual language no-match-files