Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CPU] Change kvcache type if user sets INFERENCE_PRECISION_HINT #690

Merged
merged 4 commits into from
Aug 5, 2024
Merged

[CPU] Change kvcache type if user sets INFERENCE_PRECISION_HINT #690

merged 4 commits into from
Aug 5, 2024

Conversation

luo-cheng2021
Copy link
Contributor

@luo-cheng2021 luo-cheng2021 commented Jul 26, 2024 •
edited
Loading

When user sets INFERENCE_PRECISION_HINT change the kvcache type.

Ticket:
145861

ilya-lavrenov
ilya-lavrenov reviewed Jul 30, 2024
View reviewed changes
src/cpp/src/device_config.hpp Outdated Show resolved Hide resolved
src/cpp/src/device_config.hpp Show resolved Hide resolved
@ilya-lavrenov ilya-lavrenov self-assigned this Jul 30, 2024
@ilya-lavrenov ilya-lavrenov added this to the 2024.4 milestone Jul 31, 2024
@Wovchena Wovchena added this pull request to the merge queue Aug 5, 2024
Merged via the queue into openvinotoolkit:master with commit f50aaf7 Aug 5, 2024
33 checks passed
@ilya-lavrenov ilya-lavrenov added the category: continuous batching Continuous batching label Oct 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Wovchena Wovchena Wovchena approved these changes

@ilya-lavrenov ilya-lavrenov ilya-lavrenov approved these changes

Assignees

@ilya-lavrenov ilya-lavrenov

Labels
category: continuous batching Continuous batching
Projects
None yet
Milestone
2024.4
Development

Successfully merging this pull request may close these issues.
4 participants
@luo-cheng2021 @ilya-lavrenov @Wovchena @dtrawins