Skip to content

[Continuous batching] In the event of OOM, return tokens generated so far for the request #661

Merged
Wovchena merged 1 commit intoopenvinotoolkit:masterfrom mzegla:oom_returnsJul 24, 2024
+36-46

Commits

Commits on Jul 22, 2024