Skip to content

KV cache refactor to decouple cache blocks and metadata about them#168

Merged
yelite merged 1 commit intooctoml:batch-servingfrom masahi:kv-refactorJan 20, 2024

Commits

Commits on Jan 19, 2024