Skip to content

Commit

Permalink
[Doc] Remove outdated comment to avoid misunderstanding
Browse files Browse the repository at this point in the history
The comment previously mentioned that prefix caching and sliding-window
were incomplete, but these features have now been implemented [Issue
4537](#4537)
  • Loading branch information
homeffjy committed Oct 11, 2024
1 parent 36ea790 commit 6b2976b
Showing 1 changed file with 3 additions and 4 deletions.
7 changes: 3 additions & 4 deletions vllm/core/block_manager_v2.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,8 @@ class BlockSpaceManagerV2(BlockSpaceManager):
autoregressively-generated tokens, and other advanced features such as
prefix caching, forking/copy-on-write, and sliding-window memory allocation.
The current implementation is partial; in particular prefix caching and
sliding-window are not feature complete. This class implements the design
described in https://github.com/vllm-project/vllm/pull/3492.
This class implements the design described in
https://github.com/vllm-project/vllm/pull/3492.
Lookahead slots
The block manager has the notion of a "lookahead slot". These are slots
Expand Down Expand Up @@ -190,7 +189,7 @@ def allocate(self, seq_group: SequenceGroup) -> None:

assert (request_id
not in self.cross_block_tables), \
"block table already exists"
"block table already exists"

check_no_caching_or_swa_for_blockmgr_encdec(self, seq_group)

Expand Down

0 comments on commit 6b2976b

Please sign in to comment.