Skip to content

[runtime, ops] feat: LRU prefix KV cache for cross-request prefill reuse#25

Open
Ziyi-Wang wants to merge 5 commits into
verl-project:mainfrom
Ziyi-Wang:feat/lru-prefix-cache
Open

[runtime, ops] feat: LRU prefix KV cache for cross-request prefill reuse#25
Ziyi-Wang wants to merge 5 commits into
verl-project:mainfrom
Ziyi-Wang:feat/lru-prefix-cache

split plan_prefix_cache into compute + count, cache hashes on request

2ada269
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs