Search results

Why decode use "BatchPrefillWithPagedKVCacheWrapper" too ?

Status: Open.

#65 In microsoft/sarathi-serve;

· zhanggj821 opened on Oct 20, 2025

Upgrade flashinfer to latest version

Status: Open.

#62 In microsoft/sarathi-serve;

· MohammedAdain opened on Oct 6, 2025

Logical issue in temperature handling for near-zero values

Status: Open.

#61 In microsoft/sarathi-serve;

· genglinWang opened on Jun 3, 2025

Errors with Llama models

Status: Open.

#57 In microsoft/sarathi-serve;

· kjain1810 opened on Apr 16, 2025

Error when using Vidur profiling for Llama-2-70b attention core

Status: Open.

#55 In microsoft/sarathi-serve;

· Hudayday opened on Mar 17, 2025

how does it compare to vLLM and SGLang?

Status: Open.

#53 In microsoft/sarathi-serve;

· aniketmaurya opened on Feb 5, 2025

offline_inference.py occurs error that append_paged_kv_cache() missing 1 required positional argument: 'kv_last_page_len'

Status: Open.

#49 In microsoft/sarathi-serve;

· yuhkalhic opened on Jan 9, 2025

Is VLLM compatible

Status: Open.

#48 In microsoft/sarathi-serve;

· BigCousin-z opened on Jan 6, 2025

OSDI paper experiments of figure 10 and 11

Status: Open.

#46 In microsoft/sarathi-serve;

· marwage opened on Nov 29, 2024

pipeline parallel fails at high load

Status: Open.

#41 In microsoft/sarathi-serve;

· yunoJ opened on Sep 4, 2024