Skip to content

Fix vLLM crash when max_gen_toks exceeds model context window#153

Open
Ali-Elganzory wants to merge 3 commits into
mlfoundations:mainfrom
Ali-Elganzory:fix/vllm-context-window-crash
Open

Fix vLLM crash when max_gen_toks exceeds model context window#153
Ali-Elganzory wants to merge 3 commits into
mlfoundations:mainfrom
Ali-Elganzory:fix/vllm-context-window-crash

fix: cap max_new_tokens to avoid degraded/undefined behavior when exc…

e93d565
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs