Pin vllm, preflight prompt lengths, enrich engine-init errors in capture

[mvyleteljr]

Why

Auditing the behavior-audit Pearl scoring failures traced every "Engine core initialization failed" / over-length failure to three gaps in the capture stack:

1. Unpinned vllm in the Modal image. Residual capture depends on vLLM's experimental extract_hidden_states speculative path, which changes shape across minor releases (0.19.1 regressed the v1 input validator — same reason yora pins). Any image rebuild could silently change engine behavior.
2. Over-long prompts failed late. A single prompt exceeding max_model_len surfaced as a per-request error minutes after a full GPU cold start + 131 GB weight load, without naming the offending example.
3. Engine-init root causes were lost. vLLM's EngineCore prints the real failure (typically KV cache not fitting) to worker stdout only; run catalogs stored just the generic wrapper message.

What

• Pin vllm==0.19.0 in VLLMEngine.runtime_spec() with rationale comment (keep in sync with yora/modal_base.py; bump deliberately).
• _preflight_prompt_lengths(): tokenize all prompts before LLM() init; raise SpecValidationError listing offending example keys/counts.
• _engine_init_error_message(): engine-init RuntimeErrors now carry the engine config (max_model_len, TP, gpu_memory_utilization, chunked prefill, …) and a KV-cache-fit hint into run catalogs.
• Unit tests for the preflight.

Validated end-to-end: the Pearl 1,000-session scoring run (6,355 examples, 40k context, H200:2) completed first-try in ~20 min through this code path.

🤖 Generated with Claude Code