diff --git a/.agents/roadmap_v1.md b/.agents/roadmap_v1.md index fd0b33e06..554f81173 100644 --- a/.agents/roadmap_v1.md +++ b/.agents/roadmap_v1.md @@ -105,6 +105,12 @@ issue is not yet placed. Keyed record: update in place, never append. | [#517](https://github.com/mudler/vllm.cpp/issues/517) | `MODEL-TEXT-nemotron-h-nemotron-hfor-causal-lm` | Run Nemotron-3.5-Lightning-30B-A3B-NVFP4 e2e: NemotronH hybrid (23 Mamba2 / 6 GQA / 23 non-gated relu2 MoE), first MIXED_PRECISION checkpoint | feature | | [#512](https://github.com/mudler/vllm.cpp/issues/512) | `ENG-RELEASE-WINDOWS` | Windows release runner rejects empty process arguments, so windows-msvc-cpu fails on EVERY pr independently of #514 | bug | | [#514](https://github.com/mudler/vllm.cpp/issues/514) | `ENG-RELEASE-WINDOWS` | `tests/vt/test_backend_cross_device.cpp` uses POSIX `setenv`/`unsetenv`, so both windows-msvc jobs fail on EVERY pr while `main` stays green (they are skipped on push) | bug | +| [#525](https://github.com/mudler/vllm.cpp/issues/525) | `ENG-RELEASE-WINDOWS` | Windows release contract recorder misreports non-empty argv | bug | +| [#537](https://github.com/mudler/vllm.cpp/issues/537) | `ENG-RELEASE-WINDOWS` | Windows OpenAI socket smoke fast-fails instead of reporting a reset response | bug | +| [#540](https://github.com/mudler/vllm.cpp/issues/540) | `ENG-RELEASE-WINDOWS` | Windows Vulkan strict build rejects shadowed KV-cache test locals | bug | +| [#599](https://github.com/mudler/vllm.cpp/issues/599) | `ENG-RELEASE-WINDOWS` | Windows release contract failure fixture is not portable under PowerShell on Linux | bug | +| [#645](https://github.com/mudler/vllm.cpp/issues/645) | `ENG-RELEASE-WINDOWS` | LTX2 sources reintroduce non-portable `M_PI` and break the Windows release gate | bug | +| [#648](https://github.com/mudler/vllm.cpp/issues/648) | `ENG-RELEASE-WINDOWS` | `VideoEngine` uses unguarded POSIX `stat` in the native Windows build | bug | | [#515](https://github.com/mudler/vllm.cpp/issues/515) | `ENG-RECORD-CONFLICT-SURFACES` | check-doc-checkpoint treats every root `CMakeLists.txt` edit as a usage change, so adding a source file demands a `docs/USAGE.md` edit with nothing true to say | bug | | [#547](https://github.com/mudler/vllm.cpp/issues/547) | — | GB10 reports `UnifiedMemory()` true, so `ReferenceTierEligible(kCUDA)` runs the CPU host kernel over `cudaMalloc` pointers; it needs `DeviceMemoryIsHostAddressable()` | bug | | [#569](https://github.com/mudler/vllm.cpp/issues/569) | `MODEL-TEXT-nemotron-h-nemotron-hfor-causal-lm` | The Nemotron revision pin was existence-only: a substituted checkpoint still resolved, and W6's token gate cannot see that | bug | diff --git a/.agents/specs/cpu-llamacpp-floor-x86-2026-08-11.md b/.agents/specs/cpu-llamacpp-floor-x86-2026-08-11.md index ea8caef59..a190db78f 100644 --- a/.agents/specs/cpu-llamacpp-floor-x86-2026-08-11.md +++ b/.agents/specs/cpu-llamacpp-floor-x86-2026-08-11.md @@ -5,7 +5,8 @@ matrix](../backend-matrix.md)) · `QUANT-GGUF-COMPUTE` ([quantization matrix](../quantization-matrix.md)) · `ROAD-V1-D1` punch-list item 13, CPU half ([roadmap-v1-completion](roadmap-v1-completion.md)) · -**issue:** [#433](https://github.com/mudler/vllm.cpp/issues/433) · +**issues:** [#433](https://github.com/mudler/vllm.cpp/issues/433), +[#530](https://github.com/mudler/vllm.cpp/issues/530) · **claim:** `CLAIM-CPU-X86-FLOOR-1` · **base:** `31a2b493`. ## Scope @@ -289,3 +290,48 @@ presented G4's 3.38x/8.20x/2.29x as the live position. Both now point at **Out of scope and stated as such.** The Metal/MLX half of `ROAD-V1-D1` punch-list item 13 needs an Apple M4, which this host is not, and was not touched. + +### Follow-up repair contract: issue #530 + +The exact release-repair candidate +`6f5b98afd4019d6dca492e8ee1edc135163b9d5e` passed the full operator gate, +then exposed a nondeterministic defect while the required handoff gate was +being chained to its push. The smoke harness sets `BUSY_WINDOW=0` and +`QUIET_BUSY=100`; `busy_pct` separately opens `/proc/stat` for `stat_busy` and +`stat_total` at each side of that zero-length interval. On a contended host the +counter boundaries disagree, producing impossible readings including 110%. +`test_a_contended_leg_is_discarded_and_never_summarised` then exits 4 for no +quiet window instead of reaching its intended foreign-work discard and exit 2. +The same immutable tree passed when scheduling happened not to expose the split +snapshot, and failed twice when chained to the push. + +Repair each sample, not the threshold. One read of the aggregate `cpu` line +must derive both busy and total counters, and both `busy_pct` and `run_leg` must +consume paired samples. Preserve which fields count as busy, the fresh-window +policy, builder detection, `QUIET_BUSY`, `FOREIGN_MAX`, discard semantics, +evidence format, and every accepted benchmark value. Do not clamp the result, +raise a ceiling, add a retry to the unit test, or make contention disappear. + +RED-first coverage must reject the current split `stat_busy`/`stat_total` +reads and require both call sites to use one paired snapshot. Mutation must +restore a split sample independently in `busy_pct` and `run_leg`; each must +make the focused contract red. Focused green is the complete +`test_cpu_x86_llamacpp_floor.py` suite under current host contention. The fresh +reviewer mutates both call sites, the operator runs the full preflight, and an +unchanged exact SHA must pass that gate immediately before the plain push. + +The #530 implementation replaces the separate `stat_busy` and `stat_total` +readers with one `stat_sample` reader. A single `awk` invocation derives the +unchanged busy-field sum and total-field sum from the same aggregate `cpu` line, +and both `busy_pct` and `run_leg` read paired before/after samples. No threshold, +clamp, retry, discard rule, evidence format, or benchmark value changed. + +RED-first structural coverage rejected the split reader functions and call +sites. The pre-edit full preflight also reproduced the operational failure: +`test_a_contended_leg_is_discarded_and_never_summarised` observed impossible +`busy=110%` with `BUSY_WINDOW=0` and returned 4 instead of its intended 2. +After the repair, the complete focused suite passed 11/11 under the same host +contention. Independently splitting the initial snapshot in `busy_pct` and in +`run_leg` made the focused structural contract fail for the mutated consumer; +the script was restored byte-for-byte after both mutations. The full unstaged +repository preflight then passed, including the complete CPU floor suite. diff --git a/.agents/specs/release-dry-run-gate-repairs.md b/.agents/specs/release-dry-run-gate-repairs.md index b34a5b0b7..b56f2f015 100644 --- a/.agents/specs/release-dry-run-gate-repairs.md +++ b/.agents/specs/release-dry-run-gate-repairs.md @@ -4,7 +4,12 @@ Identity: `ENG-RELEASE-WINDOWS` Issues: [#499](https://github.com/mudler/vllm.cpp/issues/499) and -[#500](https://github.com/mudler/vllm.cpp/issues/500) +[#500](https://github.com/mudler/vllm.cpp/issues/500), with post-merge follow-up +[#512](https://github.com/mudler/vllm.cpp/issues/512) and +[#514](https://github.com/mudler/vllm.cpp/issues/514), hosted-contract follow-up +[#525](https://github.com/mudler/vllm.cpp/issues/525), native socket-runtime +follow-up [#537](https://github.com/mudler/vllm.cpp/issues/537), and strict +Vulkan-test follow-up [#540](https://github.com/mudler/vllm.cpp/issues/540) Parent specifications: [release-binary-matrix.md](release-binary-matrix.md) and @@ -13,10 +18,10 @@ Parent specifications: Related native compiler contract: [windows-msvc-strict-build.md](windows-msvc-strict-build.md) -Status: `READY`. The developer approved this bounded design on 2026-08-12. It +Status: `ACTIVE`. The developer approved this bounded design on 2026-08-12. It starts from exact main commit `e1087a8812c9b7d96fca5a813981f378fcace638` -after PR #446 merged. Implementation, a fresh review, the operator gate, and a -new non-publishing ten-tuple dry run remain required. +after PR #446 merged. The #537 implementation, a fresh review, the operator +gate, and a new non-publishing ten-tuple dry run remain required. ## Scope @@ -251,6 +256,86 @@ Native Windows compilation, the complete ten-tuple non-publishing workflow, and the tag-run publication/audit remain post-merge acceptance gates. No tag or release is authorized by the local evidence alone. +### Follow-up outcome and repair contract: issue #512 + +PR #508 merged as `2bc4be070a3883f0f7115682469a289f42d86d1a`. +Exact-SHA dry run +[`31625581156`](https://github.com/mudler/vllm.cpp/actions/runs/31625581156) +proved the prior fixes far enough for Windows CPU to compile +`test_cpu_isa_x86.cpp` successfully. Job `94211117810` then failed before its +first no-argument test executed: + +```text +Cannot bind argument to parameter 'Arguments' because it is an empty array. +``` + +`scripts/build-windows-release.ps1:20-26` declares `Invoke-Checked` with a +mandatory `string[] Arguments` parameter. The script deliberately passes +`@()` when executing tests that take no arguments at lines 226-231 and 264-266. +PowerShell parameter binding rejects that explicit empty collection before the +helper invokes the executable. The build, strict MSVC warning gate, and +`test_cpu_isa_x86` link all succeeded; this is not a recurrence of #500. + +The #512 repair is limited to making `Invoke-Checked` accept an explicitly empty +argument array while preserving argument splatting and the non-zero exit-status +failure contract. It must not remove `Invoke-Checked`, bypass any test, add a +dummy argument, weaken the Windows release gate, change a release tuple, or +publish a tag. + +RED-first evidence must execute the real PowerShell helper contract with both +zero and nonzero argument counts. Before the fix, the zero-argument arm must +fail with the observed parameter-binding error. After the fix, it must prove +that the target runs exactly once with zero arguments, that nonempty arguments +arrive unchanged, and that a nonzero child exit remains rejected. Mutating the +helper back to a mandatory non-empty array must make the focused contract red. + +Focused acceptance is the PowerShell contract test plus +`tests.scripts.test_release_windows_metadata`, +`tests.scripts.test_release_pipeline`, the direct Windows portability and +release-binary checkers, and the full repository preflight. Hosted acceptance +requires both Windows CPU and Vulkan jobs in a new exact-merged-SHA ten-tuple +dry run to execute, package, and validate their archives. All other required +tuples, aggregate `build`, and `verify` must also pass on that same SHA before +`v0.0.3-pre.1` is tagged. The tag run must then pass all 15 required jobs and +the authenticated post-publication audit over exactly 32 assets. + +Issue #514 is a separate Windows Vulkan compile defect from the same dry run. +Job `94211117906` compiled and linked `test_cpu_isa_x86`, then MSVC rejected +`setenv` at `tests/vt/test_backend_cross_device.cpp:1004,1048` and `unsetenv` +at line 1050 with C3861. Those POSIX-only calls set, then restore, +`VT_FUSED_TIER` around the cross-device fused-chain test. Linux accepts them; +MSVC exposes `_putenv_s` instead. + +The #514 repair is limited to a test-local cross-platform environment seam. +On Windows it must use checked `_putenv_s`, with an empty value removing the +variable. On POSIX it must preserve checked `setenv(..., 1)` and `unsetenv`. +The fused-chain case must still execute tiers 0 and 1, assert the selected tier, +and restore the caller's prior environment state. Do not disable the case, +change fused-chain production behavior, add a global compatibility macro, or +weaken `/W4` or any release gate. + +RED-first structural coverage must reject the three live unguarded POSIX calls +and require both platform arms in the real translation unit. Mutation must +remove or bypass the Windows arm and make the focused suite fail. Focused green +is the relevant Windows portability suite and direct checker, a clean local +CPU compile/execution of `test_backend_cross_device`, and the full preflight. +Hosted acceptance remains the same exact-merged-SHA dry run: Windows Vulkan +must compile, execute, package, and validate before any tag is authorized. + +The #514 implementation keeps the environment change local to the cross-device +test. A single `SetTestEnvironment` helper uses checked `_putenv_s` on Windows, +passing an empty value for removal, and retains checked `setenv(..., 1)` and +`unsetenv` on POSIX. The fused-chain case still selects tiers 0 and 1, asserts +the active tier, and restores either the saved value or the prior absence. + +RED-first structural coverage failed on the pinned candidate because the helper +and Windows arm were absent. After the repair, the 72-case Windows portability +suite and direct checker passed. A scratch mutation deleting the `_putenv_s` +call failed the focused contract, and the candidate files were restored +byte-for-byte. A clean Release CPU configuration compiled and executed +`test_backend_cross_device`: 19 cases and 6 assertions passed. Native MSVC and +the exact-merged-SHA ten-tuple dry run remain hosted acceptance gates. + Fresh review of immutable implementation `e0b17eb9` found that the compiled version test derived its default expectation from the same cache value under test. Mutating the cache default from `${PROJECT_VERSION}` to `9.9.9` therefore @@ -260,3 +345,595 @@ declaration. With the `9.9.9` mutation applied, that assertion failed exactly with `9.9.9 != ${PROJECT_VERSION}`; after restoration it and the complete 42-test release pipeline suite passed. Production CMake and runtime code remain unchanged by this follow-up. + +Issue #512 implementation makes the existing mandatory `Arguments` parameter +explicitly accept an empty collection; argument splatting and the checked +nonzero exit path are unchanged. The Windows `-ContractTest` path now invokes a +temporary recording target once with zero arguments, invokes it with three +literal nonempty arguments and checks each value, and requires an exit-23 child +to be rejected. Before the production edit, the new focused contract failed on +the missing `AllowEmptyCollection` declaration; the hosted exact-SHA failure +above is the matching real-PowerShell RED execution. The local host has no +PowerShell runtime, so execution of the new live process contract remains a +required native Windows PR gate rather than being inferred from local Python. +After the fix, Windows metadata passed 8/8, release pipeline passed 42/42, and +both direct Windows portability and release-binary checkers passed. Removing +`AllowEmptyCollection` made the focused contract red again, and both changed +files were restored to their exact pre-mutation SHA-256 values. + +Fresh review of combined candidate `0a70da1d` found two test-strength gaps. +Adding an unconditional `return` as the first statement of +`Invoke-CheckedContractTests` left all eight Windows metadata tests green even +though the hosted `-ContractTest` path would silently skip its runtime proofs. +Changing both fused-tier restoration calls to target +`VT_FUSED_TIER_WRONG` likewise left the focused portability test and direct +checker green. The test-only repair now binds the three exact `Invoke-Checked` +calls and their ordered literal outcomes, rejects an executable early control +exit in the contract-test function, and pins both restoration branches to the +exact `VT_FUSED_TIER` key with `saved.c_str()` or `nullptr` respectively. + +After the repair, the early-return mutation and separate wrong-key mutations +in the prior-present and prior-absent restoration branches each failed their +focused test. The existing `AllowEmptyCollection`-removal and `_putenv_s`- +removal mutations also remained red. Production files were restored to their +pre-mutation SHA-256 values. The combined Windows metadata, release pipeline, +and Windows portability suites passed 122/122, followed by both direct Windows +portability and release-binary checkers. Native PowerShell execution and MSVC +compilation remain the required external Windows PR gates. The full unstaged +repository preflight passed every gate except one transient tempfile failure in +`test_check_release_binary_contract`; an immediate isolated retry of that exact +suite passed all 30 tests. + +### Hosted contract follow-up: issue #525 + +PR #524 candidate `a4d61bbddbc1a0aa744aea72c1cff3c4ed165a72` +executed the new #512 contract on both hosted Windows lanes. CPU job +`94242642198` and Vulkan job `94242642222` both accepted the explicit empty +argument array, then failed at the exact non-empty record comparison with +`nonempty arguments did not arrive unchanged`. Both stopped in +`build-windows-release.ps1 -ContractTest` before configuration. The shared +diagnostic isolates the failure to the contract harness's temporary `.cmd` +argument recorder; it does not reopen the original empty-array binding defect +or establish a product build failure. + +The #525 repair replaces only the batch recorder with a native PowerShell +target whose parameter declaration accepts all remaining arguments and writes +a structured exact record. The real `Invoke-Checked` function must still invoke +that target for both the explicit empty array and the three literals `alpha`, +`two words`, and `--flag=value`. The exit-23 target and rejection assertion +remain live. Do not weaken an equality, normalize values, remove a behavior, +or infer a hosted pass from structural Linux coverage. + +RED-first coverage must reject the current `.cmd` recorder and require the +PowerShell recorder's remaining-arguments binding plus a structured exact +empty/non-empty record. Mutations that remove the remaining-arguments binding, +drop `two words`, bypass the real helper, or accept the nonzero child must make +the focused contract red. Focused green remains Windows metadata, release +pipeline, both direct Windows checkers, and full preflight. Hosted acceptance +requires both native Windows PR jobs to execute the complete contract and +continue into their MSVC build/runtime/archive gates. + +After #525 is integrated, current `origin/main` must be merged into the task +branch before the operator gate and plain push so the PR-size checker receives +an ancestor base. The HTTP 503 downloading glslang in job `94242642545` is an +external retry condition, not authorization to change the freshness gate. No +tag is authorized until PR #524 merges and a new exact-merged-SHA dry run has +all ten tuples plus aggregate handoff and verify green. + +The #525 implementation replaces the batch argument recorder with a temporary +PowerShell script. Its `ValueFromRemainingArguments` string-array parameter +defaults to an empty array, and it serializes the exact count and argument array +as compact JSON. The unchanged real `Invoke-Checked` helper executes that script +once with `@()` and once with `alpha`, `two words`, and `--flag=value`; the +contract parses each record and compares its exact count and values. The +separate exit-23 batch target and rejection assertion remain unchanged. + +RED-first coverage rejected the prior `record-arguments.cmd` implementation. +After the repair, mutations removing the remaining-arguments binding, dropping +`two words`, bypassing `Invoke-Checked`, or disabling the nonzero rejection each +failed the focused metadata contract, and the script was restored byte-for-byte. +The Windows metadata and release pipeline suites passed 50/50, followed by the +direct Windows portability and release-binary checkers and the full unstaged +repository preflight. The local host has no PowerShell runtime, so native +execution remains a required hosted Windows gate. + +### Native socket-runtime follow-up: issue #537 + +PR #524 candidate `d47d8408ab4dc2639e47ddfc7c7a997fa45d9981` +executed the complete #525 PowerShell contract successfully on hosted Windows. +CPU job +[`94265239385`](https://github.com/mudler/vllm.cpp/actions/runs/31641616323/job/94265239385) +then built `test_openai_api_server.exe` under unchanged `/W4 /WX`, served the +final request in the first real-socket smoke test, and terminated with signed +status `-1073740791` (`0xC0000409`) before doctest printed an assertion or +summary. + +The failure has two independently grounded causes. First, vendored +cpp-httplib v0.49.0 ends `process_and_close_socket` with an immediate +shutdown/close. Upstream commit `8e702d3837b2164765ca1d98cb6d180ae4711e70` +records that on Windows this can send an abortive RST when bytes are still in +flight, making a fully written response appear to the client as a failed read. +Its accepted repair half-closes the write side, drains at most 100 ms or 1 MiB, +then performs final shutdown/close. Second, each local real-socket test owns a +raw joinable `std::thread`, contains aborting `REQUIRE` assertions after that +thread starts, and stops/joins only on the normal path. When a Windows client +read is reset, doctest unwinds through the joinable thread and `std::terminate` +turns the useful assertion into the observed process-wide fast-fail. Upstream +commit `ae8356d86eabfd3ad4a969b55266fb3ecc2aa834` documents and repairs that exact +test-lifetime pattern with scoped teardown. + +The #537 implementation is limited to both complete repairs. Backport the +upstream accepted-socket drain helper and its single production call site with +the upstream 100 ms and 1 MiB bounds unchanged. Add one test-local scoped +server-thread owner and route every real-socket OpenAI test through it so +server stop and thread join happen both normally and while an assertion +unwinds. Preserve every existing request, status, body, concurrency, capacity, +route-gating, and shutdown assertion. Do not skip the Windows runtime target, +weaken `REQUIRE`, catch and discard a test failure, lengthen a timeout, detach a +thread, change release topology, or vendor unrelated upstream cpp-httplib +changes. + +RED-first evidence must demonstrate both defects against the pinned candidate. +A focused behavioral test must make the accepted-socket peer leave unread +request data and prove the old immediate close presents as a reset/failure on +Windows while the bounded drain preserves the completed response. A separate +test must intentionally fail after a server thread starts and prove scoped +teardown lets doctest report the assertion instead of terminating the process. +Structural coverage over the real vendored header and socket tests supplements, +but does not replace, those native behaviors. Mutations that restore the old +immediate accepted-socket close, remove either drain bound, or remove the scoped +stop/join path must make the focused gates red, with the tree restored +byte-for-byte afterwards. + +Focused green requires the OpenAI API server test, the cpp-httplib regression +test, the Windows release metadata/pipeline contracts, both direct Windows +checkers, and full unstaged/staged/post-commit preflight. Hosted acceptance +requires both Windows CPU and Vulkan lanes to execute the full OpenAI API test, +all other PR jobs to pass at one immutable head, and a fresh reviewer to mutate +both guarantees. After merge, the exact merged SHA must pass all ten release +tuples plus aggregate build and verify before `v0.0.3-pre.1` is tagged. The tag +run must then pass all 15 required jobs and the authenticated audit over exactly +32 assets. + +Stop with `NEEDS_DECISION` if the bounded upstream backport changes the public +HTTP API, broadens beyond accepted-socket close, or requires any release-gate +waiver. Stop with `NEEDS_CONTEXT` if Vulkan reports a different native failure; +file and specify that defect separately rather than folding it into #537. + +### Strict Vulkan-test follow-up: issue #540 + +The same immutable PR #524 candidate reached a different boundary in hosted +Windows Vulkan job +[`94265239433`](https://github.com/mudler/vllm.cpp/actions/runs/31641616323/job/94265239433). +The PowerShell contract and prior Windows portability fixes passed; MSVC then +rejected `tests/vt/test_backend_cross_device.cpp:525-529` under unchanged +`/W4 /WX`. The unbound-flash-layout CPU-oracle block redeclares `cpu`, `cq`, +`cd`, `ck`, `cv`, and `cslots`, shadowing names at lines 462-466 in the +enclosing `ReshapeAndCache` test. MSVC C4456 diagnoses all six. Git history +grounds the collision: `2c86f79ec` added the enclosing oracle and `822b3a2e15` +later added the nested oracle with the same short names. + +The #540 repair is test-only. Rename exactly those six inner declarations to +role-specific unbound-layout names and update only their uses in that nested +oracle. Preserve the data, types, tensor shapes and strides, CPU operation, +device loop, memcmp assertions, `/W4 /WX`, build targets, release topology, and +all production files. Do not suppress C4456, relax the warning gate, introduce +a compiler conditional, remove the nested oracle, or conflate this diagnostic +with #537. + +RED-first evidence is the hosted MSVC diagnostic above plus a focused local +contract over the real `ReshapeAndCache` test that rejects each of the six +shadowing inner declarations. Each independent mutation restoring one old name +must fail the focused contract, and the source must be restored byte-for-byte. +Focused green requires that contract, the direct Windows portability checker, +a clean local CPU compile/execution of `test_backend_cross_device`, and full +unstaged/staged/post-commit preflight. Hosted acceptance requires both native +Windows lanes to compile their complete unchanged targets; Vulkan must continue +through runtime, package, and archive validation. + +After both #537 and #540 pass fresh review and operator gates, PR #524 must be +plain-pushed at one exact SHA and all required PR checks must pass. The merged +SHA must then pass the complete ten-tuple non-publishing workflow before the +prerelease tag. Tag-run publication and the authenticated exactly-32-asset +audit remain mandatory. Stop with `NEEDS_DECISION` if a production edit or +warning-policy change is required; otherwise this is a bounded test hygiene +repair. + +#### #537 implementation outcome + +The implementation backports only cpp-httplib +`8e702d3837b2164765ca1d98cb6d180ae4711e70`: accepted plain HTTP sockets now +half-close writes, drain for at most 100 ms or 1 MiB, and then perform the +existing final shutdown and close. The helper remains internal to the vendored +header and has exactly one production call site. No public HTTP API, TLS close +path, timeout, or release topology changed. + +All 12 real-socket cases in `test_openai_api_server.cpp` now use one test-local +scoped owner that stops the server and joins its thread on normal return and +assertion unwind. A deliberately failing doctest fixture must exit with +doctest's normal status 1, print the named assertion, and emit its failure +summary; a subprocess harness rejects termination or any other exit status. +The accepted-socket regression leaves 64 KiB of unread request data after a +complete close-delimited request and requires the complete response followed by +an orderly EOF. + +The focused transport tests, the full OpenAI API server test, Windows release +metadata, both direct Windows checkers, and the 74-test portability suite pass +locally. Mutations restoring immediate close, changing either drain bound, +removing scoped stop, and removing scoped join each made the focused gates red; +the last two respectively timed out and reproduced `SIGABRT`. The mutated files +were restored byte-for-byte. Linux also passes the socket behavior, but the +native reset RED and final acceptance remain correctly pending the hosted +Windows CPU and Vulkan lanes; this local result is not substituted for them. + +#### #537 hosted falsification and next diagnostic + +Hosted CPU job +[`94287249909`](https://github.com/mudler/vllm.cpp/actions/runs/31648432555/job/94287249909) +and Vulkan job +[`94287249981`](https://github.com/mudler/vllm.cpp/actions/runs/31648432555/job/94287249981) +executed candidate `f52b547d44439e7cfb005f5697142b755ff65586` with both +repairs above and still terminated with `0xC0000409`. The log and doctest's +source-order execution place the failure in teardown of +`api_server: socket smoke — real HTTP requests over an ephemeral port`: its +final chat response completes, but doctest never prints that test's duration or +summary. Therefore the accepted-socket reset plus unscoped server thread was a +real defect, but it was not the complete root cause of this native fast-fail. +That earlier causal claim is rejected; its implementation remains required by +its focused regressions. + +The next hosted probe must preserve the full unchanged test and release gate, +add phase evidence around destruction of the `httplib::Client`, scoped server +thread, and `ServerHarness`, and install a diagnostic `std::terminate` marker in +the test process. It must run the isolated socket-smoke case first with doctest +success/duration output and then retain the normal full-suite invocation if the +probe survives. The probe may only add diagnostics; it must not skip an +assertion, detach a thread, catch a failure, alter production lifetime, or be +accepted as the fix. Use the first missing teardown marker plus the terminate +marker to identify one owner, then remove the probe and write a RED regression +for that exact lifetime defect before implementing a repair. If the isolated +case passes but the full process fails, bisect test-case prefixes in fresh +processes to prove the cross-test state dependency rather than guessing. + +Candidate `f06c77fe4af502a9934e525112008e1a02bdc1ff` executed that probe in +[run `31688115193`](https://github.com/mudler/vllm.cpp/actions/runs/31688115193). +CPU job `94408881944` and Vulkan job `94408881974` produced the same boundary: +the isolated socket case passed with all six destruction markers, while the +full 54-case process produced those same six markers and then fast-failed with +exact signed status `-1073740791` (`0xC0000409`) before the doctest summary. +Neither job emitted the diagnostic `std::terminate` marker. This rules out the +three instrumented owners as the final failing boundary and establishes a +source-order cross-test dependency as the next hypothesis. + +The developer approved one diagnostic-only adaptive prefix bisect. It lists the +54 doctest cases in file order, executes every probed `--first=1 --last=N` +prefix in a fresh process, requires the full prefix to reproduce only the exact +native fast-fail, and requires the first case as a known-good short prefix. It +then binary-searches the smallest failing `N`, confirms `N-1` succeeds and `N` +fast-fails in new processes, and runs only case `N` with matching first/last +bounds to distinguish an isolated defect from cumulative contamination. The +probe emits one stable diagnostic containing `N`, the listed test name, all +three confirmation statuses, and the dependency classification. The existing +unfiltered full-suite invocation remains unchanged and still runs afterwards. +Any other probe status stops the release job rather than being classified. + +Hosted run `31728014706` resolved that boundary to source-order case 47/54, +`api_server: an explicit-cpu device-selected engine serves /v1/completions`. +Both its prefix and its isolated invocation terminate with the exact native +fast-fail status, while the confirmed predecessor succeeds. The defect is +therefore isolated to that test or the production path it exercises, not a +cross-test lifetime dependency. + +The next diagnostic is limited to that isolated process. Preserve and print +its already-captured output, and add flushed phase witnesses around construction +of `LoadedEngine`, construction of the async serving stack, completion dispatch, +response validation, and scope teardown. Do not change timing, add sleeps, +weaken assertions, or accept this instrumentation as the repair. The hosted +Windows CPU/Vulkan result must identify the last completed phase. Then remove +the diagnostic and add the smallest RED regression for the actual owner before +changing production code. If the phase evidence does not distinguish an owner, +stop and extend the spec rather than guessing. + +Hosted CPU run `31732971268` printed only +`OPENAI_EXPLICIT_CPU_PHASE: before-loaded-engine` in both the isolated probe and +the unchanged full-suite invocation, then terminated with `0xC0000409`. It did +not print `after-loaded-engine`. This excludes the async serving stack, +completion dispatch, response validation, and teardown, but it does not yet +distinguish the three expressions in the construction statement: synthetic +weight creation, tokenizer-fixture creation, and the `LoadedEngine` constructor +itself. C++ does not impose a useful ordering between those argument +evaluations, so their absence cannot identify which one terminated. + +The final diagnostic split keeps the same test and values but materializes the +synthetic weights and tokenizer into named locals, with flushed witnesses before +and after each factory and before and after `LoadedEngine` construction. It +must move those locals into the constructor so ownership matches the original +by-value call. No production source, timing, assertion, environment, or engine +parameter may change. Hosted CPU and Vulkan must agree on the last completed +factory/construction phase. If both factories complete and construction still +fast-fails, stop and spec constructor-internal attribution rather than guessing; +otherwise remove every prefix/phase diagnostic and write the smallest RED +regression for the failing factory before repairing it. + +Hosted run +[`31737430012`](https://github.com/mudler/vllm.cpp/actions/runs/31737430012) +reached that stop condition. CPU job +[`94572345175`](https://github.com/mudler/vllm.cpp/actions/runs/31737430012/job/94572345175) +and Vulkan job +[`94572345101`](https://github.com/mudler/vllm.cpp/actions/runs/31737430012/job/94572345101) +agree in both the isolated probe and the unchanged full-suite invocation: each +printed `before-make-weights`, `after-make-weights`, `before-build-fixture`, +`after-build-fixture`, and `before-loaded-engine`, then fast-failed with signed +status `-1073740791` (`0xC0000409`) without printing `after-loaded-engine`. +Synthetic weight construction and tokenizer-fixture construction are therefore +excluded. The failure is inside `LoadedEngine` construction, before its first +constructor-body log, on both explicit CPU and Vulkan release builds. + +The next increment is diagnostic-only constructor-internal attribution. Enable +it explicitly only for the fresh-process isolated case-47 invocation; the +ordinary full-suite command and every production invocation remain unmodified +and do not enable the witnesses. Emit and flush a stable before/after phase for +the delegating `MakeQwen3_5MoeLoadedModel` call, then for every private +`LoadedEngine` initialization stage in declaration order through `engine_`: +`hash_ready_`, `config_`, `resolved_spec_config_`, `dflash_draft_`, `model_`, +the default `kv_connector_`, `tokenizer_`, `kv_cfg_`, `max_model_len_`, +`max_num_batched_tokens_`, `prefix_caching_enabled_`, +`jump_forward_enabled_`, `runner_`, `async_scheduling_enabled_`, +`max_concurrent_batches_`, `structured_output_manager_`, `scheduler_`, +`executor_`, `engine_core_`, `input_processor_`, `output_processor_`, +`block_hasher_`, and `engine_`. The contract must pin this sequence to the real +member-declaration order and require exactly one matching before/after pair for +every completed stage. Instrumentation must preserve every initializer value, +move, evaluation dependency, and construction order; it may not introduce a +replacement constructor path. + +Make the same isolated diagnostic dense below `runner_` so a failure there is +attributed in this hosted run rather than another coarse loop. Bracket the +`GPUModelRunner` member-initializer boundary through `input_batch_`, then its +constructor-body assignments, `async_input_combine_` resolution, pooling-model +branch, `initialize_kv_cache`, and `ModelRegistry::Prepare`. Inside +`initialize_kv_cache`, bracket the scalar/state-slot setup, KV-group scan, +Mamba shape/dtype validation, full-attention spec geometry, residency/buffer +setup, and each layer-indexed allocation family separately: GDN SSM storage, +GDN convolution storage, and full-attention storage. Also bracket creation of +the full-attention views, optional draft-attention storage, and GDN state views, +so success of an allocation cannot be confused with failure while viewing it. +Each nested marker names its function, stage, and layer or group index where +applicable. The first before marker lacking its matching after marker must +identify one exact initializer, function, or allocation/view family. + +These witnesses are temporary evidence, not a repair. They may not add an +assertion, sleep, timeout, catch, parameter change, device substitution, or +production semantic change. Because `0xC0000409` is an MSVC fail-fast and the +earlier `std::terminate` marker did not fire, the isolated Windows process must +also install a flushed `_set_invalid_parameter_handler` witness that reports +the CRT expression, function, file, and line. Install a similarly flushed +`_set_purecall_handler` witness when the existing MSVC runtime exposes it +without a new dependency. These Windows-only handlers are diagnostic: they may +observe but must not suppress or recover from the original failure, and a +surviving process restores the prior handlers before exit. If neither the +first incomplete phase nor a CRT witness names one owner, collect a native +Windows crash dump and symbolized stack from the same isolated workload instead +of inferring one. Before any actual repair, remove all adaptive-prefix, phase, +and CRT diagnostics, write the smallest automated regression that is RED for +the attributed defect, and observe that RED against the unfixed code. Only then +may a fresh implementation repair the root cause. + +The device-leakage failure reported beside these jobs is independent. Issue +[#553](https://github.com/mudler/vllm.cpp/issues/553) fixes it on current `main` +at `11cc1d5896b480a1b652db9249319242053aca93`; this campaign absorbs that change +when it merges current `main`. This #537 diagnostic must not edit that scope. + +### Current-main portability regression: issue #645 + +Current `main` at `cefacd2d00cb9b4776331cd213116773cd97f811` added LTX2 +sources containing the non-standard `M_PI` macro. The existing real-tree +Windows portability regression is RED and names exactly these three files: + +- `src/vllm/model_executor/models/ltx2.cpp` +- `src/vllm/model_executor/models/ltx2_video_vae.cpp` +- `src/vllm/model_executor/models/ltx2_audio_vae.cpp` + +The repair is limited to replacing those uses with a standard C++ constant +whose type and value preserve the current expressions. Do not define feature +macros, add a checker exemption, weaken the real-tree scan, or alter LTX2 +algorithms. The existing failing real-tree test is the RED regression; focused +green requires that test, the complete Windows portability suite, the combined +release script suite, and a clean CPU build covering the affected translation +units. Hosted MSVC CPU and Vulkan builds remain binding. + +Fresh review must mutate at least one repaired expression back to `M_PI` and +must perturb the replacement constant enough to prove an LTX2 numerical test +detects it. Stop with `NEEDS_DECISION` if preserving the current value requires +an algorithmic or tolerance change rather than a constant substitution. + +#### #645 implementation outcome + +The regression was the non-standard macro itself: `ltx2.cpp` consumed `M_PI`, +while both VAE files carried fallback definitions even though the video VAE did +not consume the macro. The C++20 production expressions now use +`std::numbers::pi_v`, which preserves the previous double type and +rounded value, and the unused video fallback is gone. No algorithm or tolerance +changed. + +Before the repair, the real-tree portability regression failed with exactly the +three specified files. Afterwards that regression and all 76 Windows +portability tests pass; the combined Windows metadata, release-pipeline, and +portability suite passes 130 tests. A CPU Release build compiled all three +affected translation units and linked `test_ltx2` and `test_ltx2_vae`; their +upstream-golden numerical gates pass 30/30 cases with 1627 assertions and 36/36 +cases with 3039 assertions, respectively. + +### Current-main VideoEngine portability regression: issue #648 + +The configured direct Windows checker on current `main` reports three sites in +`src/vllm/multimodal/video_engine.cpp`: the unconditional `` include +and the `::stat` calls in `IsDir` and `Exists`. These landed with the generalized +video seam at `cefacd2d00cb9b4776331cd213116773cd97f811` and cannot reach the +native MSVC release build. + +Replace that POSIX dependency with C++20 `std::filesystem` queries using the +non-throwing `std::error_code` overloads. `IsDir` remains true only for a +directory; `Exists` remains true for any existing filesystem entry; missing or +uninspectable paths remain false. Do not add a Windows-only branch, guard the +POSIX include, exempt the file from the checker, or change family resolution. + +RED evidence is the configured direct checker naming all three sites. Focused +behavior must cover an existing directory, a regular file, and a missing path +through the public VideoEngine resolution surface; add only the smallest test +needed if existing coverage cannot prove each classification. Green requires +the direct checker, the complete portability and combined release suites, a +clean build of the affected translation unit, and the VideoEngine tests. + +Fresh review must restore the POSIX implementation to prove the direct checker +is red and mutate directory classification to an existence-only query to prove +behavior coverage rejects a regular file. Stop with `NEEDS_DECISION` if the +portable implementation changes an observable resolution result or error. + +#### #648 implementation outcome + +The regression came from the generalized VideoEngine seam using POSIX `stat` +for two private classification helpers even though C++20 filesystem support is +already part of the project baseline. The direct portability checker failed at +the unconditional `` include and both `::stat` calls before the +repair. `IsDir` now uses the non-throwing `std::filesystem::is_directory` +overload and `Exists` uses the corresponding `exists` overload; both inspect an +`std::error_code`, so missing and uninspectable paths still return false rather +than throwing. + +The focused public-surface case distinguishes a directory without a shard +index, a regular non-checkpoint file, and a missing path through +`ReadVideoCheckpointTensorNames`. This pins the existing three refusal reasons +and makes an existence-only directory predicate misclassify the regular file. +After the repair the direct checker passed, the VideoEngine target built and +its 12 cases / 260 assertions passed, and the combined Windows metadata, +release-pipeline, and portability suite passed all 130 tests. No family +resolution rule or public error changed. + +Fresh mutation review of the adaptive probe found three false-green contract +gaps: the listing fixture did not distinguish file order from name order, the +unexpected-status branches at an intermediate midpoint and isolated probe were +not executed, and the emitted diagnostic field names were not pinned. The +repaired live PowerShell contract uses a deliberately name-order-inverted +source-order listing, injects status 7 independently at the midpoint and +isolated boundaries and requires both invocations to throw, and captures the +real host output for an exact diagnostic-schema comparison. Mutating the +listing call to name order, accepting either unexpected status, or renaming +`predecessor_status` now makes the contract red. The restored candidate passes +the 12-test Windows metadata suite, the combined 130-test metadata/pipeline/ +portability suite, the live PowerShell contract, and all four direct Windows +release checkers. + +### Cross-platform PowerShell contract follow-up: issue #599 + +Fresh review of diagnostic candidate `5a845c28a7a9afe5addf94771e59a71cecd31e81` +ran the required direct Windows portability checker on a Linux host with +PowerShell Core installed. `Invoke-CheckedContractTests` created `fail.cmd` and +invoked that path directly. Non-Windows PowerShell handed the batch file to +`gio` instead of a Windows command interpreter; `gio` reported the unusable +path but returned success, so the contract failed at its own guard with +`nonzero child exit was accepted`. The repository preflight did not substitute +for this direct checker and its green result is not evidence for this boundary. + +The #599 repair is contract-test only. Replace the platform-specific failing +child fixture with a script directly executable by both Windows PowerShell and +PowerShell Core on non-Windows hosts. Preserve the real-process invocation, +exact exit status 23, zero- and non-empty-argument forwarding proofs, production +`Invoke-Checked` behavior, native release gate, and release topology. Do not +mock this boundary, special-case the checker host, skip the nonzero child, or +accept a desktop-opener status. + +RED-first evidence is the direct checker failure above. Focused coverage must +pin the portable fixture and its exact exit 23, then run the actual +`-ContractTest` path under installed PowerShell Core. Mutating the fixture back +to `.cmd`, changing its exit status, or bypassing the real child invocation must +make the focused gate red. Focused green requires Windows release metadata and +pipeline tests, the complete Windows portability unit suite, the direct Windows +portability and release-binary checkers, and full unstaged/staged/post-commit +preflight. Hosted CPU and Vulkan execution remains required because local +PowerShell Core does not prove Windows process semantics. + +Stop with `NEEDS_DECISION` if the repair would change production invocation or +the native release gate rather than only its contract fixture. + +#### #599 implementation outcome + +The failure was confined to the contract fixture: PowerShell Core 7.6.4 on +Linux dispatched the temporary `fail.cmd` through `gio`, whose success status +made the real `Invoke-Checked` helper accept the supposed failure child. The +fixture is now a temporary PowerShell script containing only `exit 23`; both +Windows PowerShell and cross-platform PowerShell execute that script directly. +Production `Invoke-Checked`, its argument splatting, the release gate, and the +workflow topology are unchanged. + +Before the repair, the focused metadata contract failed on `fail.cmd` and the +live `-ContractTest` failed with `nonzero child exit was accepted`. After the +repair, the live PowerShell contract, all nine Windows metadata tests, all 42 +release-pipeline tests, all 76 Windows-portability tests, and both direct +Windows checkers passed. Independently restoring the batch fixture, changing +the child to exit 24, and bypassing the real child invocation each made both +the focused structural and live-process tests fail; restoration returned the +script and test to SHA-256 values `bc98e85ec59b076e04c490b23d00c41b9a8fe57277d2ba051f4fc0d3f65c569e` +and `cb81cc1b9914306081b0861929e907cf6d50310b8a6dde7f4d27aeae40700df8`. +Native Windows CPU and Vulkan acceptance remains pending the hosted PR jobs. + +Fresh review of `bb3a4ef409d372b7f22698b8b530b4cb7f953cb9` disproved the +outcome's claim that direct invocation of the portable `.ps1` fixture preserved +a real-process boundary: the parent and invoked fixture both recorded PID +`3526837`. Direct `.ps1` invocation creates another PowerShell scope in the +same host process, so the earlier process-boundary claim is rejected. + +The follow-up resolves the running host through +`[System.Diagnostics.Process]::GetCurrentProcess().MainModule.FileName`, which +is available to Windows PowerShell 5.1 as well as PowerShell Core, and passes +that executable plus `-NoProfile -NonInteractive -File` through the unchanged +`Invoke-Checked` helper for every fixture. The failing fixture records its PID +before exact exit 23, and the live contract rejects equality with the parent +PID. Before the repair that assertion failed with `failure target did not +execute in a child process`; after it, the live contract, all 9 Windows metadata +tests, all 42 release-pipeline tests, all 76 Windows-portability tests, and the +four direct Windows release checkers pass. Mutating the failure launch back to +direct `.ps1`, changing exit 23 to exit 24, or bypassing `Invoke-Checked` makes +both structural and live-process coverage red. Native hosted Windows remains +the final authority for Windows process semantics. + +Fresh review of `8dfddfed2b27f6830768acbe32803c1bb7399459` found that the +external live Python test still trusted the contract's in-script PID guard. +Deleting that guard left the live method green, so the claimed child-process +evidence was not independently checked outside the script. The strengthened +contract emits one stable diagnostic containing only the parent and failing +child integer PIDs. The live Python method parses both values, requires each to +be positive, and independently rejects equality; it also pins the exact runtime +equality guard so removing either layer is red. + +Before the test change, deleting the runtime guard reproduced the false green. +The strengthened test was then RED because the diagnostic did not yet exist. +After adding the diagnostic only to `Invoke-CheckedContractTests`, the direct +PowerShell contract and live Python method pass with distinct PIDs. Removing +the runtime guard, launching the failing `.ps1` directly, changing exit 23 to +exit 24, and bypassing `Invoke-Checked` each made the focused live method red; +each mutation was restored before the next. All 9 Windows metadata tests, all +42 release-pipeline tests, all 76 Windows-portability tests, and the four direct +Windows release checkers pass. Production `Invoke-Checked`, the release gate, +and release topology remain unchanged; native hosted Windows is still binding. + +#### #540 implementation outcome + +Implementation evidence: the focused contract was red with all six old +declarations present (12 subtest failures), then green after the nested locals +were renamed to `unbound_cpu`, `unbound_queue`, `unbound_device`, `unbound_k`, +`unbound_v`, and `unbound_slots`. Restoring each old declaration independently +made its named contract checks red, with both source files restored to their +recorded hashes afterwards. The 73-test Windows portability suite, direct +Windows portability checker, and a clean CPU-only Release build and execution +of `test_backend_cross_device` (19 cases, 6 assertions) passed. Native MSVC +acceptance remains pending the hosted Windows CPU and Vulkan lanes. + +Fresh review found that the negative checks recognized only six exact +declaration spellings. The equivalent comma declarator +`const Device unbound_device{...}, cd(DeviceType::kCPU, 0);` therefore passed +the old contract. The follow-up contract derives declarator names within the +exact unbound oracle while respecting comma, brace, and parenthesis nesting. +That bypass and independent mutations for all six forbidden names now fail; +the restored tree passes all 74 Windows portability tests and the direct +checker. diff --git a/docs/FEATURES.md b/docs/FEATURES.md index 26317660c..cd7de747d 100644 --- a/docs/FEATURES.md +++ b/docs/FEATURES.md @@ -27,7 +27,7 @@ are our reading of their documented behavior, not measurements. | Correctness gate | token-exact vs vLLM | reference | own | own | | Architectures | 38 registered, 27 gated | 130+ | 100+ | 100+ | | Downloadable server binaries | ✅ v0.0.2: eight indexed archives with checksums, provenance, manifests, and SBOMs. Windows ZIP downloads do not exist; native CPU/Vulkan lanes await hosted runtime, dry-run, prerelease, and authenticated audit gates | ✅ wheels/containers | ✅ wheels/containers | ✅ host-specific binaries | -| Native Windows builds | ◐ CPU/Vulkan: `/MT /W4 /WX`, central `NOMINMAX`, UTF-8, aligned allocation, runtime ISA dispatch. Local closure includes the float-domain DeepSeek probe; hosted compile/runtime/release pending | ✅ | ✅ | ✅ | +| Native Windows builds | ◐ CPU/Vulkan: `/MT /W4 /WX`, `NOMINMAX`, UTF-8, aligned allocation, runtime ISA dispatch. Issue #537 has an isolated ordered constructor/KV/CRT witness. Local float probe green; hosted compile/runtime/release pending | ✅ | ✅ | ✅ | ## Serving and scheduling diff --git a/docs/USAGE.md b/docs/USAGE.md index 4f80a43ef..80456bc98 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -694,6 +694,16 @@ imports before running the staged executable's `--help`, forced-tier, or HTTP shutdown smokes. The Win32 console-control regression uses bounded waits so a teardown failure reports an error instead of hanging the gate. +While issue #537 is being attributed, the Windows release helper also runs an +internal constructor witness only in the fresh-process isolated OpenAI case +found by its prefix probe. The helper restores the process environment before +continuing; the ordinary prefix probes, complete OpenAI suite, staged server, +and production invocations do not enable or inherit that diagnostic. Its +strictly ordered before/after markers cover guaranteed-elided model-factory +construction, engine and runner member construction, and each indexed KV-cache +allocation; Windows invalid-parameter and purecall faults flush their own marker +before terminating through the CRT fallback. + The CUDA graph-replay profiler and its FIFO diagnostic controls remain POSIX-only and are not exposed by native Windows server builds. Native Windows process launch, environment updates, process IDs, and console shutdown stay on diff --git a/scripts/build-windows-release.ps1 b/scripts/build-windows-release.ps1 index 7c339dbcf..eb000a6ff 100644 --- a/scripts/build-windows-release.ps1 +++ b/scripts/build-windows-release.ps1 @@ -19,66 +19,89 @@ if ($ArtifactId -ne "windows-x86_64-msvc-$Backend") { } function Invoke-Checked { param([Parameter(Mandatory)][string]$Program, - [Parameter(Mandatory)][AllowEmptyCollection()][string[]]$Arguments, - [scriptblock]$Runner) - if ($null -eq $Runner) { - & $Program @Arguments - $exitCode = $LASTEXITCODE - } else { - $exitCode = [int](& $Runner $Program $Arguments) - } - if ($exitCode -ne 0) { - throw "$Program exited with status $exitCode" + [Parameter(Mandatory)][AllowEmptyCollection()][string[]]$Arguments) + & $Program @Arguments + if ($LASTEXITCODE -ne 0) { + throw "$Program exited with status $LASTEXITCODE" } } -# Most of this script's checked invocations run a test executable that takes no -# arguments, so `Invoke-Checked` must bind an explicitly empty argument list and -# still forward it verbatim (#512). function Invoke-CheckedContractTests { - $calls = [System.Collections.Generic.List[object]]::new() - $recorder = { - param([string]$Program, [string[]]$Arguments) - $calls.Add([pscustomobject]@{ - Program = $Program - Arguments = @($Arguments) - }) | Out-Null - return 0 - }.GetNewClosure() + $temporaryDir = Join-Path ([System.IO.Path]::GetTempPath()) ` + "vllm-invoke-checked-$([guid]::NewGuid().ToString('N'))" + $recordingTarget = Join-Path $temporaryDir "record-arguments.ps1" + $failingTarget = Join-Path $temporaryDir "fail.ps1" + $callLog = Join-Path $temporaryDir "calls.txt" + $savedCallLog = $env:VLLM_INVOKE_CHECKED_LOG + $powerShellExecutable = [System.Diagnostics.Process]::GetCurrentProcess().MainModule.FileName + try { + New-Item -ItemType Directory -Path $temporaryDir | Out-Null + @' +param( + [Parameter(ValueFromRemainingArguments = $true)] + [string[]]$RemainingArguments = @() +) +[pscustomobject]@{ + Count = @($RemainingArguments).Count + Arguments = @($RemainingArguments) +} | ConvertTo-Json -Compress | Set-Content ` + -LiteralPath $env:VLLM_INVOKE_CHECKED_LOG -Encoding utf8 +exit 0 +'@ | Set-Content -LiteralPath $recordingTarget -Encoding utf8 + @' +[string]$PID | Set-Content ` + -LiteralPath $env:VLLM_INVOKE_CHECKED_LOG -Encoding ascii +exit 23 +'@ | Set-Content -LiteralPath $failingTarget -Encoding utf8 + $env:VLLM_INVOKE_CHECKED_LOG = $callLog + $parentProcessId = $PID - Invoke-Checked "fake-empty.exe" @() -Runner $recorder - Invoke-Checked "fake-args.exe" @("--help", "--verbose") -Runner $recorder + Invoke-Checked $powerShellExecutable @( + "-NoProfile", "-NonInteractive", "-File", $recordingTarget) + $zeroArgumentRecord = Get-Content -LiteralPath $callLog -Raw | ConvertFrom-Json + if ([int]$zeroArgumentRecord.Count -ne 0 -or + @($zeroArgumentRecord.Arguments).Count -ne 0) { + throw "zero-argument target was not invoked exactly once without arguments" + } - if ($calls.Count -ne 2) { - throw "checked-invocation fake runner was not invoked exactly twice" - } - if ($calls[0].Program -ne "fake-empty.exe" -or $calls[1].Program -ne "fake-args.exe") { - throw "checked invocation did not forward its exact program" - } - if ($calls[0].Arguments.Count -ne 0) { - throw "checked invocation did not forward an explicitly empty argument list" - } - if ($calls[1].Arguments.Count -ne 2 -or - $calls[1].Arguments[0] -ne "--help" -or - $calls[1].Arguments[1] -ne "--verbose") { - throw "checked invocation did not forward its exact argument list" - } + Remove-Item -LiteralPath $callLog + Invoke-Checked $powerShellExecutable @( + "-NoProfile", "-NonInteractive", "-File", $recordingTarget, + "alpha", "two words", "--flag=value") + $nonemptyArgumentRecord = Get-Content -LiteralPath $callLog -Raw | ConvertFrom-Json + if ([int]$nonemptyArgumentRecord.Count -ne 3 -or + @($nonemptyArgumentRecord.Arguments).Count -ne 3 -or + $nonemptyArgumentRecord.Arguments[0] -cne "alpha" -or + $nonemptyArgumentRecord.Arguments[1] -cne "two words" -or + $nonemptyArgumentRecord.Arguments[2] -cne "--flag=value") { + throw "nonempty arguments did not arrive unchanged" + } - $failing = { param([string]$Program, [string[]]$Arguments) return 3 } - foreach ($rejectedName in @("empty", "non-empty")) { $rejected = $false try { - if ($rejectedName -eq "empty") { - Invoke-Checked "fake-fail.exe" @() -Runner $failing - } else { - Invoke-Checked "fake-fail.exe" @("--help") -Runner $failing - } + Invoke-Checked $powerShellExecutable @( + "-NoProfile", "-NonInteractive", "-File", $failingTarget) } catch { + if ($_.Exception.Message -notmatch 'exited with status 23') { + throw + } $rejected = $true } if (-not $rejected) { - throw "nonzero $rejectedName-argument exit status was accepted" + throw "nonzero child exit was accepted" + } + $failingChildProcessId = [int](Get-Content -LiteralPath $callLog -Raw) + Write-Host "Invoke-Checked PID contract: parent=$parentProcessId failure_child=$failingChildProcessId" + if ($failingChildProcessId -eq $parentProcessId) { + throw "failure target did not execute in a child process" } + } finally { + if ($null -eq $savedCallLog) { + Remove-Item Env:VLLM_INVOKE_CHECKED_LOG -ErrorAction SilentlyContinue + } else { + $env:VLLM_INVOKE_CHECKED_LOG = $savedCallLog + } + Remove-Item -Recurse -Force $temporaryDir -ErrorAction SilentlyContinue } } @@ -212,10 +235,384 @@ function Invoke-UnsupportedTierContractTests { } } +function Invoke-OpenAiProbeProcess { + param([Parameter(Mandatory)][string]$Program, + [Parameter(Mandatory)][AllowEmptyCollection()][string[]]$Arguments, + [scriptblock]$Runner) + if ($null -eq $Runner) { + $output = @(& $Program @Arguments 2>&1) + return [pscustomobject]@{ + ExitCode = [int]$LASTEXITCODE + Output = @($output) + } + } + $result = & $Runner $Program $Arguments + if ($null -eq $result) { + throw "OpenAI prefix probe runner returned no result" + } + return [pscustomobject]@{ + ExitCode = [int]$result.ExitCode + Output = @($result.Output) + } +} + +function Invoke-OpenAiPrefixRange { + param([Parameter(Mandatory)][string]$Program, + [Parameter(Mandatory)][int]$First, + [Parameter(Mandatory)][int]$Last, + [scriptblock]$Runner) + return Invoke-OpenAiProbeProcess -Program $Program -Arguments @( + "--order-by=file", + "--first=$First", + "--last=$Last", + "--success=true", + "--duration=true", + "--no-colors=true" + ) -Runner $Runner +} + +function Invoke-OpenAiPrefixBisect { + param([Parameter(Mandatory)][string]$TestProgram, + [scriptblock]$Runner) + $expectedFastFailStatus = -1073740791 + $listResult = Invoke-OpenAiProbeProcess -Program $TestProgram -Arguments @( + "--list-test-cases", + "--order-by=file", + "--no-version=true", + "--no-colors=true" + ) -Runner $Runner + if ($listResult.ExitCode -ne 0) { + throw "OpenAI test listing exited with status $($listResult.ExitCode)" + } + + $testCases = [System.Collections.Generic.List[string]]::new() + $separatorCount = 0 + foreach ($outputLine in @($listResult.Output)) { + $line = [string]$outputLine + if ($line -match '^=+$') { + $separatorCount++ + if ($separatorCount -eq 2) { break } + continue + } + if ($separatorCount -eq 1 -and $line.Length -gt 0) { + $testCases.Add($line) | Out-Null + } + } + if ($separatorCount -ne 2 -or $testCases.Count -lt 2) { + throw "OpenAI test listing did not contain a complete source-order case list" + } + $testCount = $testCases.Count + + $fullPrefixResult = Invoke-OpenAiPrefixRange -Program $TestProgram ` + -First 1 -Last $testCount -Runner $Runner + if ($fullPrefixResult.ExitCode -ne $expectedFastFailStatus) { + throw "OpenAI full prefix exited with status $($fullPrefixResult.ExitCode) instead of $expectedFastFailStatus" + } + $shortPrefixResult = Invoke-OpenAiPrefixRange -Program $TestProgram ` + -First 1 -Last 1 -Runner $Runner + if ($shortPrefixResult.ExitCode -ne 0) { + throw "OpenAI first-case prefix exited with status $($shortPrefixResult.ExitCode) instead of 0" + } + + $low = 1 + $high = $testCount + while ($high - $low -gt 1) { + $mid = [int][math]::Floor(($low + $high) / 2) + $prefixResult = Invoke-OpenAiPrefixRange -Program $TestProgram ` + -First 1 -Last $mid -Runner $Runner + if ($prefixResult.ExitCode -eq 0) { + $low = $mid + } elseif ($prefixResult.ExitCode -eq $expectedFastFailStatus) { + $high = $mid + } else { + throw "OpenAI prefix 1..$mid exited with unexpected status $($prefixResult.ExitCode)" + } + } + + $firstBad = $high + $predecessorResult = Invoke-OpenAiPrefixRange -Program $TestProgram ` + -First 1 -Last ($firstBad - 1) -Runner $Runner + if ($predecessorResult.ExitCode -ne 0) { + throw "OpenAI confirmed predecessor prefix exited with status $($predecessorResult.ExitCode) instead of 0" + } + $badPrefixResult = Invoke-OpenAiPrefixRange -Program $TestProgram ` + -First 1 -Last $firstBad -Runner $Runner + if ($badPrefixResult.ExitCode -ne $expectedFastFailStatus) { + throw "OpenAI confirmed bad prefix exited with status $($badPrefixResult.ExitCode) instead of $expectedFastFailStatus" + } + $diagnosticEnvironmentName = "VLLM_WINDOWS_CTOR_DIAGNOSTIC" + $priorDiagnosticEnvironment = [Environment]::GetEnvironmentVariable( + $diagnosticEnvironmentName, "Process") + try { + [Environment]::SetEnvironmentVariable($diagnosticEnvironmentName, "1", "Process") + $isolatedResult = Invoke-OpenAiPrefixRange -Program $TestProgram ` + -First $firstBad -Last $firstBad -Runner $Runner + } finally { + [Environment]::SetEnvironmentVariable($diagnosticEnvironmentName, $priorDiagnosticEnvironment, "Process") + } + Write-Host "OpenAI isolated output: begin case=$firstBad" + foreach ($isolatedOutputLine in @($isolatedResult.Output)) { + Write-Host ([string]$isolatedOutputLine) + } + Write-Host "OpenAI isolated output: end case=$firstBad" + if ($isolatedResult.ExitCode -eq 0) { + $dependency = "cumulative" + } elseif ($isolatedResult.ExitCode -eq $expectedFastFailStatus) { + $dependency = "isolated" + } else { + throw "OpenAI isolated case $firstBad exited with unexpected status $($isolatedResult.ExitCode)" + } + + $testName = $testCases[$firstBad - 1] + Write-Host "OpenAI prefix bisect: first_bad=$firstBad/$testCount test=`"$testName`" predecessor_status=$($predecessorResult.ExitCode) prefix_status=$($badPrefixResult.ExitCode) isolated_status=$($isolatedResult.ExitCode) dependency=$dependency" + return [pscustomobject]@{ + FirstBad = $firstBad + TestCount = $testCount + TestName = $testName + Dependency = $dependency + } +} + +function Invoke-OpenAiPrefixBisectContractTests { + $firstBad = 27 + $testCount = 54 + $caseNames = @( + foreach ($index in 1..$testCount) { + if ($index -eq $firstBad) { + "aaa source case $index" + } elseif ($index -eq 1) { + "zzz source case $index" + } else { + "source case $index" + } + } + ) + if (@($caseNames | Sort-Object)[0] -ceq $caseNames[0]) { + throw "OpenAI prefix bisect fixture does not distinguish file order from name order" + } + $listing = [System.Collections.Generic.List[string]]::new() + $listing.Add("[doctest] listing all test case names") | Out-Null + $listing.Add("===============================================================================") | Out-Null + foreach ($caseName in $caseNames) { + $listing.Add($caseName) | Out-Null + } + $listing.Add("===============================================================================") | Out-Null + $listing.Add("[doctest] unskipped test cases passing the current filters: $testCount") | Out-Null + + $calls = [System.Collections.Generic.List[object]]::new() + $runner = { + param([string]$Program, [string[]]$Arguments) + $calls.Add([pscustomobject]@{ + Program = $Program + Arguments = @($Arguments) + DiagnosticEnvironment = [Environment]::GetEnvironmentVariable( + "VLLM_WINDOWS_CTOR_DIAGNOSTIC", "Process") + }) | Out-Null + if ($Arguments -contains "--list-test-cases") { + $orderArguments = @($Arguments | Where-Object { $_ -like "--order-by=*" }) + if ($orderArguments.Count -ne 1 -or + $orderArguments[0] -cne "--order-by=file") { + return [pscustomobject]@{ + ExitCode = 7 + Output = @($caseNames | Sort-Object) + } + } + return [pscustomobject]@{ ExitCode = 0; Output = @($listing) } + } + + $firstArgument = @($Arguments | Where-Object { $_ -like "--first=*" }) + $lastArgument = @($Arguments | Where-Object { $_ -like "--last=*" }) + if ($firstArgument.Count -ne 1 -or $lastArgument.Count -ne 1 -or + $Arguments -notcontains "--order-by=file") { + return [pscustomobject]@{ ExitCode = 2; Output = @("bad range arguments") } + } + $first = [int]$firstArgument[0].Substring("--first=".Length) + $last = [int]$lastArgument[0].Substring("--last=".Length) + $exitCode = if ($first -eq 1 -and $last -ge $firstBad) { + -1073740791 + } else { + 0 + } + $output = if ($first -eq $firstBad -and $last -eq $firstBad) { + @("OPENAI_EXPLICIT_CPU_PHASE: fixture-isolated") + } else { + @() + } + return [pscustomobject]@{ ExitCode = $exitCode; Output = @($output) } + }.GetNewClosure() + + $contractDiagnosticEnvironment = [Environment]::GetEnvironmentVariable( + "VLLM_WINDOWS_CTOR_DIAGNOSTIC", "Process") + try { + [Environment]::SetEnvironmentVariable( + "VLLM_WINDOWS_CTOR_DIAGNOSTIC", "fixture-prior", "Process") + $captured = @(& { + Invoke-OpenAiPrefixBisect -TestProgram "fake-openai-test.exe" ` + -Runner $runner + } 6>&1) + if ([Environment]::GetEnvironmentVariable( + "VLLM_WINDOWS_CTOR_DIAGNOSTIC", "Process") -cne + "fixture-prior") { + throw "OpenAI constructor diagnostic environment was not restored" + } + } finally { + [Environment]::SetEnvironmentVariable( + "VLLM_WINDOWS_CTOR_DIAGNOSTIC", $contractDiagnosticEnvironment, + "Process") + } + Write-Host "OpenAI constructor diagnostic restoration contract OK" + $diagnosticCalls = @($calls | Where-Object { + $_.DiagnosticEnvironment -ceq "1" + }) + if ($diagnosticCalls.Count -ne 1 -or + $diagnosticCalls[0].Arguments -notcontains "--first=27" -or + $diagnosticCalls[0].Arguments -notcontains "--last=27") { + throw "OpenAI constructor diagnostic was not limited to the isolated case" + } + $ordinaryDiagnosticCalls = @($calls | Where-Object { + $_.DiagnosticEnvironment -ne "1" + }) + if ($ordinaryDiagnosticCalls.Count -ne ($calls.Count - 1)) { + throw "OpenAI constructor diagnostic leaked into an ordinary probe" + } + Write-Host "OpenAI constructor diagnostic activation contract OK" + $results = @($captured | Where-Object { + $null -ne $_.PSObject.Properties["FirstBad"] + }) + if ($results.Count -ne 1) { + throw "OpenAI prefix bisect did not return exactly one result" + } + $result = $results[0] + if ($result.FirstBad -ne $firstBad -or + $result.TestCount -ne $testCount -or + $result.TestName -cne "aaa source case 27" -or + $result.Dependency -cne "cumulative") { + throw "OpenAI prefix bisect returned the wrong boundary" + } + $listCalls = @($calls | Where-Object { + $_.Arguments -contains "--list-test-cases" + }) + if ($listCalls.Count -ne 1) { + throw "OpenAI prefix bisect did not list tests exactly once" + } + if ($listCalls[0].Arguments.Count -ne 4 -or + $listCalls[0].Arguments[0] -cne "--list-test-cases" -or + $listCalls[0].Arguments[1] -cne "--order-by=file" -or + $listCalls[0].Arguments[2] -cne "--no-version=true" -or + $listCalls[0].Arguments[3] -cne "--no-colors=true") { + throw "OpenAI prefix bisect listing did not request exact file order" + } + Write-Host "OpenAI prefix bisect listing order contract OK" + foreach ($call in @($calls | Where-Object { + $_.Arguments -notcontains "--list-test-cases" + })) { + $firstArguments = @($call.Arguments | Where-Object { $_ -like "--first=*" }) + $lastArguments = @($call.Arguments | Where-Object { $_ -like "--last=*" }) + if ($firstArguments.Count -ne 1 -or $lastArguments.Count -ne 1 -or + $call.Arguments -notcontains "--order-by=file") { + throw "OpenAI prefix bisect emitted a non-source-order range probe" + } + } + foreach ($bounds in @( + @("--first=1", "--last=26"), + @("--first=1", "--last=27"), + @("--first=27", "--last=27") + )) { + $matches = @($calls | Where-Object { + $_.Arguments -contains $bounds[0] -and + $_.Arguments -contains $bounds[1] + }) + if ($matches.Count -lt 1) { + throw "OpenAI prefix bisect omitted confirmation $($bounds -join '..')" + } + } + $forwardedOutput = @( + "OpenAI isolated output: begin case=27", + "OPENAI_EXPLICIT_CPU_PHASE: fixture-isolated", + "OpenAI isolated output: end case=27" + ) + foreach ($line in $forwardedOutput) { + if (@($captured | Where-Object { [string]$_ -ceq $line }).Count -ne 1) { + throw "OpenAI prefix bisect did not forward isolated output exactly once: $line" + } + } + Write-Host "OpenAI prefix bisect isolated output forwarding contract OK" + + $midpointInjected = $false + $unexpectedMidpointRunner = { + param([string]$Program, [string[]]$Arguments) + if ($Arguments -notcontains "--list-test-cases") { + $firstArgument = @($Arguments | Where-Object { $_ -like "--first=*" }) + $lastArgument = @($Arguments | Where-Object { $_ -like "--last=*" }) + if ($firstArgument.Count -eq 1 -and $lastArgument.Count -eq 1 -and + $firstArgument[0] -ceq "--first=1" -and + $lastArgument[0] -ceq "--last=27" -and + -not $midpointInjected) { + $midpointInjected = $true + return [pscustomobject]@{ ExitCode = 7; Output = @("injected midpoint failure") } + } + } + return & $runner $Program $Arguments + }.GetNewClosure() + $midpointRejected = $false + try { + Invoke-OpenAiPrefixBisect -TestProgram "fake-openai-test.exe" ` + -Runner $unexpectedMidpointRunner | Out-Null + } catch { + if ($_.Exception.Message -cne + "OpenAI prefix 1..27 exited with unexpected status 7") { + throw + } + $midpointRejected = $true + } + if (-not $midpointRejected) { + throw "OpenAI prefix bisect accepted an unexpected midpoint status" + } + Write-Host "OpenAI prefix bisect unexpected midpoint status contract OK" + + $unexpectedIsolatedRunner = { + param([string]$Program, [string[]]$Arguments) + if ($Arguments -notcontains "--list-test-cases" -and + $Arguments -contains "--first=27" -and + $Arguments -contains "--last=27") { + return [pscustomobject]@{ ExitCode = 7; Output = @("injected isolated failure") } + } + return & $runner $Program $Arguments + }.GetNewClosure() + $isolatedRejected = $false + try { + Invoke-OpenAiPrefixBisect -TestProgram "fake-openai-test.exe" ` + -Runner $unexpectedIsolatedRunner | Out-Null + } catch { + if ($_.Exception.Message -cne + "OpenAI isolated case 27 exited with unexpected status 7") { + throw + } + $isolatedRejected = $true + } + if (-not $isolatedRejected) { + throw "OpenAI prefix bisect accepted an unexpected isolated status" + } + Write-Host "OpenAI prefix bisect unexpected isolated status contract OK" + + $expectedDiagnostic = 'OpenAI prefix bisect: first_bad=27/54 test="aaa source case 27" predecessor_status=0 prefix_status=-1073740791 isolated_status=0 dependency=cumulative' + $diagnostics = @($captured | Where-Object { + ([string]$_).StartsWith("OpenAI prefix bisect: first_bad=") + }) + if ($diagnostics.Count -ne 1 -or + [string]$diagnostics[0] -cne $expectedDiagnostic) { + throw "OpenAI prefix bisect emitted an unstable diagnostic schema" + } + Write-Host "OpenAI prefix bisect diagnostic schema contract OK" + Write-Host $expectedDiagnostic + Write-Host "OpenAI prefix bisect contract OK" +} + if ($ContractTest) { Invoke-CheckedContractTests Invoke-CrtContractTests Invoke-UnsupportedTierContractTests + Invoke-OpenAiPrefixBisectContractTests Write-Host "Windows PowerShell/CRT contract tests OK" exit 0 } @@ -274,6 +671,14 @@ if ($Backend -eq "vulkan") { } Invoke-Checked cmake (@("--build", $BuildDir, "--config", "Release", "--target") + $targets) +$openaiApiServerTest = Join-Path $BuildDir "tests/Release/test_openai_api_server.exe" +Invoke-Checked $openaiApiServerTest @( + "--test-case=api_server: socket smoke *real HTTP requests over an ephemeral port", + "--success=true", + "--duration=true" +) +Invoke-OpenAiPrefixBisect -TestProgram $openaiApiServerTest + foreach ($test in @( "test_openai_api_server.exe", "test_lmcache_client.exe", diff --git a/scripts/cpu-x86-llamacpp-floor.sh b/scripts/cpu-x86-llamacpp-floor.sh index bd7a51925..029cdd4c0 100755 --- a/scripts/cpu-x86-llamacpp-floor.sh +++ b/scripts/cpu-x86-llamacpp-floor.sh @@ -97,17 +97,18 @@ builders() { # "busy" and "total" jiffies across all cpus. iowait is NOT counted as busy: a # neighbour waiting on disk does not steal our cores. steal IS counted -- on a # KVM guest that is exactly the co-tenant this box keeps losing series to. -stat_busy() { awk '/^cpu /{print $2+$3+$4+$7+$8+$9}' /proc/stat; } -stat_total() { awk '/^cpu /{s=0; for(i=2;i<=NF;i++)s+=$i; print s}' /proc/stat; } +stat_sample() { + awk '/^cpu /{busy=$2+$3+$4+$7+$8+$9; for(i=2;i<=NF;i++) total+=$i; print busy, total}' /proc/stat +} # Percent of the whole machine busy over a fresh BUSY_WINDOW. Nothing here is # decayed and nothing here counts a process that has already exited, so the # harness cannot gate on its own previous leg. busy_pct() { local b0 t0 b1 t1 db dt - b0=$(stat_busy); t0=$(stat_total) + read -r b0 t0 < <(stat_sample) sleep "$BUSY_WINDOW" - b1=$(stat_busy); t1=$(stat_total) + read -r b1 t1 < <(stat_sample) db=$((b1 - b0)); dt=$((t1 - t0)) if [ "$dt" -le 0 ]; then echo 100; return; fi echo $((100 * db / dt)) @@ -154,7 +155,7 @@ run_leg() { # engine rep -> 0 accepted, 1 discard echo "before builders: $(builders)" } > "$stem.load" echo "$eng rep=$rep START load=$(loadall) builders=$(builders)" - b0=$(stat_busy); t0=$(stat_total) + read -r b0 t0 < <(stat_sample) if [ "$eng" = ours ]; then # shellcheck disable=SC2086 $TIMEV $TASKSET env VLLM_CPP_CPU_THREADS=$T \ @@ -171,7 +172,7 @@ run_leg() { # engine rep -> 0 accepted, 1 discard rc=$? own=$(own_cpu_jiffies "$OUT/llama-bench-$rep.time") fi - b1=$(stat_busy); t1=$(stat_total) + read -r b1 t1 < <(stat_sample) bafter=$(builders) # Everything the machine burned while our leg ran, minus what our leg burned, # as a share of the machine. This is the check the old post-leg test did not diff --git a/scripts/env-doc-allowlist.txt b/scripts/env-doc-allowlist.txt index 36cab9ceb..cf10cd7a5 100644 --- a/scripts/env-doc-allowlist.txt +++ b/scripts/env-doc-allowlist.txt @@ -1,6 +1,7 @@ VLLM_CPP_QWEN3_DENSE_DECODE_GRAPH VLLM_GEMMA4_MM_DEBUG VLLM_MM_TOWER_PROFILE +VLLM_WINDOWS_CTOR_DIAGNOSTIC VT_ARCH_TACTIC_STATS VT_ASYNC_EXECUTOR_NO_DBUF VT_ASYNC_EXECUTOR_POISON diff --git a/src/vllm/diagnostics/constructor_witness.h b/src/vllm/diagnostics/constructor_witness.h new file mode 100644 index 000000000..72bab8159 --- /dev/null +++ b/src/vllm/diagnostics/constructor_witness.h @@ -0,0 +1,76 @@ +#ifndef VLLM_DIAGNOSTICS_CONSTRUCTOR_WITNESS_H_ +#define VLLM_DIAGNOSTICS_CONSTRUCTOR_WITNESS_H_ + +#include +#include +#include + +namespace vllm::diagnostics { + +constexpr const char* kConstructorWitnessEnvironment = "VLLM_WINDOWS_CTOR_DIAGNOSTIC"; + +inline bool ConstructorWitnessEnabled() noexcept { + const char* value = std::getenv(kConstructorWitnessEnvironment); + return value != nullptr && value[0] == '1' && value[1] == '\0'; +} + +inline void ConstructorWitness(const char* function, const char* stage, + const char* phase, + long long index = -1) noexcept { + if (!ConstructorWitnessEnabled()) return; + if (index < 0) { + std::fprintf(stderr, + "VLLM_CTOR_DIAGNOSTIC: function=%s stage=%s phase=%s\n", + function, stage, phase); + } else { + std::fprintf( + stderr, + "VLLM_CTOR_DIAGNOSTIC: function=%s stage=%s phase=%s index=%lld\n", + function, stage, phase, index); + } + std::fflush(stderr); +} + +inline void ConstructorWitnessBefore(const char* function, const char* stage, + long long index = -1) noexcept { + ConstructorWitness(function, stage, "before", index); +} + +inline void ConstructorWitnessAfter(const char* function, const char* stage, + long long index = -1) noexcept { + ConstructorWitness(function, stage, "after", index); +} + +// A mem-initializer is one full expression. This temporary is constructed before +// that expression and destroyed only after the target member was successfully +// initialized. During exception unwinding it deliberately omits "after", so the +// first unmatched marker remains the failing initializer. +class ConstructorWitnessPhase { + public: + ConstructorWitnessPhase(const char* function, const char* stage) noexcept + : function_(function), + stage_(stage), + enabled_(ConstructorWitnessEnabled()), + uncaught_exceptions_(std::uncaught_exceptions()) { + if (enabled_) ConstructorWitnessBefore(function_, stage_); + } + + ConstructorWitnessPhase(const ConstructorWitnessPhase&) = delete; + ConstructorWitnessPhase& operator=(const ConstructorWitnessPhase&) = delete; + + ~ConstructorWitnessPhase() noexcept { + if (enabled_ && std::uncaught_exceptions() == uncaught_exceptions_) { + ConstructorWitnessAfter(function_, stage_); + } + } + + private: + const char* function_; + const char* stage_; + bool enabled_; + int uncaught_exceptions_; +}; + +} // namespace vllm::diagnostics + +#endif // VLLM_DIAGNOSTICS_CONSTRUCTOR_WITNESS_H_ diff --git a/src/vllm/entrypoints/model_loader.cpp b/src/vllm/entrypoints/model_loader.cpp index d99c1d770..cc92cb0e2 100644 --- a/src/vllm/entrypoints/model_loader.cpp +++ b/src/vllm/entrypoints/model_loader.cpp @@ -4,6 +4,7 @@ // e24d1b24) as exercised by examples/server/main.cpp and the test harness. #include "vllm/entrypoints/model_loader.h" #include "vllm/model_executor/models/qwen3_dflash_gguf.h" +#include "vllm/diagnostics/constructor_witness.h" #include #include @@ -43,6 +44,7 @@ namespace vllm::entrypoints { namespace fs = std::filesystem; +using diagnostics::ConstructorWitnessPhase; // `architecture` is the model's registered architecture string. It is what lets // a PARTIAL backend decline a model whose kernels it has not registered, instead @@ -963,59 +965,82 @@ LoadedEngine::LoadedEngine(HfConfig config, const EngineParams& params, vt::Queue* preselected_queue, std::unique_ptr dflash_draft) - : hash_ready_(EnsureNoneHash()), - config_(std::move(config)), + : hash_ready_((ConstructorWitnessPhase{"LoadedEngine", "hash_ready_"}, + EnsureNoneHash())), + config_((ConstructorWitnessPhase{"LoadedEngine", "config_"}, + std::move(config))), // SPEC-MTP I5d: finalize the speculative config against the checkpoint // (n_predict + resolved k). nullopt on the production default path. - resolved_spec_config_(ResolveSpecConfig(params, config_)), + resolved_spec_config_((ConstructorWitnessPhase{ + "LoadedEngine", "resolved_spec_config_"}, + ResolveSpecConfig(params, config_))), // SPEC-DFLASH D5: the separately-loaded DFlash draft (null for mtp/non-spec). - dflash_draft_(std::move(dflash_draft)), - model_(std::move(model)), - tokenizer_(std::move(tokenizer)), + dflash_draft_((ConstructorWitnessPhase{"LoadedEngine", "dflash_draft_"}, + std::move(dflash_draft))), + model_((ConstructorWitnessPhase{"LoadedEngine", "model_"}, + std::move(model))), + kv_connector_((ConstructorWitnessPhase{"LoadedEngine", "kv_connector_"}, + nullptr)), + tokenizer_((ConstructorWitnessPhase{"LoadedEngine", "tokenizer_"}, + std::move(tokenizer))), // ROAD-V1-MEM M1: resolve the block count from the sizing knobs // (num_blocks override > kv_cache_memory_bytes > util fallback) against the // model's own per-block byte geometry. FIRST, because max_model_len_ is // resolved against this pool. - kv_cfg_(MakeKVCacheResolved( - *model_, config_, params.block_size > 0 ? params.block_size : 32, - params, resolved_spec_config_)), + kv_cfg_((ConstructorWitnessPhase{"LoadedEngine", "kv_cfg_"}, + MakeKVCacheResolved( + *model_, config_, + params.block_size > 0 ? params.block_size : 32, params, + resolved_spec_config_))), // The serving length, checked (pinned) or auto-fitted (unpinned) against // kv_cfg_. See ResolveMaxModelLen. - max_model_len_(ResolveMaxModelLen( - params, config_, kv_cfg_, - params.block_size > 0 ? params.block_size : 32)), - max_num_batched_tokens_(ResolveMaxNumBatchedTokens( - params, max_model_len_, ModelRegistry::IsDenseModel(*model_))), - prefix_caching_enabled_(ResolveEnablePrefixCaching( - params, model_->registration().info)), + max_model_len_((ConstructorWitnessPhase{"LoadedEngine", "max_model_len_"}, + ResolveMaxModelLen( + params, config_, kv_cfg_, + params.block_size > 0 ? params.block_size : 32))), + max_num_batched_tokens_((ConstructorWitnessPhase{ + "LoadedEngine", "max_num_batched_tokens_"}, + ResolveMaxNumBatchedTokens( + params, max_model_len_, + ModelRegistry::IsDenseModel(*model_)))), + prefix_caching_enabled_((ConstructorWitnessPhase{ + "LoadedEngine", "prefix_caching_enabled_"}, + ResolveEnablePrefixCaching( + params, model_->registration().info))), // ENG-SGLANG-BEHAVIOR-FLAG SW3: resolve jump-forward once (config field + // VT_ENABLE_JUMP_FORWARD env override). Default nullopt+no-env => false => // the byte-identical decode path (jump-forward is inert until enabled). - jump_forward_enabled_( - vllm::v1::JumpForwardEnabled(params.enable_jump_forward)), + jump_forward_enabled_((ConstructorWitnessPhase{ + "LoadedEngine", "jump_forward_enabled_"}, + vllm::v1::JumpForwardEnabled( + params.enable_jump_forward))), // runner_ FIRST (W3): the async-scheduling flip reads // runner_.runner_supports_async(). SPEC-MTP I5d: when speculation is on, // pass the resolved config + the MTP draft (built from the retained mtp.* // weights, sharing the target embed/lm_head). The draft KV `fa_draft` group // is allocated by the runner from kv_cfg_ (empty vector here), so the loop // reaches it via runner-owned storage. nullopt/null on the default path. - runner_(config_, *model_, kv_cfg_, - preselected_queue != nullptr - ? *preselected_queue - : SelectQueueForModel(model_->registration().architecture, - params.device), - /*max_num_reqs=*/params.max_num_seqs > 0 ? params.max_num_seqs : 8, - max_model_len_, - /*max_num_batched_tokens=*/max_num_batched_tokens_, - resolved_spec_config_, - // Only the MTP method builds an in-target MTP draft; DFlash (D5) - // loads a SEPARATE draft, wired via set_dflash_draft in the body. - resolved_spec_config_.has_value() && - resolved_spec_config_->method == "mtp" && - model_->supports_mtp_draft() - ? model_->BuildMtpDraft(config_) - : nullptr, - /*draft_kv=*/{}), + runner_((ConstructorWitnessPhase{"LoadedEngine", "runner_"}, + vllm::v1::GPUModelRunner( + config_, *model_, kv_cfg_, + preselected_queue != nullptr + ? *preselected_queue + : SelectQueueForModel( + model_->registration().architecture, params.device), + /*max_num_reqs=*/params.max_num_seqs > 0 + ? params.max_num_seqs + : 8, + max_model_len_, + /*max_num_batched_tokens=*/max_num_batched_tokens_, + resolved_spec_config_, + // Only the MTP method builds an in-target MTP draft; DFlash + // (D5) loads a SEPARATE draft, wired in the body. + resolved_spec_config_.has_value() && + resolved_spec_config_->method == "mtp" && + model_->supports_mtp_draft() + ? model_->BuildMtpDraft(config_) + : nullptr, + /*draft_kv=*/{}))), // Resolve the enable-flip from the now-constructed runner + VT_ASYNC_SCHED, // then size the batch-queue depth (2 under async scheduling → depth-2 // step_with_batch_queue; 1 otherwise). Since the 2026-07-17 flip the default @@ -1025,55 +1050,84 @@ LoadedEngine::LoadedEngine(HfConfig config, // post_step path, which is the SYNCHRONOUS scheduler's contract; the // async-scheduling draft-in-output variant is deferred (spec §2.5), so a // configured speculator forces sync scheduling here. - async_scheduling_enabled_(!resolved_spec_config_.has_value() && - ResolveAsyncEnabled( - MakeSchedulerConfig( - max_model_len_, - params.max_num_seqs > 0 ? params.max_num_seqs : 8, - max_num_batched_tokens_, params.policy), - runner_.runner_supports_async(), - model_->registration().info.is_pooling_model)), - max_concurrent_batches_(MakeSchedulerConfig( - max_model_len_, - params.max_num_seqs > 0 ? params.max_num_seqs - : 8, - max_num_batched_tokens_, params.policy) - .MaxConcurrentBatches(async_scheduling_enabled_)), + async_scheduling_enabled_((ConstructorWitnessPhase{ + "LoadedEngine", + "async_scheduling_enabled_"}, + !resolved_spec_config_.has_value() && + ResolveAsyncEnabled( + MakeSchedulerConfig( + max_model_len_, + params.max_num_seqs > 0 + ? params.max_num_seqs + : 8, + max_num_batched_tokens_, + params.policy), + runner_.runner_supports_async(), + model_->registration() + .info.is_pooling_model))), + max_concurrent_batches_((ConstructorWitnessPhase{ + "LoadedEngine", "max_concurrent_batches_"}, + MakeSchedulerConfig( + max_model_len_, + params.max_num_seqs > 0 ? params.max_num_seqs + : 8, + max_num_batched_tokens_, params.policy) + .MaxConcurrentBatches( + async_scheduling_enabled_))), // The engine-wide structured-output manager, native backend over the // tokenizer (upstream EngineCore constructs one unconditionally, // core.py:134). Wired into the scheduler + engine cores below so // response_format / C-ABI structured constraints gate decoding. - structured_output_manager_( - params.max_num_seqs > 0 ? params.max_num_seqs : 8, - vllm::v1::MakeNativeBackendFactory( - tokenizer_, static_cast(config_.vocab_size))), + structured_output_manager_((ConstructorWitnessPhase{ + "LoadedEngine", + "structured_output_manager_"}, + vllm::v1::StructuredOutputManager( + params.max_num_seqs > 0 + ? params.max_num_seqs + : 8, + vllm::v1::MakeNativeBackendFactory( + tokenizer_, static_cast( + config_.vocab_size))))), // AsyncScheduler when the flip resolved ON, else the synchronous Scheduler. - scheduler_(MakeScheduler( - async_scheduling_enabled_, - MakeSchedulerConfig( - max_model_len_, params.max_num_seqs > 0 ? params.max_num_seqs : 8, - max_num_batched_tokens_, params.policy), - kv_cfg_, params.block_size > 0 ? params.block_size : 32, - /*enable_caching=*/prefix_caching_enabled_, - &structured_output_manager_, resolved_spec_config_)), - executor_(runner_), + scheduler_((ConstructorWitnessPhase{"LoadedEngine", "scheduler_"}, + MakeScheduler( + async_scheduling_enabled_, + MakeSchedulerConfig( + max_model_len_, + params.max_num_seqs > 0 ? params.max_num_seqs : 8, + max_num_batched_tokens_, params.policy), + kv_cfg_, params.block_size > 0 ? params.block_size : 32, + /*enable_caching=*/prefix_caching_enabled_, + &structured_output_manager_, resolved_spec_config_))), + executor_((ConstructorWitnessPhase{"LoadedEngine", "executor_"}, + vllm::v1::Executor(runner_))), // SPEC-MTP I5d: with a speculator configured, EngineCore pulls the runner's // out-of-band drafts each step (take_draft_token_ids -> update_draft_token_ids) // so the next verify step schedules them. Default false (no-op post_step). - engine_core_(*scheduler_, executor_, &structured_output_manager_, - /*check_for_draft_tokens=*/resolved_spec_config_.has_value()), + engine_core_((ConstructorWitnessPhase{"LoadedEngine", "engine_core_"}, + vllm::v1::EngineCore( + *scheduler_, executor_, &structured_output_manager_, + /*check_for_draft_tokens=*/ + resolved_spec_config_.has_value()))), // The admission-time prompt-length check validates against the RESOLVED // serving length, which is what upstream's model_config.max_model_len is // (input_processor.py:399-401). Passing config_ alone would check against // the raw checkpoint context and let through prompts the pool cannot hold. - input_processor_(tokenizer_, config_, max_model_len_), - output_processor_(&tokenizer_), - block_hasher_(prefix_caching_enabled_ - ? vllm::v1::get_request_block_hasher( - params.block_size > 0 ? params.block_size : 32, - vllm::v1::sha256_cbor) - : nullptr), - engine_(input_processor_, engine_core_, output_processor_, block_hasher_) { + input_processor_((ConstructorWitnessPhase{"LoadedEngine", "input_processor_"}, + vllm::v1::InputProcessor(tokenizer_, config_, + max_model_len_))), + output_processor_((ConstructorWitnessPhase{ + "LoadedEngine", "output_processor_"}, + vllm::v1::OutputProcessor(&tokenizer_))), + block_hasher_((ConstructorWitnessPhase{"LoadedEngine", "block_hasher_"}, + prefix_caching_enabled_ + ? vllm::v1::get_request_block_hasher( + params.block_size > 0 ? params.block_size : 32, + vllm::v1::sha256_cbor) + : nullptr)), + engine_((ConstructorWitnessPhase{"LoadedEngine", "engine_"}, + vllm::v1::LLMEngine(input_processor_, engine_core_, + output_processor_, block_hasher_))) { (void)hash_ready_; // issue #371: REFUSE an unservable recurrent-state budget instead of // allocating it. Speculation widens the Mamba/GDN state to k+1 snapshot slots diff --git a/src/vllm/model_executor/models/ltx2.cpp b/src/vllm/model_executor/models/ltx2.cpp index 3d9af69ce..f4ec1202f 100644 --- a/src/vllm/model_executor/models/ltx2.cpp +++ b/src/vllm/model_executor/models/ltx2.cpp @@ -13,6 +13,7 @@ #include #include #include +#include #include #include @@ -69,7 +70,7 @@ float Silu(float x) { return x / (1.0f + std::exp(-x)); } // torch.nn.functional.gelu(..., approximate="tanh"), the activation // `activation_fn="gelu-approximate"` selects (gelu_approx.py:10). float GeluTanh(float x) { - const float kBeta = static_cast(std::sqrt(2.0 / M_PI)); + const float kBeta = static_cast(std::sqrt(2.0 / std::numbers::pi_v)); const float kKappa = 0.044715f; const float inner = kBeta * (x + kKappa * x * x * x); return 0.5f * x * (1.0f + std::tanh(inner)); @@ -554,7 +555,8 @@ std::vector FreqGridPytorch(double theta, int64_t n_pos_dims, int64_t dim const float t = i < halfway ? step * static_cast(i) : 1.0f - step * static_cast(n - 1 - i); out[static_cast(i)] = - std::pow(static_cast(theta), t) * static_cast(M_PI / 2.0); + std::pow(static_cast(theta), t) * + static_cast(std::numbers::pi_v / 2.0); } return out; } @@ -569,7 +571,8 @@ std::vector FreqGridNumpy(double theta, int64_t n_pos_dims, int64_t dim) for (int64_t i = 0; i < n; ++i) { // numpy's linspace is arange(n) * step, with the final sample forced to `stop`. const double t = (i == n - 1) ? 1.0 : step * static_cast(i); - out[static_cast(i)] = static_cast(std::pow(theta, t) * (M_PI / 2.0)); + out[static_cast(i)] = + static_cast(std::pow(theta, t) * (std::numbers::pi_v / 2.0)); } return out; } diff --git a/src/vllm/model_executor/models/ltx2_audio_vae.cpp b/src/vllm/model_executor/models/ltx2_audio_vae.cpp index cff349f48..dcbbbc1bb 100644 --- a/src/vllm/model_executor/models/ltx2_audio_vae.cpp +++ b/src/vllm/model_executor/models/ltx2_audio_vae.cpp @@ -24,6 +24,7 @@ #include #include #include +#include #include #include @@ -31,10 +32,6 @@ #include "vllm/model_executor/models/vocoder1d.h" #include "vt/dtype.h" -#ifndef M_PI -#define M_PI 3.14159265358979323846 -#endif - namespace vllm { const std::vector& Ltx2VaeWeights::Get(const std::string& name) const { @@ -434,8 +431,14 @@ std::vector Ltx2HannSincResampleFilter1d(int64_t ratio, int64_t* kernel_s std::max(-static_cast(lowpass_filter_width), std::min(static_cast(lowpass_filter_width), t)); const double window = - std::pow(std::cos(clamped * M_PI / static_cast(lowpass_filter_width) / 2.0), 2.0); - const double sinc = t == 0.0 ? 1.0 : std::sin(M_PI * t) / (M_PI * t); + std::pow(std::cos(clamped * std::numbers::pi_v / + static_cast(lowpass_filter_width) / 2.0), + 2.0); + const double sinc = + t == 0.0 + ? 1.0 + : std::sin(std::numbers::pi_v * t) / + (std::numbers::pi_v * t); filter[static_cast(i)] = static_cast(sinc * window * rolloff / static_cast(ratio)); } @@ -1043,7 +1046,8 @@ std::vector Ltx2WaveformToLogMel(const Ltx2AudioProcessorConfig& config, std::vector window(static_cast(n_fft)); for (int64_t i = 0; i < n_fft; ++i) { window[static_cast(i)] = - 0.5 - 0.5 * std::cos(2.0 * M_PI * static_cast(i) / static_cast(n_fft)); + 0.5 - 0.5 * std::cos(2.0 * std::numbers::pi_v * static_cast(i) / + static_cast(n_fft)); } // `center=True, pad_mode="reflect"`: pad n_fft/2 on BOTH sides, so frame 0 is @@ -1081,7 +1085,8 @@ std::vector Ltx2WaveformToLogMel(const Ltx2AudioProcessorConfig& config, for (int64_t f = 0; f < n_freqs; ++f) { double real = 0.0; double imag = 0.0; - const double omega = -2.0 * M_PI * static_cast(f) / static_cast(n_fft); + const double omega = -2.0 * std::numbers::pi_v * static_cast(f) / + static_cast(n_fft); for (int64_t i = 0; i < n_fft; ++i) { const double angle = omega * static_cast(i); real += frame[static_cast(i)] * std::cos(angle); diff --git a/src/vllm/model_executor/models/ltx2_video_vae.cpp b/src/vllm/model_executor/models/ltx2_video_vae.cpp index c7cb88825..592f5b8b4 100644 --- a/src/vllm/model_executor/models/ltx2_video_vae.cpp +++ b/src/vllm/model_executor/models/ltx2_video_vae.cpp @@ -55,10 +55,6 @@ #include "vllm/model_executor/models/minimax_h3.h" #include "vt/dtype.h" -#ifndef M_PI -#define M_PI 3.14159265358979323846 -#endif - namespace vllm { namespace { diff --git a/src/vllm/model_executor/models/qwen3_5_moe.cpp b/src/vllm/model_executor/models/qwen3_5_moe.cpp index 1e33afb76..4c03db8e7 100644 --- a/src/vllm/model_executor/models/qwen3_5_moe.cpp +++ b/src/vllm/model_executor/models/qwen3_5_moe.cpp @@ -23,6 +23,7 @@ #include "vllm/model_executor/models/qwen3_5_mtp.h" // SPEC-MTP I5d-pre draft #include "vllm/model_executor/models/qwen3_5_weights.h" #include "vllm/platforms/interface.h" // GetPlatform(device.type) memory-model seam +#include "vllm/diagnostics/constructor_witness.h" namespace vllm { namespace { @@ -200,9 +201,12 @@ const ModelFactory kQwen3_5MoeFactory{ std::unique_ptr MakeQwen3_5MoeLoadedModel( Qwen3_5MoeWeights weights) { - return std::make_unique( - RegistrationFor("Qwen3_5MoeForConditionalGeneration"), - std::move(weights)); + using diagnostics::ConstructorWitnessPhase; + return (ConstructorWitnessPhase{"LoadedEngine", + "MakeQwen3_5MoeLoadedModel"}, + std::make_unique( + RegistrationFor("Qwen3_5MoeForConditionalGeneration"), + std::move(weights))); } std::unique_ptr BorrowQwen3_5MoeLoadedModel( diff --git a/src/vllm/multimodal/video_engine.cpp b/src/vllm/multimodal/video_engine.cpp index 656a312f1..9736ebdfa 100644 --- a/src/vllm/multimodal/video_engine.cpp +++ b/src/vllm/multimodal/video_engine.cpp @@ -11,15 +11,15 @@ #include #include #include +#include #include #include #include #include +#include #include #include -#include - #include #include "vllm/entrypoints/openai/video_api.h" @@ -55,13 +55,13 @@ std::vector& RegistryStorage() { const std::vector& OrderedRegistry() { return RegistryStorage(); } bool IsDir(const std::string& path) { - struct stat st {}; - return ::stat(path.c_str(), &st) == 0 && S_ISDIR(st.st_mode); + std::error_code ec; + return std::filesystem::is_directory(path, ec) && !ec; } bool Exists(const std::string& path) { - struct stat st {}; - return ::stat(path.c_str(), &st) == 0; + std::error_code ec; + return std::filesystem::exists(path, ec) && !ec; } std::string StripTrailingSlash(const std::string& dir) { diff --git a/src/vllm/v1/worker/gpu/runner.cpp b/src/vllm/v1/worker/gpu/runner.cpp index 22ce5b197..2a9d7d1cf 100644 --- a/src/vllm/v1/worker/gpu/runner.cpp +++ b/src/vllm/v1/worker/gpu/runner.cpp @@ -5,6 +5,7 @@ // See include/vllm/v1/worker/gpu/runner.h for scope, the V1-algorithm / MRV2- // contract composition, the four-way ordering contract, and the deferred paths. #include "vllm/v1/worker/gpu/runner.h" +#include "vllm/diagnostics/constructor_witness.h" #include #include @@ -40,6 +41,10 @@ namespace vllm::v1 { +using diagnostics::ConstructorWitnessAfter; +using diagnostics::ConstructorWitnessBefore; +using diagnostics::ConstructorWitnessPhase; + // Logits-gather A/B toggle (perf). Default ON: the forward gathers the // per-request last-token hidden rows BEFORE lm_head (prefill/mixed), so lm_head // runs on num_reqs rows and only [num_reqs,vocab] is Downloaded. VT_LOGITS_GATHER=0 @@ -317,18 +322,29 @@ GPUModelRunner::GPUModelRunner( std::optional spec_config, std::unique_ptr draft_model, std::vector draft_kv) - : config_(config), - model_(&model), - spec_config_(std::move(spec_config)), - draft_model_(std::move(draft_model)), - draft_attn_kv_(std::move(draft_kv)), - queue_(queue), - input_batch_(max_num_reqs, max_model_len, max_num_batched_tokens, - static_cast(config.vocab_size), - group_block_sizes(kv_cache_config), - group_block_sizes(kv_cache_config)) { + : config_((ConstructorWitnessPhase{"GPUModelRunner", "config_"}, config)), + owned_model_((ConstructorWitnessPhase{"GPUModelRunner", "owned_model_"}, + nullptr)), + model_((ConstructorWitnessPhase{"GPUModelRunner", "model_"}, &model)), + spec_config_((ConstructorWitnessPhase{"GPUModelRunner", "spec_config_"}, + std::move(spec_config))), + draft_model_((ConstructorWitnessPhase{"GPUModelRunner", "draft_model_"}, + std::move(draft_model))), + draft_attn_kv_((ConstructorWitnessPhase{"GPUModelRunner", "draft_attn_kv_"}, + std::move(draft_kv))), + queue_((ConstructorWitnessPhase{"GPUModelRunner", "queue_"}, queue)), + input_batch_((ConstructorWitnessPhase{"GPUModelRunner", "input_batch_"}, + InputBatch(max_num_reqs, max_model_len, + max_num_batched_tokens, + static_cast(config.vocab_size), + group_block_sizes(kv_cache_config), + group_block_sizes(kv_cache_config)))) { + ConstructorWitnessBefore("GPUModelRunner", "assign-max-num-reqs"); max_num_reqs_ = max_num_reqs; + ConstructorWitnessAfter("GPUModelRunner", "assign-max-num-reqs"); + ConstructorWitnessBefore("GPUModelRunner", "assign-max-num-batched-tokens"); max_num_batched_tokens_ = max_num_batched_tokens; + ConstructorWitnessAfter("GPUModelRunner", "assign-max-num-batched-tokens"); // SPEC-MTP I5e: the async input-combine splices the device-resident // last_sampled token over each decode row's input id with // num_new_sampled_tokens==1; it is NOT spec-aware and would overwrite the @@ -337,17 +353,25 @@ GPUModelRunner::GPUModelRunner( // spliced into token_ids_cpu by update_req_spec_token_ids + prepare_inputs, // so force the sync host input path here. Byte-identical for non-spec // (spec_config_ is nullopt there, so this is AsyncRunnerEnvDefault()). + ConstructorWitnessBefore("GPUModelRunner", "resolve-async-input-combine"); async_input_combine_ = AsyncRunnerEnvDefault() && !spec_config_.has_value() && QueueSupportsAsyncInputCombine(queue_); + ConstructorWitnessAfter("GPUModelRunner", "resolve-async-input-combine"); // ARCH-ONE-SURFACE ROW 6 (mirror gpu/model_runner.py:368-369): a POOLING // model's runner pools instead of sampling — build the PoolingRunner over // the model-owned Pooler. Null for every text arch (byte-identical). + ConstructorWitnessBefore("GPUModelRunner", "pooling-model-branch"); if (model_->registration().info.is_pooling_model && model_->pooler() != nullptr) { pooling_runner_ = std::make_unique(*model_->pooler()); } + ConstructorWitnessAfter("GPUModelRunner", "pooling-model-branch"); + ConstructorWitnessBefore("GPUModelRunner", "initialize-kv-cache"); initialize_kv_cache(kv_cache_config); + ConstructorWitnessAfter("GPUModelRunner", "initialize-kv-cache"); + ConstructorWitnessBefore("GPUModelRunner", "model-registry-prepare"); ModelRegistry::Prepare(*model_, config_, queue_); + ConstructorWitnessAfter("GPUModelRunner", "model-registry-prepare"); } GPUModelRunner::GPUModelRunner( @@ -437,6 +461,8 @@ GPUModelRunner::CacheBuffer::~CacheBuffer() { } void GPUModelRunner::initialize_kv_cache(const KVCacheConfig& kv_cache_config) { + ConstructorWitnessBefore("GPUModelRunner::initialize_kv_cache", + "scalar-state-slot-setup"); num_blocks_ = kv_cache_config.num_blocks; // GDN mamba-state slots = max concurrent sequences (one recurrent state per // sequence), decoupled from the attention num_blocks. Guard against a 0 (e.g. @@ -457,6 +483,8 @@ void GPUModelRunner::initialize_kv_cache(const KVCacheConfig& kv_cache_config) { // spec_cols; without speculation spec_cols==1 and this is the pre-spec pool. for (int64_t b = base_slots - 1; b >= 0; --b) gdn_free_slots_.push_back(static_cast(b * spec_cols)); + ConstructorWitnessAfter("GPUModelRunner::initialize_kv_cache", + "scalar-state-slot-setup"); // Resolve the full-attn + GDN(mamba) KV group ids (T0 gate models: exactly one // of each). The block-table group order == kv_cache_groups order. @@ -472,6 +500,8 @@ void GPUModelRunner::initialize_kv_cache(const KVCacheConfig& kv_cache_config) { // byte-identical to the old behavior — the first and only group is chosen. for (int g = 0; g < static_cast(kv_cache_config.kv_cache_groups.size()); ++g) { + ConstructorWitnessBefore("GPUModelRunner::initialize_kv_cache", + "kv-group-scan", g); const auto& group = kv_cache_config.kv_cache_groups[static_cast(g)]; const KVCacheSpecKind kind = group.kv_cache_spec->kind(); // MLA campaign W7: an `MLAAttentionSpec` group IS the model's attention @@ -489,6 +519,8 @@ void GPUModelRunner::initialize_kv_cache(const KVCacheConfig& kv_cache_config) { } else if (kind == KVCacheSpecKind::kMamba) { gdn_group_id_ = g; } + ConstructorWitnessAfter("GPUModelRunner::initialize_kv_cache", + "kv-group-scan", g); } // Allocate one PagedKvCache per full-attn layer and one GdnStateCache per GDN @@ -508,6 +540,8 @@ void GPUModelRunner::initialize_kv_cache(const KVCacheConfig& kv_cache_config) { const MambaSpec* mamba_spec = nullptr; if (gdn_group_id_ >= 0) { + ConstructorWitnessBefore("GPUModelRunner::initialize_kv_cache", + "mamba-validation", gdn_group_id_); mamba_spec = dynamic_cast( kv_cache_config.kv_cache_groups[static_cast(gdn_group_id_)] .kv_cache_spec.get()); @@ -534,6 +568,8 @@ void GPUModelRunner::initialize_kv_cache(const KVCacheConfig& kv_cache_config) { VT_CHECK(supported_state_dtype(gdn_conv_cache_dtype_) && supported_state_dtype(gdn_ssm_cache_dtype_), "runner: Qwen3.5 MambaSpec state dtypes must be floating"); + ConstructorWitnessAfter("GPUModelRunner::initialize_kv_cache", + "mamba-validation", gdn_group_id_); } // SPEC-DRIVEN attention-cache sizing and layout (MLA campaign W1). @@ -564,6 +600,9 @@ void GPUModelRunner::initialize_kv_cache(const KVCacheConfig& kv_cache_config) { int64_t fa_page_bytes = 0; vt::DType kv_dtype = ResolveKvCacheDType(); if (full_attn_group_id_ >= 0) { + ConstructorWitnessBefore("GPUModelRunner::initialize_kv_cache", + "full-attention-geometry", + full_attn_group_id_); const KVCacheSpec* fa_spec = kv_cache_config.kv_cache_groups[static_cast(full_attn_group_id_)] .kv_cache_spec.get(); @@ -600,12 +639,17 @@ void GPUModelRunner::initialize_kv_cache(const KVCacheConfig& kv_cache_config) { static_cast(fa_page_bytes), static_cast(num_blocks_)); } + ConstructorWitnessAfter("GPUModelRunner::initialize_kv_cache", + "full-attention-geometry", + full_attn_group_id_); } // Recorded for the gates: the exact per-block byte cost the allocator used, // sourced from the spec. `fa_page_size_bytes() > 0` is the runtime proof that // the spec-driven path RAN (a compiled-but-unexercised path leaves it 0). fa_page_size_bytes_ = fa_page_bytes; + ConstructorWitnessBefore("GPUModelRunner::initialize_kv_cache", + "residency-buffer-setup"); const vt::Device dev = queue_.device; const char* device_cache_env = std::getenv("VT_DEVICE_KV_CACHE"); // W0b-1 / work row M3a: this read `is_cuda()`, which is the SAME defect the @@ -656,6 +700,8 @@ void GPUModelRunner::initialize_kv_cache(const KVCacheConfig& kv_cache_config) { vt::DType dtype; }; std::vector fa_dims; + ConstructorWitnessAfter("GPUModelRunner::initialize_kv_cache", + "residency-buffer-setup"); for (int64_t l = 0; l < config_.num_hidden_layers; ++l) { const bool is_gdn = has_mamba_group && !config_.layer_types.empty() && @@ -669,14 +715,22 @@ void GPUModelRunner::initialize_kv_cache(const KVCacheConfig& kv_cache_config) { const size_t conv_es = vt::SizeOf(gdn_conv_cache_dtype_); const int64_t conv_row_elems = conv_dim * conv_state_len; const int64_t ssm_row_elems = Hv * Dv * Dk; + ConstructorWitnessBefore("GPUModelRunner::initialize_kv_cache", + "gdn-ssm-allocation", l); ssm_buf_.push_back(std::make_unique( dev, queue_, static_cast(gdn_state_slots_ * ssm_row_elems) * ssm_es, kv_cache_backend_resident_)); + ConstructorWitnessAfter("GPUModelRunner::initialize_kv_cache", + "gdn-ssm-allocation", l); + ConstructorWitnessBefore("GPUModelRunner::initialize_kv_cache", + "gdn-conv-allocation", l); conv_buf_.push_back(std::make_unique( dev, queue_, static_cast(gdn_state_slots_ * conv_row_elems) * conv_es, kv_cache_backend_resident_)); + ConstructorWitnessAfter("GPUModelRunner::initialize_kv_cache", + "gdn-conv-allocation", l); } else { // Bytes come from the SPEC, not from HF-config arithmetic: exactly // `num_blocks * spec->page_size_bytes()`, mirroring upstream's @@ -713,11 +767,15 @@ void GPUModelRunner::initialize_kv_cache(const KVCacheConfig& kv_cache_config) { VT_CHECK(l_page > 0, "runner: per-layer attention spec reported a non-positive page"); } + ConstructorWitnessBefore("GPUModelRunner::initialize_kv_cache", + "full-attention-allocation", l); full_attn_buf_.push_back(std::make_unique( dev, queue_, static_cast(num_blocks_) * static_cast(l_page), kv_cache_backend_resident_)); fa_dims.push_back(FaDims{l_Hkv, l_Dh, l_dtype}); + ConstructorWitnessAfter("GPUModelRunner::initialize_kv_cache", + "full-attention-allocation", l); } } @@ -727,6 +785,9 @@ void GPUModelRunner::initialize_kv_cache(const KVCacheConfig& kv_cache_config) { "runner: per-layer KV view geometry out of sync with buffers"); attn_kv_.clear(); for (size_t i = 0; i < full_attn_buf_.size(); ++i) { + ConstructorWitnessBefore("GPUModelRunner::initialize_kv_cache", + "full-attention-view", + static_cast(i)); PagedKvCache kv; kv.data = full_attn_buf_[i]->data(); kv.dtype = fa_dims[i].dtype; @@ -735,6 +796,9 @@ void GPUModelRunner::initialize_kv_cache(const KVCacheConfig& kv_cache_config) { kv.num_kv_heads = fa_dims[i].num_kv_heads; kv.head_size = fa_dims[i].head_size; attn_kv_.push_back(kv); + ConstructorWitnessAfter("GPUModelRunner::initialize_kv_cache", + "full-attention-view", + static_cast(i)); } // SPEC-MTP I5d: allocate the MTP draft's own paged KV layer (the `fa_draft` @@ -753,6 +817,8 @@ void GPUModelRunner::initialize_kv_cache(const KVCacheConfig& kv_cache_config) { if (group.kv_cache_spec->kind() != KVCacheSpecKind::kFullAttention) { continue; // the GDN group and any non-attn group are not the draft. } + ConstructorWitnessBefore("GPUModelRunner::initialize_kv_cache", + "draft-attention-storage", g); draft_attn_buf_.push_back(std::make_unique( dev, queue_, static_cast(num_blocks_) * static_cast(fa_page_bytes), @@ -765,12 +831,16 @@ void GPUModelRunner::initialize_kv_cache(const KVCacheConfig& kv_cache_config) { dkv.num_kv_heads = Hkv; dkv.head_size = Dh; draft_attn_kv_.push_back(dkv); + ConstructorWitnessAfter("GPUModelRunner::initialize_kv_cache", + "draft-attention-storage", g); break; // exactly one fa_draft group at k=1. } } gdn_state_.clear(); for (size_t g = 0; g < ssm_buf_.size(); ++g) { + ConstructorWitnessBefore("GPUModelRunner::initialize_kv_cache", + "gdn-state-view", static_cast(g)); GdnStateCache gs; gs.ssm_state = vt::Tensor::Contiguous(ssm_buf_[g]->data(), gdn_ssm_cache_dtype_, @@ -781,6 +851,8 @@ void GPUModelRunner::initialize_kv_cache(const KVCacheConfig& kv_cache_config) { {gdn_state_slots_, conv_dim, conv_state_len}); gdn_state_.push_back(gs); + ConstructorWitnessAfter("GPUModelRunner::initialize_kv_cache", + "gdn-state-view", static_cast(g)); } } diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 291f8cfdb..fe07970f7 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -943,6 +943,16 @@ target_compile_definitions(test_chat_template PRIVATE VLLM_TEST_FIXTURES_DIR="${CMAKE_SOURCE_DIR}/tests/fixtures") # The HTTP api_server smoke test needs the vendored cpp-httplib (gated). if(VLLM_CPP_SERVER) + add_executable(test_httplib_accepted_socket_close + vllm/entrypoints/openai/test_httplib_accepted_socket_close.cpp) + target_link_libraries(test_httplib_accepted_socket_close PRIVATE + vllm_test_main Threads::Threads) + if(WIN32) + target_link_libraries(test_httplib_accepted_socket_close PRIVATE ws2_32) + endif() + vllm_cpp_set_warnings(test_httplib_accepted_socket_close) + add_test(NAME test_httplib_accepted_socket_close + COMMAND test_httplib_accepted_socket_close) vllm_cpp_add_test(test_openai_api_server vllm/entrypoints/openai/test_api_server.cpp) # /v1/audio/transcriptions dispatch + socket smoke run against the REAL # library transcription seam on the committed parakeet_e2e fixture @@ -954,6 +964,18 @@ if(VLLM_CPP_SERVER) # (ARCH-ONE-SURFACE ROW 6). LLAMA_EMBED_FIXTURE_DIR="${CMAKE_SOURCE_DIR}/tests/vllm/models/fixtures/llama_embed_e2e") target_include_directories(test_openai_api_server PRIVATE ${CMAKE_SOURCE_DIR}/src) + add_executable(test_openai_server_thread_failure_fixture + vllm/entrypoints/openai/test_server_thread_failure_fixture.cpp) + target_link_libraries(test_openai_server_thread_failure_fixture PRIVATE + vllm_test_main Threads::Threads) + vllm_cpp_set_warnings(test_openai_server_thread_failure_fixture) + add_test(NAME test_openai_server_thread_failure_fixture + COMMAND ${Python3_EXECUTABLE} + "${CMAKE_SOURCE_DIR}/tests/scripts/run_expected_doctest_failure.py" + $ + "scoped teardown fixture assertion") + set_tests_properties(test_openai_server_thread_failure_fixture PROPERTIES + TIMEOUT 10) # M3.6: the OpenAI server CONFORMANCE suite — the full API contract exercised # end to end over the REAL cpp-httplib server on an ephemeral port. vllm_cpp_add_test(test_openai_conformance vllm/entrypoints/openai/test_conformance.cpp) diff --git a/tests/doctest_main.cpp b/tests/doctest_main.cpp index 0a3f254ea..d4c4ef00c 100644 --- a/tests/doctest_main.cpp +++ b/tests/doctest_main.cpp @@ -1,2 +1,40 @@ -#define DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN +#define DOCTEST_CONFIG_IMPLEMENT #include + +#include +#include +#include + +namespace { + +[[noreturn]] void DiagnosticTerminate() noexcept { + const std::exception_ptr exception = std::current_exception(); + if (exception == nullptr) { + std::fputs( + "[vllm-test-probe] std::terminate current_exception=none\n", stderr); + } else { + std::fputs( + "[vllm-test-probe] std::terminate current_exception=present\n", + stderr); + try { + std::rethrow_exception(exception); + } catch (const std::exception& error) { + std::fprintf(stderr, + "[vllm-test-probe] std::terminate std::exception what=%s\n", + error.what()); + } catch (...) { + std::fputs( + "[vllm-test-probe] std::terminate exception=non-std-exception\n", + stderr); + } + } + std::fflush(stderr); + std::abort(); +} + +} // namespace + +int main(int argc, char** argv) { + std::set_terminate(DiagnosticTerminate); + return doctest::Context(argc, argv).run(); +} diff --git a/tests/scripts/run_expected_doctest_failure.py b/tests/scripts/run_expected_doctest_failure.py new file mode 100644 index 000000000..7ae546228 --- /dev/null +++ b/tests/scripts/run_expected_doctest_failure.py @@ -0,0 +1,36 @@ +#!/usr/bin/env python3 +"""Require one intentional doctest assertion failure without process abort.""" + +from __future__ import annotations + +import subprocess +import sys + + +def main() -> int: + if len(sys.argv) != 3: + print("usage: run_expected_doctest_failure.py TEST EXPECTED_TEXT") + return 2 + + completed = subprocess.run( + [sys.argv[1]], + check=False, + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + text=True, + ) + print(completed.stdout, end="") + if completed.returncode != 1: + print(f"expected doctest exit 1, got {completed.returncode}") + return 1 + if sys.argv[2] not in completed.stdout: + print(f"missing expected doctest assertion: {sys.argv[2]}") + return 1 + if "[doctest] Status: FAILURE!" not in completed.stdout: + print("doctest failure summary was not emitted") + return 1 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/scripts/test_check_windows_portability.py b/tests/scripts/test_check_windows_portability.py index a5202a936..ed3f432c3 100644 --- a/tests/scripts/test_check_windows_portability.py +++ b/tests/scripts/test_check_windows_portability.py @@ -33,6 +33,41 @@ ) +_ORACLE_DECLARATION_TYPE = re.compile( + r"\b(?:vt::Backend|Queue|(?:const\s+)?Device|" + r"std::vector\s*<\s*(?:float|int64_t)\s*>)\s*(?:[&*]\s*)?" +) + + +def _oracle_declarator_names(source: str) -> set[str]: + """Return names declared with the unbound oracle's relevant types.""" + names: set[str] = set() + for type_match in _ORACLE_DECLARATION_TYPE.finditer(source): + declarators = [] + start = type_match.end() + depths = {"(": 0, "[": 0, "{": 0} + closing = {")": "(", "]": "[", "}": "{"} + cursor = start + for index in range(start, len(source)): + char = source[index] + if char in depths: + depths[char] += 1 + elif char in closing: + opener = closing[char] + if depths[opener] > 0: + depths[opener] -= 1 + elif char in {",", ";"} and not any(depths.values()): + declarators.append(source[cursor:index]) + cursor = index + 1 + if char == ";": + break + for declarator in declarators: + match = re.match(r"\s*(?:[&*]\s*)*([A-Za-z_]\w*)", declarator) + if match is not None: + names.add(match.group(1)) + return names + + SAFE_FILES = { "CMakeLists.txt": """ set(CMAKE_MSVC_RUNTIME_LIBRARY "MultiThreaded$<$:Debug>") @@ -547,6 +582,63 @@ def test_windows_command_line_expectation_is_not_stringized(self) -> None: r"\(\s*argv\s*\)\s*==\s*expected\s*\)\s*;", ) + def test_accepted_server_socket_close_is_bounded_and_drained(self) -> None: + source = (REPO / "third_party/httplib/httplib.h").read_text( + encoding="utf-8" + ) + helper = re.search( + r"inline\s+void\s+drain_and_close_socket\s*\(.*?\n\}", + source, + re.DOTALL, + ) + self.assertIsNotNone(helper, "accepted-socket drain helper") + assert helper is not None + body = helper.group(0) + self.assertIn("shutdown(sock, SD_SEND)", body) + self.assertIn("shutdown(sock, SHUT_WR)", body) + self.assertRegex(body, r"milliseconds\s*\(\s*100\s*\)") + self.assertRegex(body, r"1024u\s*\*\s*1024u") + self.assertLess(body.index("shutdown_socket(sock)"), + body.index("close_socket(sock)")) + + process_close = re.search( + r"inline\s+bool\s+Server::process_and_close_socket\s*\(.*?\n\}", + source, + re.DOTALL, + ) + self.assertIsNotNone(process_close) + assert process_close is not None + self.assertEqual( + process_close.group(0).count( + "detail::drain_and_close_socket(sock);" + ), + 1, + ) + self.assertEqual( + source.count("detail::drain_and_close_socket(sock);"), 1 + ) + + def test_real_socket_tests_use_scoped_server_threads(self) -> None: + source = ( + REPO / "tests/vllm/entrypoints/openai/test_api_server.cpp" + ).read_text(encoding="utf-8") + owner = ( + REPO + / "tests/vllm/entrypoints/openai/scoped_server_thread.h" + ).read_text(encoding="utf-8") + + bound_servers = len( + re.findall(r"\.server\.bind_to_any_port\s*\(", source) + ) + scoped_threads = len( + re.findall(r"ScopedServerThread\s+server_thread\s*\(", source) + ) + self.assertGreater(bound_servers, 0) + self.assertEqual(scoped_threads, bound_servers) + self.assertNotRegex(source, r"std::thread\s+server_thread\b") + self.assertIn("stop_();", owner) + self.assertIn("if (thread_.joinable()) thread_.join();", owner) + def test_cpu_isa_test_owns_ostream_for_string_view_diagnostics(self) -> None: source = (REPO / "tests/vt/test_cpu_isa_x86.cpp").read_text( encoding="utf-8" @@ -562,6 +654,109 @@ def test_cpu_isa_test_owns_ostream_for_string_view_diagnostics(self) -> None: ) self.assertIn("std::string_view::npos", source) + def test_unbound_flash_oracle_names_do_not_shadow_enclosing_oracle(self) -> None: + source = (REPO / "tests/vt/test_backend_cross_device.cpp").read_text( + encoding="utf-8" + ) + start = source.index("// --- Unbind flash layout:") + end = source.index("for (DeviceType dt : RegisteredDevices())", start) + oracle = checker._cpp_structural_view(source[start:end]) + + declarations = { + "unbound_cpu": r"vt::Backend\s*&\s*unbound_cpu\s*=", + "unbound_queue": r"\bQueue\s+unbound_queue\s*=", + "unbound_device": r"\bDevice\s+unbound_device\s*\{", + "unbound_k": r"std::vector\s+unbound_k\s*=", + "unbound_v": r"\bunbound_v\s*=", + "unbound_slots": r"std::vector\s+unbound_slots\s*=", + } + for name, declaration in declarations.items(): + with self.subTest(name=name): + self.assertEqual(len(re.findall(declaration, oracle)), 1) + + forbidden = {"cpu", "cq", "cd", "ck", "cv", "cslots"} + self.assertTrue( + forbidden.isdisjoint(_oracle_declarator_names(oracle)), + "unbound oracle redeclares an enclosing-oracle name", + ) + + def test_unbound_flash_oracle_declarator_scan_handles_initializer_forms( + self) -> None: + declarations = checker._cpp_structural_view(""" + vt::Backend& unbound_cpu = backend, &cpu(get_backend()); + Queue unbound_queue = make_queue(), cq{make_queue()}; + const Device unbound_device{DeviceType::kCPU, 0}, + cd(DeviceType::kCPU, 0); + std::vector unbound_k = input, ck{1.0f, 2.0f}, + unbound_v(input), cv = input; + std::vector unbound_slots = slots, cslots{0, 1}; + """) + self.assertEqual( + {"cpu", "cq", "cd", "ck", "cv", "cslots"}, + _oracle_declarator_names(declarations) + & {"cpu", "cq", "cd", "ck", "cv", "cslots"}, + ) + + def test_cross_device_fused_tier_environment_is_windows_portable(self) -> None: + source = (REPO / "tests/vt/test_backend_cross_device.cpp").read_text( + encoding="utf-8" + ) + helper_span = checker._cpp_function_body_span( + source, r"\bSetTestEnvironment\s*\(" + ) + self.assertIsNotNone(helper_span, "SetTestEnvironment helper") + assert helper_span is not None + helper = source[helper_span[1]:helper_span[2]] + + directives = checker._cpp_directive_view(helper) + self.assertRegex(directives, r"(?m)^\s*#\s*ifdef\s+_WIN32\s*$") + self.assertRegex(directives, r"(?m)^\s*#\s*else\s*$") + self.assertRegex(directives, r"(?m)^\s*#\s*endif\s*$") + + active = checker._cpp_structural_view(helper) + self.assertRegex( + directives, + r"REQUIRE\s*\(\s*::_putenv_s\s*\(\s*name\s*,\s*" + r'value\s*!=\s*nullptr\s*\?\s*value\s*:\s*""\s*\)\s*' + r"==\s*0\s*\)", + ) + self.assertRegex( + active, + r"REQUIRE\s*\(\s*setenv\s*\(\s*name\s*,\s*value\s*,\s*1\s*\)\s*" + r"==\s*0\s*\)", + ) + self.assertRegex( + active, + r"REQUIRE\s*\(\s*unsetenv\s*\(\s*name\s*\)\s*==\s*0\s*\)", + ) + + windows_lines = "\n".join( + line for _, line in checker.windows_possible_lines( + checker.without_cpp_comments(source) + ) + ) + self.assertIn("::_putenv_s", windows_lines) + self.assertNotRegex(windows_lines, r"(? None: source = "src/vt/cuda/nvfp4_persistent_cache.cpp" for condition, expected in (("NOT WIN32", {source}), ("WIN32", set()), ("NOT APPLE", set())): diff --git a/tests/scripts/test_cpu_x86_llamacpp_floor.py b/tests/scripts/test_cpu_x86_llamacpp_floor.py index 358927d17..f7283ae47 100644 --- a/tests/scripts/test_cpu_x86_llamacpp_floor.py +++ b/tests/scripts/test_cpu_x86_llamacpp_floor.py @@ -118,6 +118,36 @@ def run_harness( timeout=300, ) + def test_proc_stat_samples_pair_busy_and_total_from_one_read(self) -> None: + script = SCRIPT.read_text() + + def function(name: str) -> str: + match = re.search(rf"(?ms)^{re.escape(name)}\(\).*?^\}}$", script) + self.assertIsNotNone(match, f"{name} function is missing") + assert match is not None + return match.group(0) + + self.assertNotRegex(script, r"(?m)^stat_(?:busy|total)\(\)") + sample = function("stat_sample") + self.assertEqual(sample.count("/proc/stat"), 1) + self.assertIn("busy=$2+$3+$4+$7+$8+$9", sample) + self.assertIn("for(i=2;i<=NF;i++) total+=$i", sample) + self.assertIn("print busy, total", sample) + + for consumer in ("busy_pct", "run_leg"): + body = function(consumer) + with self.subTest(consumer=consumer): + self.assertEqual( + len( + re.findall( + r"(?m)^\s*read -r b[01] t[01] < <\(stat_sample\)$", + body, + ) + ), + 2, + ) + self.assertNotRegex(body, r"\bstat_(?:busy|total)\b") + def test_runs_to_completion_and_creates_its_own_output_dir(self) -> None: out = self.tmp / "nested" / "evi" # does not exist: the shipped bug got = self.run_harness(out) diff --git a/tests/scripts/test_release_windows_metadata.py b/tests/scripts/test_release_windows_metadata.py index 4ed2cfe30..b2a6c10d2 100644 --- a/tests/scripts/test_release_windows_metadata.py +++ b/tests/scripts/test_release_windows_metadata.py @@ -6,6 +6,9 @@ import argparse import importlib.util import json +import re +import shutil +import subprocess import tempfile import unittest from pathlib import Path @@ -99,6 +102,654 @@ def test_contract_test_precedes_required_release_environment(self) -> None: self.assertLess(contract, contract_exit) self.assertLess(contract_exit, environment) + def test_openai_socket_probe_and_prefix_bisect_precede_unchanged_full_suite(self) -> None: + script = (ROOT / "scripts/build-windows-release.ps1").read_text( + encoding="utf-8" + ) + test_path = ( + '$openaiApiServerTest = Join-Path $BuildDir ' + '"tests/Release/test_openai_api_server.exe"' + ) + probe = ( + "Invoke-Checked $openaiApiServerTest @(\n" + ' "--test-case=api_server: socket smoke *real HTTP requests over an ephemeral port",\n' + ' "--success=true",\n' + ' "--duration=true"\n' + ")" + ) + bisect = "Invoke-OpenAiPrefixBisect -TestProgram $openaiApiServerTest" + full_suite = ( + 'foreach ($test in @(\n' + ' "test_openai_api_server.exe",' + ) + full_suite_invoke = ( + 'Invoke-Checked (Join-Path $BuildDir ' + '"tests/Release/$test") @()' + ) + self.assertEqual(script.count(test_path), 1) + self.assertEqual(script.count(probe), 1) + self.assertEqual(script.count(bisect), 1) + self.assertEqual(script.count(full_suite), 1) + self.assertLess(script.index(test_path), script.index(probe)) + self.assertLess(script.index(probe), script.index(bisect)) + self.assertLess(script.index(bisect), script.index(full_suite)) + + suite_start = script.index(full_suite) + suite_end = script.index("\n}\n", suite_start) + 2 + suite = script[suite_start:suite_end] + body_start = suite.index(")) {") + len(")) {") + self.assertEqual(suite.count(full_suite_invoke), 1) + invoke_start = suite.index(full_suite_invoke, body_start) + self.assertEqual( + suite[body_start:invoke_start].strip(), + "", + "the unchanged full suite must run before any loop-body control flow", + ) + + def test_openai_prefix_bisect_pins_adaptive_exact_status_contract(self) -> None: + script = (ROOT / "scripts/build-windows-release.ps1").read_text( + encoding="utf-8" + ) + range_start = script.find("function Invoke-OpenAiPrefixRange {") + self.assertNotEqual(range_start, -1, "fresh-process prefix range is missing") + range_end = script.index("\n}\n", range_start) + len("\n}\n") + range_helper = script[range_start:range_end] + for argument in ( + '"--order-by=file"', + '"--first=$First"', + '"--last=$Last"', + ): + with self.subTest(argument=argument): + self.assertEqual(range_helper.count(argument), 1) + self.assertEqual(range_helper.count("Invoke-OpenAiProbeProcess"), 1) + + helper_start = script.find("function Invoke-OpenAiPrefixBisect {") + self.assertNotEqual(helper_start, -1, "adaptive prefix bisect is missing") + helper_end = script.index("\n}\n", helper_start) + len("\n}\n") + helper = script[helper_start:helper_end] + for statement in ( + "$expectedFastFailStatus = -1073740791", + '"--list-test-cases"', + "$fullPrefixResult.ExitCode -ne $expectedFastFailStatus", + "$shortPrefixResult.ExitCode -ne 0", + "$prefixResult.ExitCode -eq 0", + "$prefixResult.ExitCode -eq $expectedFastFailStatus", + "$predecessorResult.ExitCode -ne 0", + "$badPrefixResult.ExitCode -ne $expectedFastFailStatus", + "$isolatedResult.ExitCode -eq 0", + "$isolatedResult.ExitCode -eq $expectedFastFailStatus", + 'dependency=$dependency', + ): + with self.subTest(statement=statement): + self.assertIn(statement, helper) + self.assertEqual(helper.count("Invoke-OpenAiPrefixRange"), 6) + self.assertIn("while ($high - $low -gt 1)", helper) + self.assertIn("[math]::Floor(($low + $high) / 2)", helper) + + contract_start = script.find( + "function Invoke-OpenAiPrefixBisectContractTests {" + ) + self.assertNotEqual(contract_start, -1, "prefix bisect contract is missing") + contract_end = script.index("\n}\n", contract_start) + len("\n}\n") + contract = script[contract_start:contract_end] + for guarantee in ( + "$firstBad = 27", + "$testCount = 54", + '"--first=1"', + '"--last=26"', + '"--last=27"', + '"--first=27"', + '"OpenAI prefix bisect contract OK"', + ): + with self.subTest(guarantee=guarantee): + self.assertIn(guarantee, contract) + + def test_openai_isolated_probe_forwards_stable_phase_witness_output(self) -> None: + script = (ROOT / "scripts/build-windows-release.ps1").read_text( + encoding="utf-8" + ) + helper_start = script.find("function Invoke-OpenAiPrefixBisect {") + self.assertNotEqual(helper_start, -1, "adaptive prefix bisect is missing") + helper_end = script.index("\n}\n", helper_start) + len("\n}\n") + helper = script[helper_start:helper_end] + for statement in ( + 'Write-Host "OpenAI isolated output: begin case=$firstBad"', + "foreach ($isolatedOutputLine in @($isolatedResult.Output))", + "Write-Host ([string]$isolatedOutputLine)", + 'Write-Host "OpenAI isolated output: end case=$firstBad"', + ): + with self.subTest(statement=statement): + self.assertEqual(helper.count(statement), 1) + + source = ( + ROOT / "tests/vllm/entrypoints/openai/test_api_server.cpp" + ).read_text(encoding="utf-8") + case_start = source.index( + 'TEST_CASE("api_server: an explicit-cpu device-selected engine serves ' + '/v1/completions") {' + ) + case_end = source.index("\n}\n", case_start) + len("\n}\n") + explicit_cpu_case = source[case_start:case_end] + for phase in ( + "before-make-weights", + "after-make-weights", + "before-build-fixture", + "after-build-fixture", + "before-loaded-engine", + "after-loaded-engine", + "before-serving-stack", + "after-serving-stack", + "before-completion-dispatch", + "after-completion-dispatch", + "before-response-validation", + "after-response-validation", + "before-scope-teardown", + "after-scope-teardown", + ): + with self.subTest(phase=phase): + self.assertEqual( + source.count(f'OpenAiExplicitCpuPhaseWitness("{phase}")'), 1 + ) + ordered_construction = ( + 'OpenAiExplicitCpuPhaseWitness("before-make-weights")', + "Qwen3_5MoeWeights weights = MakeWeights(c)", + 'OpenAiExplicitCpuPhaseWitness("after-make-weights")', + 'OpenAiExplicitCpuPhaseWitness("before-build-fixture")', + "Tokenizer tokenizer = BuildFixture()", + 'OpenAiExplicitCpuPhaseWitness("after-build-fixture")', + 'OpenAiExplicitCpuPhaseWitness("before-loaded-engine")', + "std::move(weights)", + "std::move(tokenizer)", + 'OpenAiExplicitCpuPhaseWitness("after-loaded-engine")', + ) + positions = [] + for statement in ordered_construction: + with self.subTest(statement=statement): + self.assertEqual(explicit_cpu_case.count(statement), 1) + positions.append(explicit_cpu_case.index(statement)) + self.assertEqual(positions, sorted(positions)) + self.assertNotIn( + "LoadedEngine loaded(c, MakeWeights(c), BuildFixture(), params)", + explicit_cpu_case, + ) + witness_start = source.index( + "void OpenAiExplicitCpuPhaseWitness(const char* phase) noexcept {" + ) + witness_end = source.index("\n}", witness_start) + witness = source[witness_start:witness_end] + self.assertEqual(witness.count('"OPENAI_EXPLICIT_CPU_PHASE: %s\\n"'), 1) + self.assertEqual(witness.count("std::fflush(stderr)"), 1) + contract_start = script.index( + "function Invoke-OpenAiPrefixBisectContractTests {" + ) + contract_end = script.index("\n}\n", contract_start) + len("\n}\n") + contract = script[contract_start:contract_end] + for forwarded_line in ( + "OpenAI isolated output: begin case=27", + "OPENAI_EXPLICIT_CPU_PHASE: fixture-isolated", + "OpenAI isolated output: end case=27", + "OpenAI prefix bisect isolated output forwarding contract OK", + ): + with self.subTest(forwarded_line=forwarded_line): + self.assertIn(forwarded_line, contract) + + def test_openai_constructor_diagnostic_is_isolated_ordered_and_nonrecovering(self) -> None: + script = (ROOT / "scripts/build-windows-release.ps1").read_text( + encoding="utf-8" + ) + helper_start = script.find("function Invoke-OpenAiPrefixBisect {") + self.assertNotEqual(helper_start, -1, "adaptive prefix bisect is missing") + helper_end = script.index("\n}\n", helper_start) + len("\n}\n") + helper = script[helper_start:helper_end] + env_name = "VLLM_WINDOWS_CTOR_DIAGNOSTIC" + for statement in ( + f'$diagnosticEnvironmentName = "{env_name}"', + "$priorDiagnosticEnvironment = [Environment]::GetEnvironmentVariable(", + '[Environment]::SetEnvironmentVariable($diagnosticEnvironmentName, "1", "Process")', + "try {", + "finally {", + "[Environment]::SetEnvironmentVariable($diagnosticEnvironmentName, $priorDiagnosticEnvironment, \"Process\")", + ): + with self.subTest(statement=statement): + self.assertEqual(helper.count(statement), 1) + isolated = helper.index("$isolatedResult = Invoke-OpenAiPrefixRange") + enable = helper.index( + '[Environment]::SetEnvironmentVariable($diagnosticEnvironmentName, "1", "Process")' + ) + restore = helper.index( + '[Environment]::SetEnvironmentVariable($diagnosticEnvironmentName, $priorDiagnosticEnvironment, "Process")' + ) + self.assertLess(enable, isolated) + self.assertLess(isolated, restore) + + contract_start = script.index( + "function Invoke-OpenAiPrefixBisectContractTests {" + ) + contract_end = script.index("\n}\n", contract_start) + len("\n}\n") + contract = script[contract_start:contract_end] + for guarantee in ( + "DiagnosticEnvironment = [Environment]::GetEnvironmentVariable(", + 'DiagnosticEnvironment -ceq "1"', + 'DiagnosticEnvironment -ne "1"', + 'Write-Host "OpenAI constructor diagnostic activation contract OK"', + 'Write-Host "OpenAI constructor diagnostic restoration contract OK"', + ): + with self.subTest(guarantee=guarantee): + self.assertIn(guarantee, contract) + + diagnostic = ( + ROOT / "src/vllm/diagnostics/constructor_witness.h" + ).read_text(encoding="utf-8") + for guarantee in ( + f'constexpr const char* kConstructorWitnessEnvironment = "{env_name}"', + '"VLLM_CTOR_DIAGNOSTIC: function=%s stage=%s phase=%s\\n"', + '"VLLM_CTOR_DIAGNOSTIC: function=%s stage=%s phase=%s index=%lld\\n"', + "std::fflush(stderr)", + "std::uncaught_exceptions() == uncaught_exceptions_", + ): + with self.subTest(guarantee=guarantee): + self.assertIn(guarantee, diagnostic) + + loader = (ROOT / "src/vllm/entrypoints/model_loader.cpp").read_text( + encoding="utf-8" + ) + constructor_start = loader.index( + "LoadedEngine::LoadedEngine(HfConfig config,\n" + " std::unique_ptr model," + ) + constructor_end = loader.index("\n}\n", constructor_start) + len("\n}\n") + constructor = loader[constructor_start:constructor_end] + compact_constructor = re.sub(r"\s+", "", constructor) + loaded_stages = ( + "hash_ready_", "config_", "resolved_spec_config_", "dflash_draft_", + "model_", "kv_connector_", "tokenizer_", "kv_cfg_", "max_model_len_", + "max_num_batched_tokens_", "prefix_caching_enabled_", + "jump_forward_enabled_", "runner_", "async_scheduling_enabled_", + "max_concurrent_batches_", "structured_output_manager_", "scheduler_", + "executor_", "engine_core_", "input_processor_", "output_processor_", + "block_hasher_", "engine_", + ) + loader_header = ( + ROOT / "include/vllm/entrypoints/model_loader.h" + ).read_text(encoding="utf-8") + loader_members = loader_header[ + loader_header.index(" bool hash_ready_;"): + loader_header.index(" vllm::v1::LLMEngine engine_;") + + len(" vllm::v1::LLMEngine engine_;") + ] + declared_loaded_stages = tuple( + re.findall( + r"^\s*(?!//)(?:const\s+)?(?:[\w:]+(?:<[^;\n]+>)?)" + r"[\s*&]+([A-Za-z]\w*_)\s*(?:=[^;]*)?;\s*(?://.*)?$", + loader_members, + re.MULTILINE, + ) + ) + self.assertEqual(loaded_stages, declared_loaded_stages) + positions = [] + for stage in loaded_stages: + marker = f'ConstructorWitnessPhase{{"LoadedEngine","{stage}"}}' + with self.subTest(loaded_engine_stage=stage): + self.assertEqual(compact_constructor.count(marker), 1) + positions.append(compact_constructor.index(marker)) + self.assertEqual(positions, sorted(positions)) + model_factory = ( + ROOT / "src/vllm/model_executor/models/qwen3_5_moe.cpp" + ).read_text(encoding="utf-8") + compact_model_factory = re.sub(r"\s+", "", model_factory) + self.assertNotIn("ConstructorWitnessCall", diagnostic + loader) + self.assertIn( + 'return(ConstructorWitnessPhase{"LoadedEngine",' + '"MakeQwen3_5MoeLoadedModel"},std::make_unique<', + compact_model_factory, + ) + + runner = (ROOT / "src/vllm/v1/worker/gpu/runner.cpp").read_text( + encoding="utf-8" + ) + runner_start = runner.index( + "GPUModelRunner::GPUModelRunner(\n const HfConfig& config, LoadedModel& model," + ) + runner_end = runner.index("\n}\n", runner_start) + len("\n}\n") + runner_constructor = runner[runner_start:runner_end] + compact_runner_constructor = re.sub(r"\s+", "", runner_constructor) + runner_stages = ( + "config_", "owned_model_", "model_", "spec_config_", "draft_model_", + "draft_attn_kv_", "queue_", "input_batch_", + ) + runner_header = ( + ROOT / "include/vllm/v1/worker/gpu/runner.h" + ).read_text(encoding="utf-8") + runner_members = runner_header[ + runner_header.index(" const HfConfig& config_;"): + runner_header.index(" InputBatch input_batch_;") + + len(" InputBatch input_batch_;") + ] + declared_runner_stages = tuple( + re.findall( + r"^\s*(?!//)(?:const\s+)?(?:[\w:]+(?:<[^;\n]+>)?)" + r"[\s*&]+([A-Za-z]\w*_)\s*(?:=[^;]*)?;\s*(?://.*)?$", + runner_members, + re.MULTILINE, + ) + ) + self.assertEqual(runner_stages, declared_runner_stages) + runner_positions = [] + for stage in runner_stages: + marker = f'ConstructorWitnessPhase{{"GPUModelRunner","{stage}"}}' + with self.subTest(runner_stage=stage): + self.assertEqual(compact_runner_constructor.count(marker), 1) + runner_positions.append(compact_runner_constructor.index(marker)) + self.assertEqual(runner_positions, sorted(runner_positions)) + for body_stage in ( + "assign-max-num-reqs", "assign-max-num-batched-tokens", + "resolve-async-input-combine", "pooling-model-branch", + "initialize-kv-cache", "model-registry-prepare", + ): + with self.subTest(runner_body_stage=body_stage): + self.assertEqual( + runner_constructor.count( + f'ConstructorWitnessBefore("GPUModelRunner", "{body_stage}")' + ), + 1, + ) + self.assertEqual( + runner_constructor.count( + f'ConstructorWitnessAfter("GPUModelRunner", "{body_stage}")' + ), + 1, + ) + + init_start = runner.index( + "void GPUModelRunner::initialize_kv_cache(const KVCacheConfig& kv_cache_config) {" + ) + init_end = runner.index("\n}\n", init_start) + len("\n}\n") + initialize = runner[init_start:init_end] + compact_initialize = re.sub(r"\s+", "", initialize) + kv_stages = { + "scalar-state-slot-setup": None, + "kv-group-scan": "g", + "mamba-validation": "gdn_group_id_", + "full-attention-geometry": "full_attn_group_id_", + "residency-buffer-setup": None, + "gdn-ssm-allocation": "l", + "gdn-conv-allocation": "l", + "full-attention-allocation": "l", + "full-attention-view": "static_cast(i)", + "draft-attention-storage": "g", + "gdn-state-view": "static_cast(g)", + } + for stage, index in kv_stages.items(): + suffix = "" if index is None else f",{index}" + for phase in ("Before", "After"): + marker = ( + f'ConstructorWitness{phase}(' + f'"GPUModelRunner::initialize_kv_cache","{stage}"' + f'{suffix});' + ) + with self.subTest(kv_stage=stage, phase=phase): + self.assertEqual(compact_initialize.count(marker), 1) + + api_test = ( + ROOT / "tests/vllm/entrypoints/openai/test_api_server.cpp" + ).read_text(encoding="utf-8") + invalid_start = api_test.index( + "[[noreturn]] void WindowsCrtInvalidParameterWitness(" + ) + pure_start = api_test.index( + "[[noreturn]] void WindowsCrtPurecallWitness() {" + ) + invalid_handler = api_test[invalid_start:pure_start] + pure_end = api_test.index("\n}\n#endif", pure_start) + len("\n}\n") + pure_handler = api_test[pure_start:pure_end] + handler_contracts = ( + ( + "invalid-parameter", + invalid_handler, + "[[noreturn]] void WindowsCrtInvalidParameterWitness(", + "WINDOWS_CRT_INVALID_PARAMETER:", + "prior_invalid_parameter_handler(expression, function, file, line, reserved);", + ), + ( + "purecall", + pure_handler, + "[[noreturn]] void WindowsCrtPurecallWitness() {", + "WINDOWS_CRT_PURECALL", + "prior_purecall_handler();", + ), + ) + for name, body, signature, diagnostic_line, prior_call in handler_contracts: + with self.subTest(crt_handler=name): + self.assertTrue(body.startswith(signature)) + self.assertEqual(body.count(diagnostic_line), 1) + self.assertEqual(body.count("std::fflush(stderr)"), 1) + self.assertEqual(body.count(prior_call), 1) + self.assertEqual(body.count("_invoke_watson("), 1) + self.assertLess(body.index(diagnostic_line), body.index("std::fflush(stderr)")) + self.assertLess(body.index("std::fflush(stderr)"), body.index(prior_call)) + self.assertLess(body.index(prior_call), body.index("_invoke_watson(")) + + def test_socket_teardown_probe_marks_each_owner_and_terminate_reason(self) -> None: + main = (ROOT / "tests/doctest_main.cpp").read_text(encoding="utf-8") + handler_start = main.index( + "[[noreturn]] void DiagnosticTerminate() noexcept {" + ) + handler_end = main.index("\n}\n\n} // namespace", handler_start) + 2 + handler = main[handler_start:handler_end] + for diagnostic in ( + "std::current_exception()", + "std::rethrow_exception", + "catch (const std::exception& error)", + "error.what()", + "std::abort()", + ): + with self.subTest(diagnostic=diagnostic): + self.assertEqual(handler.count(diagnostic), 1) + self.assertLess( + handler.index("std::fflush(stderr)"), handler.index("std::abort()") + ) + + entry_start = main.index("int main(int argc, char** argv) {") + entry_end = main.index("\n}", entry_start) + 2 + entry = main[entry_start:entry_end] + terminate_install = "std::set_terminate(DiagnosticTerminate);" + self.assertEqual(entry.count("std::set_terminate("), 1) + self.assertEqual(entry.count(terminate_install), 1) + self.assertLess( + entry.index(terminate_install), + entry.index("doctest::Context(argc, argv).run()"), + ) + + source = ( + ROOT / "tests/vllm/entrypoints/openai/test_api_server.cpp" + ).read_text(encoding="utf-8") + marker_start = source.index("class TeardownProbeMarker {") + marker_end = source.index("\n};", marker_start) + 3 + marker = source[marker_start:marker_end] + destructor_start = marker.index("~TeardownProbeMarker() {") + destructor_end = marker.index("\n }", destructor_start) + 4 + destructor = marker[destructor_start:destructor_end] + emit = "std::fputs(message_, stderr);" + flush = "std::fflush(stderr);" + self.assertEqual(destructor.count(emit), 1) + self.assertEqual(destructor.count(flush), 1) + self.assertLess(destructor.index(emit), destructor.index(flush)) + + start = source.index( + 'TEST_CASE("api_server: socket smoke — real HTTP requests over an ephemeral port")' + ) + end = source.index( + "// Route-registration gate over a real socket", start + ) + socket_smoke = source[start:end] + owners = ( + ("ServerHarness", "h"), + ("ScopedServerThread", "server_thread"), + ("httplib::Client", "client"), + ) + for owner, variable in owners: + with self.subTest(owner=owner): + completed = f'"[vllm-test-probe] {owner} destruction complete\\n"' + started = f'"[vllm-test-probe] {owner} destruction start\\n"' + owner_declaration = f"{owner} {variable}" + self.assertEqual(socket_smoke.count(completed), 1) + self.assertEqual(socket_smoke.count(started), 1) + self.assertLess(socket_smoke.index(completed), + socket_smoke.index(owner_declaration)) + self.assertLess(socket_smoke.index(owner_declaration), + socket_smoke.index(started)) + + def test_invoke_checked_contract_covers_empty_exact_and_failing_arguments(self) -> None: + script = (ROOT / "scripts/build-windows-release.ps1").read_text(encoding="utf-8") + helper_start = script.index("function Invoke-Checked {") + helper_end = script.index("\n}\n", helper_start) + len("\n}\n") + helper = script[helper_start:helper_end] + self.assertIn("[AllowEmptyCollection()][string[]]$Arguments", helper) + self.assertIn("& $Program @Arguments", helper) + self.assertIn("if ($LASTEXITCODE -ne 0)", helper) + + contract_start = script.index("function Invoke-CheckedContractTests {") + contract_end = script.index("\n}\n", contract_start) + len("\n}\n") + contract = script[contract_start:contract_end] + self.assertNotIn("record-arguments.cmd", contract) + self.assertEqual(contract.count('"record-arguments.ps1"'), 1) + self.assertNotIn('"fail.cmd"', contract) + self.assertEqual(contract.count('"fail.ps1"'), 1) + self.assertEqual(contract.count("exit 23"), 1) + for recorder_statement in ( + "[Parameter(ValueFromRemainingArguments = $true)]", + "[string[]]$RemainingArguments = @()", + "Count = @($RemainingArguments).Count", + "Arguments = @($RemainingArguments)", + "ConvertTo-Json -Compress", + ): + with self.subTest(recorder_statement=recorder_statement): + self.assertEqual(contract.count(recorder_statement), 1) + + self.assertEqual( + contract.count( + "$powerShellExecutable = " + "[System.Diagnostics.Process]::GetCurrentProcess().MainModule.FileName" + ), + 1, + ) + self.assertEqual(contract.count("Invoke-Checked $powerShellExecutable @("), 3) + self.assertNotIn("Invoke-Checked $recordingTarget", contract) + self.assertNotIn("Invoke-Checked $failingTarget", contract) + + required_statements = ( + '"-NoProfile", "-NonInteractive", "-File", $recordingTarget)', + "$zeroArgumentRecord = Get-Content -LiteralPath $callLog -Raw | ConvertFrom-Json", + "[int]$zeroArgumentRecord.Count -ne 0", + "@($zeroArgumentRecord.Arguments).Count -ne 0", + '"-NoProfile", "-NonInteractive", "-File", $recordingTarget,', + '"alpha", "two words", "--flag=value")', + "$nonemptyArgumentRecord = Get-Content -LiteralPath $callLog -Raw | ConvertFrom-Json", + "[int]$nonemptyArgumentRecord.Count -ne 3", + "@($nonemptyArgumentRecord.Arguments).Count -ne 3", + '$nonemptyArgumentRecord.Arguments[0] -cne "alpha"', + '$nonemptyArgumentRecord.Arguments[1] -cne "two words"', + '$nonemptyArgumentRecord.Arguments[2] -cne "--flag=value"', + '"-NoProfile", "-NonInteractive", "-File", $failingTarget)', + "if (-not $rejected)", + "$failingChildProcessId = [int](Get-Content -LiteralPath $callLog -Raw)", + "if ($failingChildProcessId -eq $parentProcessId)", + ) + cursor = 0 + for statement in required_statements: + with self.subTest(statement=statement): + self.assertEqual(contract.count(statement), 1) + offset = contract.index(statement) + self.assertGreaterEqual(offset, cursor) + cursor = offset + len(statement) + executable_contract = re.sub( + r"(?ms)^[ \t]*@'\s*$.*?^[ \t]*'@[^\n]*$", + "", + contract, + ) + self.assertNotRegex( + executable_contract, + r"(?m)^\s*(?:return|exit|break|continue)(?:\s|$)", + "Invoke-Checked contract proofs must not be bypassable", + ) + for behavior in ( + "zero-argument target was not invoked exactly once without arguments", + "nonempty arguments did not arrive unchanged", + "nonzero child exit was accepted", + "failure target did not execute in a child process", + ): + with self.subTest(behavior=behavior): + self.assertEqual(contract.count(behavior), 1) + + dispatch = script[script.index("if ($ContractTest)"):] + self.assertEqual(dispatch.count("Invoke-CheckedContractTests"), 1) + + def test_contract_test_executes_real_children_under_powershell_core(self) -> None: + script = (ROOT / "scripts/build-windows-release.ps1").read_text( + encoding="utf-8" + ) + runtime_pid_guard = ( + " if ($failingChildProcessId -eq $parentProcessId) {\n" + ' throw "failure target did not execute in a child process"\n' + " }\n" + ) + self.assertEqual(script.count(runtime_pid_guard), 1) + + pwsh = shutil.which("pwsh") + if pwsh is None: + snap_pwsh = Path("/snap/bin/pwsh") + if snap_pwsh.is_file(): + pwsh = str(snap_pwsh) + if pwsh is None: + self.skipTest("PowerShell Core is not installed") + + result = subprocess.run( + [ + pwsh, + "-NoProfile", + "-NonInteractive", + "-File", + str(ROOT / "scripts/build-windows-release.ps1"), + "-SourceDir", + str(ROOT), + "-ContractTest", + ], + cwd=ROOT, + text=True, + capture_output=True, + check=False, + ) + self.assertEqual(result.returncode, 0, result.stdout + result.stderr) + self.assertIn("Windows PowerShell/CRT contract tests OK", result.stdout) + for proof in ( + "OpenAI prefix bisect listing order contract OK", + "OpenAI prefix bisect unexpected midpoint status contract OK", + "OpenAI prefix bisect unexpected isolated status contract OK", + "OpenAI prefix bisect diagnostic schema contract OK", + "OpenAI prefix bisect isolated output forwarding contract OK", + ): + with self.subTest(proof=proof): + self.assertIn(proof, result.stdout) + self.assertIn( + 'OpenAI prefix bisect: first_bad=27/54 test="aaa source case 27" ' + 'predecessor_status=0 prefix_status=-1073740791 isolated_status=0 ' + 'dependency=cumulative', + result.stdout, + ) + self.assertIn("OpenAI prefix bisect contract OK", result.stdout) + pid_diagnostic = re.search( + r"(?m)^Invoke-Checked PID contract: parent=(?P[0-9]+) " + r"failure_child=(?P[0-9]+)\r?$", + result.stdout, + ) + if pid_diagnostic is None: + self.fail(f"missing Invoke-Checked PID diagnostic:\n{result.stdout}") + parent_process_id = int(pid_diagnostic.group("parent")) + failure_child_process_id = int(pid_diagnostic.group("child")) + self.assertGreater(parent_process_id, 0) + self.assertGreater(failure_child_process_id, 0) + self.assertNotEqual(parent_process_id, failure_child_process_id) + def test_pe_report_rejects_msys_debug_crt_and_developer_paths(self) -> None: with tempfile.TemporaryDirectory() as temporary: args = self.fixture(Path(temporary), "cpu") diff --git a/tests/vllm/entrypoints/openai/scoped_server_thread.h b/tests/vllm/entrypoints/openai/scoped_server_thread.h new file mode 100644 index 000000000..950d97c92 --- /dev/null +++ b/tests/vllm/entrypoints/openai/scoped_server_thread.h @@ -0,0 +1,32 @@ +#pragma once + +#include +#include +#include + +namespace vllm::test { + +// Owns a test server's background thread. Fatal doctest assertions unwind the +// test body, so teardown must stop the server and join before std::thread's +// destructor observes a joinable thread. +class ScopedServerThread { + public: + template + ScopedServerThread(Start&& start, Stop&& stop) + : stop_(std::forward(stop)), + thread_(std::forward(start)) {} + + ~ScopedServerThread() { + stop_(); + if (thread_.joinable()) thread_.join(); + } + + ScopedServerThread(const ScopedServerThread&) = delete; + ScopedServerThread& operator=(const ScopedServerThread&) = delete; + + private: + std::function stop_; + std::thread thread_; +}; + +} // namespace vllm::test diff --git a/tests/vllm/entrypoints/openai/test_api_server.cpp b/tests/vllm/entrypoints/openai/test_api_server.cpp index 8015de896..edccb2213 100644 --- a/tests/vllm/entrypoints/openai/test_api_server.cpp +++ b/tests/vllm/entrypoints/openai/test_api_server.cpp @@ -12,7 +12,9 @@ // (tiny hybrid-MoE Qwen3.6 + the BPE fixture, vocab ids 0..21). #include "vllm/entrypoints/openai/api_server.h" #include "vllm/entrypoints/openai/video_api.h" +#include "vllm/diagnostics/constructor_witness.h" #include "vllm/multimodal/parakeet_transcription.h" +#include "scoped_server_thread.h" #include @@ -35,6 +37,11 @@ #include #endif +#if defined(_MSC_VER) +#include +#include +#endif + #include #include @@ -110,9 +117,98 @@ using vllm::v1::OutputProcessor; using vllm::v1::Scheduler; using vllm::v1::sha256_cbor; using vt::DType; +using vllm::test::ScopedServerThread; namespace { +#if defined(_MSC_VER) +_invalid_parameter_handler prior_invalid_parameter_handler = nullptr; +_purecall_handler prior_purecall_handler = nullptr; + +[[noreturn]] void WindowsCrtInvalidParameterWitness( + const wchar_t* expression, const wchar_t* function, const wchar_t* file, + unsigned int line, uintptr_t reserved) { + std::fprintf(stderr, + "WINDOWS_CRT_INVALID_PARAMETER: expression=%ls function=%ls " + "file=%ls line=%u\n", + expression != nullptr ? expression : L"(null)", + function != nullptr ? function : L"(null)", + file != nullptr ? file : L"(null)", line); + std::fflush(stderr); + if (prior_invalid_parameter_handler != nullptr) { + prior_invalid_parameter_handler(expression, function, file, line, reserved); + } + _invoke_watson(expression, function, file, line, reserved); +} + +[[noreturn]] void WindowsCrtPurecallWitness() { + std::fputs("WINDOWS_CRT_PURECALL\n", stderr); + std::fflush(stderr); + if (prior_purecall_handler != nullptr) { + prior_purecall_handler(); + } + _invoke_watson(L"pure virtual function call", L"_purecall", L"(runtime)", + 0, 0); +} +#endif + +class ScopedWindowsCrtConstructorWitness final { + public: + ScopedWindowsCrtConstructorWitness() noexcept { +#if defined(_MSC_VER) + if (!vllm::diagnostics::ConstructorWitnessEnabled()) return; + prior_invalid_parameter_handler_ = + _set_invalid_parameter_handler(WindowsCrtInvalidParameterWitness); + prior_invalid_parameter_handler = prior_invalid_parameter_handler_; + prior_purecall_handler_ = _set_purecall_handler(WindowsCrtPurecallWitness); + prior_purecall_handler = prior_purecall_handler_; + installed_ = true; +#endif + } + + ScopedWindowsCrtConstructorWitness( + const ScopedWindowsCrtConstructorWitness&) = delete; + ScopedWindowsCrtConstructorWitness& operator=( + const ScopedWindowsCrtConstructorWitness&) = delete; + + ~ScopedWindowsCrtConstructorWitness() noexcept { +#if defined(_MSC_VER) + if (!installed_) return; + _set_purecall_handler(prior_purecall_handler_); + _set_invalid_parameter_handler(prior_invalid_parameter_handler_); + prior_purecall_handler = nullptr; + prior_invalid_parameter_handler = nullptr; +#endif + } + + private: +#if defined(_MSC_VER) + _invalid_parameter_handler prior_invalid_parameter_handler_ = nullptr; + _purecall_handler prior_purecall_handler_ = nullptr; + bool installed_ = false; +#endif +}; + +void OpenAiExplicitCpuPhaseWitness(const char* phase) noexcept { + std::fprintf(stderr, "OPENAI_EXPLICIT_CPU_PHASE: %s\n", phase); + std::fflush(stderr); +} + +class TeardownProbeMarker { + public: + explicit TeardownProbeMarker(const char* message) : message_(message) {} + ~TeardownProbeMarker() { + std::fputs(message_, stderr); + std::fflush(stderr); + } + + TeardownProbeMarker(const TeardownProbeMarker&) = delete; + TeardownProbeMarker& operator=(const TeardownProbeMarker&) = delete; + + private: + const char* message_; +}; + // ─── Synthetic weights (mirrors test_serving.cpp) ──────────────────────────── uint64_t Mix(uint64_t x) { x += 0x9E3779B97F4A7C15ULL; @@ -1234,17 +1330,30 @@ TEST_CASE("api_server: /abort_requests aborts an in-flight engine request") { TEST_CASE("api_server: socket smoke — real HTTP requests over an ephemeral port") { const HfConfig c = MakeConfig(); const Qwen3_5MoeWeights w = MakeWeights(c); + [[maybe_unused]] TeardownProbeMarker harness_destroyed( + "[vllm-test-probe] ServerHarness destruction complete\n"); ServerHarness h(c, w, Fixture()); + [[maybe_unused]] TeardownProbeMarker harness_destroying( + "[vllm-test-probe] ServerHarness destruction start\n"); const int port = h.server.bind_to_any_port("127.0.0.1"); REQUIRE(port > 0); - std::thread server_thread([&h]() { h.server.serve(); }); + [[maybe_unused]] TeardownProbeMarker server_thread_destroyed( + "[vllm-test-probe] ScopedServerThread destruction complete\n"); + ScopedServerThread server_thread([&h]() { h.server.serve(); }, + [&h]() { h.server.stop(); }); + [[maybe_unused]] TeardownProbeMarker server_thread_destroying( + "[vllm-test-probe] ScopedServerThread destruction start\n"); // Wait until the accept loop is up. for (int i = 0; i < 500 && !h.server.is_running(); ++i) std::this_thread::sleep_for(std::chrono::milliseconds(2)); REQUIRE(h.server.is_running()); + [[maybe_unused]] TeardownProbeMarker client_destroyed( + "[vllm-test-probe] httplib::Client destruction complete\n"); httplib::Client client("127.0.0.1", port); + [[maybe_unused]] TeardownProbeMarker client_destroying( + "[vllm-test-probe] httplib::Client destruction start\n"); client.set_connection_timeout(5, 0); client.set_read_timeout(15, 0); @@ -1297,8 +1406,6 @@ TEST_CASE("api_server: socket smoke — real HTTP requests over an ephemeral por CHECK(j.at("choices").at(0).at("message").at("role") == "assistant"); } - h.server.stop(); - server_thread.join(); } // Route-registration gate over a real socket: /tokenizer_info is ABSENT (404) @@ -1317,7 +1424,8 @@ TEST_CASE("api_server: /tokenizer_info + /abort_requests are opt-in routes") { h.server.set_tokenizer(&Fixture(), kMaxModelLen); const int port = h.server.bind_to_any_port("127.0.0.1"); REQUIRE(port > 0); - std::thread server_thread([&h]() { h.server.serve(); }); + ScopedServerThread server_thread([&h]() { h.server.serve(); }, + [&h]() { h.server.stop(); }); for (int i = 0; i < 500 && !h.server.is_running(); ++i) std::this_thread::sleep_for(std::chrono::milliseconds(2)); REQUIRE(h.server.is_running()); @@ -1332,8 +1440,6 @@ TEST_CASE("api_server: /tokenizer_info + /abort_requests are opt-in routes") { REQUIRE(abort); CHECK(abort->status == 404); // no callback → route not registered - h.server.stop(); - server_thread.join(); } SUBCASE("backings attached → routes serve (200)") { @@ -1348,7 +1454,8 @@ TEST_CASE("api_server: /tokenizer_info + /abort_requests are opt-in routes") { }); const int port = h.server.bind_to_any_port("127.0.0.1"); REQUIRE(port > 0); - std::thread server_thread([&h]() { h.server.serve(); }); + ScopedServerThread server_thread([&h]() { h.server.serve(); }, + [&h]() { h.server.stop(); }); for (int i = 0; i < 500 && !h.server.is_running(); ++i) std::this_thread::sleep_for(std::chrono::milliseconds(2)); REQUIRE(h.server.is_running()); @@ -1367,8 +1474,6 @@ TEST_CASE("api_server: /tokenizer_info + /abort_requests are opt-in routes") { CHECK(json::parse(abort->body).at("aborted") == 2); CHECK(aborted_calls == 1); - h.server.stop(); - server_thread.join(); } } @@ -1387,15 +1492,14 @@ TEST_CASE("api_server: ConfigureUtilityEndpoints wires the production C8 surface auto with_server = [](ServerHarness& h, auto&& body) { const int port = h.server.bind_to_any_port("127.0.0.1"); REQUIRE(port > 0); - std::thread server_thread([&h]() { h.server.serve(); }); + ScopedServerThread server_thread([&h]() { h.server.serve(); }, + [&h]() { h.server.stop(); }); for (int i = 0; i < 500 && !h.server.is_running(); ++i) std::this_thread::sleep_for(std::chrono::milliseconds(2)); REQUIRE(h.server.is_running()); httplib::Client client("127.0.0.1", port); client.set_read_timeout(5, 0); body(client); - h.server.stop(); - server_thread.join(); }; // RED: a default production server WITHOUT the wiring seam 404s every C8 route, @@ -1529,7 +1633,8 @@ TEST_CASE("api_server: concurrent requests share AsyncLLM without state races") const int port = h.server.bind_to_any_port("127.0.0.1"); REQUIRE(port > 0); - std::thread server_thread([&h]() { h.server.serve(); }); + ScopedServerThread server_thread([&h]() { h.server.serve(); }, + [&h]() { h.server.stop(); }); for (int i = 0; i < 500 && !h.server.is_running(); ++i) std::this_thread::sleep_for(std::chrono::milliseconds(2)); REQUIRE(h.server.is_running()); @@ -1569,8 +1674,6 @@ TEST_CASE("api_server: concurrent requests share AsyncLLM without state races") for (int i = 1; i < kClients; ++i) CHECK(texts[static_cast(i)] == texts[0]); - h.server.stop(); - server_thread.join(); } TEST_CASE("api_server: configured persistent-stream capacity remains readable") { @@ -1584,7 +1687,8 @@ TEST_CASE("api_server: configured persistent-stream capacity remains readable") kStreamCapacity + ApiServer::kControlWorkerHeadroom); const int port = h.server.bind_to_any_port("127.0.0.1"); REQUIRE(port > 0); - std::thread server_thread([&h]() { h.server.serve(); }); + ScopedServerThread server_thread([&h]() { h.server.serve(); }, + [&h]() { h.server.stop(); }); for (int i = 0; i < 500 && !h.server.is_running(); ++i) std::this_thread::sleep_for(std::chrono::milliseconds(2)); REQUIRE(h.server.is_running()); @@ -1616,8 +1720,6 @@ TEST_CASE("api_server: configured persistent-stream capacity remains readable") CHECK(response->status == 200); parked.clear(); - h.server.stop(); - server_thread.join(); } TEST_CASE("api_server: stream capacity must be positive") { @@ -1657,7 +1759,8 @@ TEST_CASE( const int port = h.server.bind_to_any_port("127.0.0.1"); REQUIRE(port > 0); - std::thread server_thread([&h]() { h.server.serve(); }); + ScopedServerThread server_thread([&h]() { h.server.serve(); }, + [&h]() { h.server.stop(); }); for (int i = 0; i < 500 && !h.server.is_running(); ++i) std::this_thread::sleep_for(std::chrono::milliseconds(2)); REQUIRE(h.server.is_running()); @@ -1695,8 +1798,6 @@ TEST_CASE( CHECK(nodelay == 1); // RED until ApiServer calls set_tcp_nodelay(true) ::close(client_fd); - h.server.stop(); - server_thread.join(); #endif // defined(__linux__) } @@ -2096,15 +2197,14 @@ TEST_CASE("api_server: the /v1/videos routes do not exist without a runner") { auto with_server = [](ServerHarness& h, auto&& body) { const int port = h.server.bind_to_any_port("127.0.0.1"); REQUIRE(port > 0); - std::thread server_thread([&h]() { h.server.serve(); }); + ScopedServerThread server_thread([&h]() { h.server.serve(); }, + [&h]() { h.server.stop(); }); for (int i = 0; i < 500 && !h.server.is_running(); ++i) std::this_thread::sleep_for(std::chrono::milliseconds(2)); REQUIRE(h.server.is_running()); httplib::Client client("127.0.0.1", port); client.set_read_timeout(5, 0); body(client); - h.server.stop(); - server_thread.join(); }; SUBCASE("no runner: every video route 404s, and the core routes are unaffected") { @@ -2262,7 +2362,8 @@ TEST_CASE("api_server: transcriptions socket smoke (multipart), generate routes AsrHarness h; const int port = h.server.bind_to_any_port("127.0.0.1"); REQUIRE(port > 0); - std::thread server_thread([&h]() { h.server.serve(); }); + ScopedServerThread server_thread([&h]() { h.server.serve(); }, + [&h]() { h.server.stop(); }); for (int i = 0; i < 500 && !h.server.is_running(); ++i) std::this_thread::sleep_for(std::chrono::milliseconds(2)); REQUIRE(h.server.is_running()); @@ -2309,8 +2410,6 @@ TEST_CASE("api_server: transcriptions socket smoke (multipart), generate routes "parakeet-fixture"); } - h.server.stop(); - server_thread.join(); } TEST_CASE("api_server: the audio routes do not exist on a TEXT server") { @@ -2328,7 +2427,8 @@ TEST_CASE("api_server: the audio routes do not exist on a TEXT server") { const int port = h.server.bind_to_any_port("127.0.0.1"); REQUIRE(port > 0); - std::thread server_thread([&h]() { h.server.serve(); }); + ScopedServerThread server_thread([&h]() { h.server.serve(); }, + [&h]() { h.server.stop(); }); for (int i = 0; i < 500 && !h.server.is_running(); ++i) std::this_thread::sleep_for(std::chrono::milliseconds(2)); REQUIRE(h.server.is_running()); @@ -2361,8 +2461,6 @@ TEST_CASE("api_server: the audio routes do not exist on a TEXT server") { CHECK(health->status == 200); } - h.server.stop(); - server_thread.join(); } // ─── ARCH-ONE-SURFACE ROW 8: the server's --device seam ────────────────────── @@ -2374,6 +2472,7 @@ TEST_CASE("api_server: the audio routes do not exist on a TEXT server") { // accelerator; explicit cuda never falls back) is test_loaded_engine_dense.cpp; // the C-ABI plumb is test_capi.cpp. TEST_CASE("api_server: an explicit-cpu device-selected engine serves /v1/completions") { + ScopedWindowsCrtConstructorWitness crt_witness; const HfConfig c = MakeConfig(); vllm::entrypoints::EngineParams params; params.block_size = kBlockSize; @@ -2383,28 +2482,48 @@ TEST_CASE("api_server: an explicit-cpu device-selected engine serves /v1/complet // The server's own parse of `--device cpu` (an unknown name throws there at // startup; pinned in test_loaded_engine_dense.cpp). params.device = vllm::DeviceFromString("cpu"); - vllm::entrypoints::LoadedEngine loaded(c, MakeWeights(c), BuildFixture(), - params); - // The observable seam: the runner of the explicitly-cpu engine is on the CPU - // device (on a CUDA build this is the force-CPU pin; auto would select CUDA). - CHECK(loaded.runner().device().type == vt::DeviceType::kCPU); - - OpenAIServingModels models("test-model"); - OpenAIServingCompletion completion(loaded.async_engine(), "test-model", - /*enable_force_include_usage=*/false); - OpenAIServingChat chat(loaded.async_engine(), "test-model", InVocabChatPrompt, - "hermes", /*reasoning_parser_name=*/std::string(), - /*enable_force_include_usage=*/false); - ApiServer server(completion, chat, models, "9.9.9"); - - const std::string body = - R"({"model":"test-model","prompt":"hello","max_tokens":5,"temperature":0.0})"; - ApiServer::DispatchResult r = server.handle_completions(body); - CHECK(r.status == 200); - json j = json::parse(r.body); - CHECK(j.at("object") == "text_completion"); - CHECK(j.at("choices").at(0).at("finish_reason") == "length"); - CHECK(j.at("usage").at("completion_tokens") == 5); + OpenAiExplicitCpuPhaseWitness("before-make-weights"); + Qwen3_5MoeWeights weights = MakeWeights(c); + OpenAiExplicitCpuPhaseWitness("after-make-weights"); + OpenAiExplicitCpuPhaseWitness("before-build-fixture"); + Tokenizer tokenizer = BuildFixture(); + OpenAiExplicitCpuPhaseWitness("after-build-fixture"); + OpenAiExplicitCpuPhaseWitness("before-loaded-engine"); + { + vllm::entrypoints::LoadedEngine loaded( + c, std::move(weights), std::move(tokenizer), params); + OpenAiExplicitCpuPhaseWitness("after-loaded-engine"); + // The observable seam: the runner of the explicitly-cpu engine is on the + // CPU device (on a CUDA build this is the force-CPU pin; auto would select + // CUDA). + CHECK(loaded.runner().device().type == vt::DeviceType::kCPU); + + OpenAiExplicitCpuPhaseWitness("before-serving-stack"); + OpenAIServingModels models("test-model"); + OpenAIServingCompletion completion(loaded.async_engine(), "test-model", + /*enable_force_include_usage=*/false); + OpenAIServingChat chat( + loaded.async_engine(), "test-model", InVocabChatPrompt, "hermes", + /*reasoning_parser_name=*/std::string(), + /*enable_force_include_usage=*/false); + ApiServer server(completion, chat, models, "9.9.9"); + OpenAiExplicitCpuPhaseWitness("after-serving-stack"); + + const std::string body = + R"({"model":"test-model","prompt":"hello","max_tokens":5,"temperature":0.0})"; + OpenAiExplicitCpuPhaseWitness("before-completion-dispatch"); + ApiServer::DispatchResult r = server.handle_completions(body); + OpenAiExplicitCpuPhaseWitness("after-completion-dispatch"); + OpenAiExplicitCpuPhaseWitness("before-response-validation"); + CHECK(r.status == 200); + json j = json::parse(r.body); + CHECK(j.at("object") == "text_completion"); + CHECK(j.at("choices").at(0).at("finish_reason") == "length"); + CHECK(j.at("usage").at("completion_tokens") == 5); + OpenAiExplicitCpuPhaseWitness("after-response-validation"); + OpenAiExplicitCpuPhaseWitness("before-scope-teardown"); + } + OpenAiExplicitCpuPhaseWitness("after-scope-teardown"); } // ─── /v1/embeddings (ARCH-ONE-SURFACE ROW 6) ───────────────────────────────── @@ -2516,7 +2635,8 @@ TEST_CASE("api_server: embeddings socket smoke; generate routes 404 on the " EmbedHarness h; const int port = h.server.bind_to_any_port("127.0.0.1"); REQUIRE(port > 0); - std::thread server_thread([&h]() { h.server.serve(); }); + ScopedServerThread server_thread([&h]() { h.server.serve(); }, + [&h]() { h.server.stop(); }); for (int i = 0; i < 500 && !h.server.is_running(); ++i) std::this_thread::sleep_for(std::chrono::milliseconds(2)); REQUIRE(h.server.is_running()); @@ -2552,8 +2672,6 @@ TEST_CASE("api_server: embeddings socket smoke; generate routes 404 on the " "llama-embed-fixture"); } - h.server.stop(); - server_thread.join(); } TEST_CASE("api_server: /v1/embeddings does not exist on a TEXT server") { @@ -2569,7 +2687,8 @@ TEST_CASE("api_server: /v1/embeddings does not exist on a TEXT server") { const int port = h.server.bind_to_any_port("127.0.0.1"); REQUIRE(port > 0); - std::thread server_thread([&h]() { h.server.serve(); }); + ScopedServerThread server_thread([&h]() { h.server.serve(); }, + [&h]() { h.server.stop(); }); for (int i = 0; i < 500 && !h.server.is_running(); ++i) std::this_thread::sleep_for(std::chrono::milliseconds(2)); REQUIRE(h.server.is_running()); @@ -2584,8 +2703,6 @@ TEST_CASE("api_server: /v1/embeddings does not exist on a TEXT server") { CHECK(res->status == 404); } - h.server.stop(); - server_thread.join(); } TEST_CASE("platform process: Windows command line preserves every argv byte") { diff --git a/tests/vllm/entrypoints/openai/test_httplib_accepted_socket_close.cpp b/tests/vllm/entrypoints/openai/test_httplib_accepted_socket_close.cpp new file mode 100644 index 000000000..3e53d4cda --- /dev/null +++ b/tests/vllm/entrypoints/openai/test_httplib_accepted_socket_close.cpp @@ -0,0 +1,57 @@ +#include + +#include +#include +#include + +#include + +TEST_CASE("cpp-httplib: accepted socket drains unread peer bytes before close") { + httplib::Server server; + server.Get("/complete", [](const httplib::Request&, httplib::Response& res) { + res.set_content("completed-response", "text/plain"); + }); + + const int port = server.bind_to_any_port("127.0.0.1"); + REQUIRE(port > 0); + std::thread server_thread([&server]() { server.listen_after_bind(); }); + auto cleanup = httplib::detail::scope_exit([&]() { + server.stop(); + if (server_thread.joinable()) server_thread.join(); + }); + server.wait_until_ready(); + + httplib::Error error = httplib::Error::Success; + const socket_t sock = httplib::detail::create_client_socket( + "127.0.0.1", "", port, AF_UNSPEC, false, false, nullptr, + /*connection_timeout_sec=*/5, /*connection_timeout_usec=*/0, + /*read_timeout_sec=*/5, /*read_timeout_usec=*/0, + /*write_timeout_sec=*/5, /*write_timeout_usec=*/0, "", error); + REQUIRE(sock != INVALID_SOCKET); + auto close_client = httplib::detail::scope_exit( + [&]() { httplib::detail::close_socket(sock); }); + + std::string request = + "GET /complete HTTP/1.1\r\nHost: 127.0.0.1\r\nConnection: close\r\n\r\n"; + request.append(64 * 1024, 'x'); + size_t sent = 0; + while (sent < request.size()) { + const auto n = httplib::detail::send_socket( + sock, request.data() + sent, request.size() - sent, + CPPHTTPLIB_SEND_FLAGS); + REQUIRE(n > 0); + sent += static_cast(n); + } + + std::string response; + char buffer[4096]; + ssize_t received = 0; + while ((received = httplib::detail::read_socket( + sock, buffer, sizeof(buffer), CPPHTTPLIB_RECV_FLAGS)) > 0) { + response.append(buffer, static_cast(received)); + } + + CHECK(response.find("completed-response") != std::string::npos); + CHECK_MESSAGE(received == 0, + "accepted-socket close reset the completed response"); +} diff --git a/tests/vllm/entrypoints/openai/test_server_thread_failure_fixture.cpp b/tests/vllm/entrypoints/openai/test_server_thread_failure_fixture.cpp new file mode 100644 index 000000000..1b6c9625c --- /dev/null +++ b/tests/vllm/entrypoints/openai/test_server_thread_failure_fixture.cpp @@ -0,0 +1,36 @@ +#include "scoped_server_thread.h" + +#include + +#include +#include +#include + +using vllm::test::ScopedServerThread; + +TEST_CASE("scoped server thread reports assertion failures without terminating") { + std::mutex mutex; + std::condition_variable changed; + bool running = false; + bool stopped = false; + + ScopedServerThread server_thread( + [&]() { + std::unique_lock lock(mutex); + running = true; + changed.notify_all(); + changed.wait(lock, [&]() { return stopped; }); + }, + [&]() { + std::lock_guard lock(mutex); + stopped = true; + changed.notify_all(); + }); + + { + std::unique_lock lock(mutex); + REQUIRE(changed.wait_for(lock, std::chrono::seconds(5), + [&]() { return running; })); + } + REQUIRE_MESSAGE(false, "scoped teardown fixture assertion"); +} diff --git a/tests/vllm/multimodal/test_video_engine.cpp b/tests/vllm/multimodal/test_video_engine.cpp index 8854fd3a8..3fcc433db 100644 --- a/tests/vllm/multimodal/test_video_engine.cpp +++ b/tests/vllm/multimodal/test_video_engine.cpp @@ -152,6 +152,25 @@ TEST_CASE("video engine registry: detection resolves the H3 checkpoint by what i CHECK(vllm::multimodal::DetectVideoFamilies(not_a_dit).empty()); } +TEST_CASE("video checkpoint reader: directory, regular file, and missing path stay distinct") { + SeamWorkspace ws; + std::vector names; + std::string why; + + CHECK_FALSE(vllm::multimodal::ReadVideoCheckpointTensorNames(ws.fixture, &names, &why)); + CHECK(why.find("directory holds neither") != std::string::npos); + + why.clear(); + CHECK_FALSE(vllm::multimodal::ReadVideoCheckpointTensorNames( + ws.fixture + "/video_vae_config.json", &names, &why)); + CHECK(why.find("neither GGUF nor safetensors") != std::string::npos); + + why.clear(); + CHECK_FALSE(vllm::multimodal::ReadVideoCheckpointTensorNames( + ws.root + "/missing-checkpoint", &names, &why)); + CHECK(why == "no such file or directory"); +} + // ─── the byte-identity gate: the ABSTRACT seam lands on the pre-fold bytes ─── TEST_CASE("video engine seam: an auto-detected H3 render reproduces the pre-fold goldens") { SeamWorkspace ws; diff --git a/tests/vt/test_backend_cross_device.cpp b/tests/vt/test_backend_cross_device.cpp index 186500e77..19b5a1cc5 100644 --- a/tests/vt/test_backend_cross_device.cpp +++ b/tests/vt/test_backend_cross_device.cpp @@ -511,15 +511,18 @@ TEST_CASE("ReshapeAndCache scatters into the KV cache BIT-EXACTLY") { } std::vector ref_comb = combined; { - vt::Backend& cpu = vt::GetBackend(DeviceType::kCPU); - Queue cq = cpu.CreateQueue(); - const Device cd{DeviceType::kCPU, 0}; - std::vector ck = knew, cv = vnew, cslots_f; - std::vector cslots = slots; - Tensor tk = Tensor::Contiguous(ck.data(), DType::kF32, cd, {kTokens, kHk, kD}); - Tensor tv = Tensor::Contiguous(cv.data(), DType::kF32, cd, {kTokens, kHk, kD}); + vt::Backend& unbound_cpu = vt::GetBackend(DeviceType::kCPU); + Queue unbound_queue = unbound_cpu.CreateQueue(); + const Device unbound_device{DeviceType::kCPU, 0}; + std::vector unbound_k = knew, unbound_v = vnew, cslots_f; + std::vector unbound_slots = slots; + Tensor tk = Tensor::Contiguous(unbound_k.data(), DType::kF32, unbound_device, + {kTokens, kHk, kD}); + Tensor tv = Tensor::Contiguous(unbound_v.data(), DType::kF32, unbound_device, + {kTokens, kHk, kD}); Tensor tcomb = - Tensor::Contiguous(ref_comb.data(), DType::kF32, cd, {kBlocks * 2 * within}); + Tensor::Contiguous(ref_comb.data(), DType::kF32, unbound_device, + {kBlocks * 2 * within}); auto slice = [&](int which) { Tensor t = tcomb; t.data = static_cast(t.data) + @@ -535,10 +538,11 @@ TEST_CASE("ReshapeAndCache scatters into the KV cache BIT-EXACTLY") { t.stride[3] = 1; return t; }; - Tensor tsm = Tensor::Contiguous(cslots.data(), DType::kI64, cd, {kTokens}); + Tensor tsm = Tensor::Contiguous(unbound_slots.data(), DType::kI64, + unbound_device, {kTokens}); Tensor tkc = slice(0), tvc = slice(1); - vt::ReshapeAndCache(cq, tk, tv, tkc, tvc, tsm); - cpu.DestroyQueue(cq); + vt::ReshapeAndCache(unbound_queue, tk, tv, tkc, tvc, tsm); + unbound_cpu.DestroyQueue(unbound_queue); } for (DeviceType dt : RegisteredDevices()) { diff --git a/third_party/httplib/httplib.h b/third_party/httplib/httplib.h index 90498af2b..9eb3162c0 100644 --- a/third_party/httplib/httplib.h +++ b/third_party/httplib/httplib.h @@ -5820,6 +5820,39 @@ inline int shutdown_socket(socket_t sock) noexcept { #endif } +// Half-close the write side and drain any in-flight/queued bytes before the +// final shutdown+close. Closing with unread data in the receive queue, or with +// bytes arriving after the receive side is closed, can make the stack send an +// abortive RST and make a fully written response appear to have failed. +// Backported from cpp-httplib 8e702d3837b2164765ca1d98cb6d180ae4711e70. +inline void drain_and_close_socket(socket_t sock) noexcept { +#ifdef _WIN32 + shutdown(sock, SD_SEND); +#else + shutdown(sock, SHUT_WR); +#endif + + char buf[CPPHTTPLIB_RECV_BUFSIZ]; + size_t total = 0; + const auto deadline = std::chrono::steady_clock::now() + + std::chrono::milliseconds(100); // bound #1 + + while (total < size_t(1024u * 1024u)) { // bound #2 + const auto remaining = + std::chrono::duration_cast( + deadline - std::chrono::steady_clock::now()) + .count(); + if (remaining <= 0) { break; } + if (select_read(sock, 0, static_cast(remaining)) <= 0) { break; } + const auto n = read_socket(sock, buf, sizeof(buf), CPPHTTPLIB_RECV_FLAGS); + if (n <= 0) { break; } + total += static_cast(n); + } + + shutdown_socket(sock); + close_socket(sock); +} + inline std::string escape_abstract_namespace_unix_domain(const std::string &s) { if (s.size() > 1 && s[0] == '\0') { auto ret = s; @@ -12686,8 +12719,7 @@ inline bool Server::process_and_close_socket(socket_t sock) { nullptr, &websocket_upgraded); }); - detail::shutdown_socket(sock); - detail::close_socket(sock); + detail::drain_and_close_socket(sock); return ret; }