Summary
A clean run of the compiled Test262 baseline (Test262CompiledTests.CompiledBaseline) against the committed baselines/compiled.txt shows 62 regressions and 108 new passes (net +46; suite still improved overall). All 62 regressions are deterministic — the 20 sampled in the run log reproduced 20/20, and the full set was reconstructed and independently re-verified (882 candidate paths re-run → exactly 62, zero flake).
The root context: baselines/compiled.txt is 187 commits stale — last regenerated at 0494a763 (#693, 2026-06-15), while baselines/interpreted.txt is current as of b9d3aa3f (#802, 2026-06-17). So this drift is the accumulated compiled-mode delta across 187 commits, never recorded.
Cross-referencing each regressed path against the (current) interpreted baseline splits the 62 cleanly:
| Class |
Count |
Meaning |
Compiled-only (interp still Pass) |
14 |
Genuine compiled/codegen regressions — interpreter gets these right, compiled does not. Priority. |
Shared (interp baseline already Fail/RuntimeError) |
48 |
Behavior changes already accepted on the interpreted side at #802; the stale compiled baseline just never recorded them. |
Buckets: 37 RuntimeError + 25 Fail.
Reproduce
dotnet build SharpTS.Test262.Worker/SharpTS.Test262.Worker.csproj -c Release # worker is NOT rebuilt by dotnet test
dotnet test SharpTS.Test262/SharpTS.Test262.csproj -c Release \
--filter "FullyQualifiedName~CompiledBaseline" # ~17 min; run backgrounded
# Regenerate (only after the 14 compiled-only are triaged — don't bury them):
# SHARPTS_TEST262_UPDATE_BASELINE=1 dotnet test ... --filter "FullyQualifiedName~CompiledBaseline"
The 14 compiled-only regressions (priority — codegen bugs)
These pass in the interpreter but fail compiled. Three themes:
-
Sloppy-mode this not bound to the global object — Array/prototype/filter/15.4.4.20-5-1, some/15.4.4.17-5-1 (callback reads this._x where this should be global), language/expressions/call/11.2.3-3_8 (this.bar(foo()) at top level). Compiled throws instead of resolving this → globalThis.
-
Object.defineProperty/defineProperties/create reading descriptor fields off an attributes object that has an accessor without a getter — 15.2.3.6-3-215..217, 15.2.3.7-5-b-201..203, 15.2.3.5-4-241..243. e.g. Object.defineProperty(attrs, "get", {set(){}}) then defineProperty(obj, "p", attrs); reading attrs.get must yield undefined, compiled throws (RuntimeError).
-
eval with exotic whitespace between new and the operand — language/expressions/new/S11.2.2_A1.1, A1.2 evaluate new<WS>Number for WS ∈ {U+0009 tab, U+000B VT, U+000C FF, U+00A0 NBSP, U+2028, U+2029, …} via eval(...) and expect 0.
-
test/built-ins/Array/prototype/filter/15.4.4.20-5-1.js Pass -> RuntimeError (interp: Pass)
-
test/built-ins/Array/prototype/some/15.4.4.17-5-1.js Pass -> RuntimeError (interp: Pass)
-
test/built-ins/Object/create/15.2.3.5-4-241.js Pass -> RuntimeError (interp: Pass)
-
test/built-ins/Object/create/15.2.3.5-4-242.js Pass -> RuntimeError (interp: Pass)
-
test/built-ins/Object/create/15.2.3.5-4-243.js Pass -> RuntimeError (interp: Pass)
-
test/built-ins/Object/defineProperties/15.2.3.7-5-b-201.js Pass -> RuntimeError (interp: Pass)
-
test/built-ins/Object/defineProperties/15.2.3.7-5-b-202.js Pass -> RuntimeError (interp: Pass)
-
test/built-ins/Object/defineProperties/15.2.3.7-5-b-203.js Pass -> RuntimeError (interp: Pass)
-
test/built-ins/Object/defineProperty/15.2.3.6-3-215.js Pass -> RuntimeError (interp: Pass)
-
test/built-ins/Object/defineProperty/15.2.3.6-3-216.js Pass -> RuntimeError (interp: Pass)
-
test/built-ins/Object/defineProperty/15.2.3.6-3-217.js Pass -> RuntimeError (interp: Pass)
-
test/language/expressions/call/11.2.3-3_8.js Pass -> Fail (interp: Pass)
-
test/language/expressions/new/S11.2.2_A1.1.js Pass -> Fail (interp: Pass)
-
test/language/expressions/new/S11.2.2_A1.2.js Pass -> Fail (interp: Pass)
The 48 shared regressions (interp baseline already records these failing)
Lower priority — these mirror behavior already accepted in interpreted.txt at #802; regenerating the compiled baseline will fold them in. They cluster in the same Object.* descriptor area, sloppy-this Promise reaction handlers (*-nonstrict/*-sloppy), sparse-array inherited-accessor iteration (Array/prototype/{reduce,reduceRight,...}), and for-in/propertyIsEnumerable DontEnum attribute tests (Number.MAX_VALUE/MIN_VALUE, Boolean/Error/String prototype).
Full list of all 48 shared regressions
test/built-ins/Array/prototype/every/15.4.4.16-7-c-i-22.js Pass -> Fail (interp: RuntimeError)
test/built-ins/Array/prototype/filter/15.4.4.20-5-30.js Pass -> RuntimeError (interp: Fail)
test/built-ins/Array/prototype/filter/15.4.4.20-9-c-i-22.js Pass -> Fail (interp: RuntimeError)
test/built-ins/Array/prototype/forEach/15.4.4.18-5-25.js Pass -> RuntimeError (interp: Fail)
test/built-ins/Array/prototype/forEach/15.4.4.18-7-c-i-22.js Pass -> Fail (interp: RuntimeError)
test/built-ins/Array/prototype/indexOf/15.4.4.14-9-b-i-21.js Pass -> Fail (interp: RuntimeError)
test/built-ins/Array/prototype/lastIndexOf/15.4.4.15-8-b-i-20.js Pass -> Fail (interp: RuntimeError)
test/built-ins/Array/prototype/lastIndexOf/15.4.4.15-8-b-i-21.js Pass -> Fail (interp: RuntimeError)
test/built-ins/Array/prototype/map/15.4.4.19-8-c-i-22.js Pass -> Fail (interp: RuntimeError)
test/built-ins/Array/prototype/reduce/15.4.4.21-8-b-iii-1-18.js Pass -> Fail (interp: RuntimeError)
test/built-ins/Array/prototype/reduce/15.4.4.21-8-b-iii-1-20.js Pass -> Fail (interp: RuntimeError)
test/built-ins/Array/prototype/reduce/15.4.4.21-8-b-iii-1-22.js Pass -> Fail (interp: RuntimeError)
test/built-ins/Array/prototype/reduce/15.4.4.21-8-c-4.js Pass -> RuntimeError (interp: RuntimeError)
test/built-ins/Array/prototype/reduce/15.4.4.21-9-c-i-18.js Pass -> Fail (interp: RuntimeError)
test/built-ins/Array/prototype/reduce/15.4.4.21-9-c-i-20.js Pass -> Fail (interp: RuntimeError)
test/built-ins/Array/prototype/reduce/15.4.4.21-9-c-i-22.js Pass -> Fail (interp: RuntimeError)
test/built-ins/Array/prototype/reduceRight/15.4.4.22-8-b-iii-1-22.js Pass -> Fail (interp: RuntimeError)
test/built-ins/Array/prototype/reduceRight/15.4.4.22-8-c-4.js Pass -> RuntimeError (interp: RuntimeError)
test/built-ins/Array/prototype/reduceRight/15.4.4.22-9-c-i-18.js Pass -> Fail (interp: RuntimeError)
test/built-ins/Array/prototype/reduceRight/15.4.4.22-9-c-i-20.js Pass -> Fail (interp: RuntimeError)
test/built-ins/Array/prototype/reduceRight/15.4.4.22-9-c-i-22.js Pass -> Fail (interp: RuntimeError)
test/built-ins/Array/prototype/some/15.4.4.17-5-25.js Pass -> RuntimeError (interp: Fail)
test/built-ins/Array/prototype/some/15.4.4.17-7-c-i-22.js Pass -> Fail (interp: RuntimeError)
test/built-ins/Boolean/prototype/S15.6.3.1_A4.js Pass -> RuntimeError (interp: RuntimeError)
test/built-ins/Error/prototype/S15.11.3.1_A2_T1.js Pass -> RuntimeError (interp: RuntimeError)
test/built-ins/Error/the-initial-value-of-errorprototypemessage-is-the-empty-string.js Pass -> RuntimeError (interp: RuntimeError)
test/built-ins/Number/MAX_VALUE/S15.7.3.2_A4.js Pass -> RuntimeError (interp: RuntimeError)
test/built-ins/Number/MIN_VALUE/S15.7.3.3_A4.js Pass -> RuntimeError (interp: RuntimeError)
test/built-ins/Object/defineProperties/15.2.3.7-6-a-102.js Pass -> RuntimeError (interp: Fail)
test/built-ins/Object/defineProperties/15.2.3.7-6-a-92.js Pass -> RuntimeError (interp: Fail)
test/built-ins/Object/defineProperties/15.2.3.7-6-a-93.js Pass -> RuntimeError (interp: RuntimeError)
test/built-ins/Object/defineProperty/15.2.3.6-4-108.js Pass -> RuntimeError (interp: Fail)
test/built-ins/Object/defineProperty/15.2.3.6-4-470.js Pass -> RuntimeError (interp: Fail)
test/built-ins/Object/preventExtensions/15.2.3.10-3-10.js Pass -> RuntimeError (interp: RuntimeError)
test/built-ins/Object/preventExtensions/15.2.3.10-3-18.js Pass -> RuntimeError (interp: Fail)
test/built-ins/Object/preventExtensions/15.2.3.10-3-19.js Pass -> RuntimeError (interp: Fail)
test/built-ins/Object/preventExtensions/15.2.3.10-3-20.js Pass -> RuntimeError (interp: RuntimeError)
test/built-ins/Object/preventExtensions/15.2.3.10-3-8.js Pass -> RuntimeError (interp: Fail)
test/built-ins/Object/preventExtensions/15.2.3.10-3-9.js Pass -> RuntimeError (interp: Fail)
test/built-ins/Object/prototype/S15.2.3.1_A2.js Pass -> RuntimeError (interp: RuntimeError)
test/built-ins/Promise/executor-call-context-sloppy.js Pass -> Fail (interp: RuntimeError)
test/built-ins/Promise/prototype/finally/rejected-observable-then-calls-argument.js Pass -> RuntimeError (interp: RuntimeError)
test/built-ins/Promise/prototype/then/rxn-handler-fulfilled-invoke-nonstrict.js Pass -> RuntimeError (interp: RuntimeError)
test/built-ins/Promise/prototype/then/rxn-handler-rejected-invoke-nonstrict.js Pass -> RuntimeError (interp: RuntimeError)
test/built-ins/String/prototype/S15.5.3.1_A2.js Pass -> RuntimeError (interp: RuntimeError)
test/language/expressions/new/spread-err-mult-err-expr-throws.js Pass -> Fail (interp: Fail)
test/language/expressions/new/spread-err-sngl-err-expr-throws.js Pass -> Fail (interp: Fail)
test/language/expressions/property-accessors/S11.2.1_A4_T1.js Pass -> Fail (interp: RuntimeError)
Recommended next steps
- Triage/fix the 14 compiled-only regressions — start with the sloppy-mode
this → globalThis binding in compiled call/codegen, which spans the Array, call, and (in the shared set) Promise-reaction clusters.
- Spot-check the
Object.defineProperty descriptor-extraction path (accessor-without-getter on the attributes object) — contiguous test numbers suggest one codegen path.
- Then regenerate
baselines/compiled.txt (SHARPTS_TEST262_UPDATE_BASELINE=1) so the 108 genuine new passes (incl. parser gains: ParseError 88→10) are recorded and the baseline stops being 187 commits stale.
Provenance
- Branch:
main @ 33f266f9; full compiled run: 11,384 tests in 1007.5 s → Pass=7942 Fail=1843 ParseError=10 RuntimeError=690 Timeout=2 HarnessError=18 Skipped=879.
- Subset config:
config/subset.json (timeout 15 s). Env: win32, .NET 10, 12 logical cores.
- Regressions independently re-verified deterministic (882-path targeted re-run, recycle-safe).
Summary
A clean run of the compiled Test262 baseline (
Test262CompiledTests.CompiledBaseline) against the committedbaselines/compiled.txtshows 62 regressions and 108 new passes (net +46; suite still improved overall). All 62 regressions are deterministic — the 20 sampled in the run log reproduced 20/20, and the full set was reconstructed and independently re-verified (882 candidate paths re-run → exactly 62, zero flake).The root context:
baselines/compiled.txtis 187 commits stale — last regenerated at0494a763(#693, 2026-06-15), whilebaselines/interpreted.txtis current as ofb9d3aa3f(#802, 2026-06-17). So this drift is the accumulated compiled-mode delta across 187 commits, never recorded.Cross-referencing each regressed path against the (current) interpreted baseline splits the 62 cleanly:
Pass)Fail/RuntimeError)Buckets: 37
RuntimeError+ 25Fail.Reproduce
The 14 compiled-only regressions (priority — codegen bugs)
These pass in the interpreter but fail compiled. Three themes:
Sloppy-mode
thisnot bound to the global object —Array/prototype/filter/15.4.4.20-5-1,some/15.4.4.17-5-1(callback readsthis._xwherethisshould be global),language/expressions/call/11.2.3-3_8(this.bar(foo())at top level). Compiled throws instead of resolvingthis→ globalThis.Object.defineProperty/defineProperties/createreading descriptor fields off an attributes object that has an accessor without a getter —15.2.3.6-3-215..217,15.2.3.7-5-b-201..203,15.2.3.5-4-241..243. e.g.Object.defineProperty(attrs, "get", {set(){}})thendefineProperty(obj, "p", attrs); readingattrs.getmust yieldundefined, compiled throws (RuntimeError).evalwith exotic whitespace betweennewand the operand —language/expressions/new/S11.2.2_A1.1,A1.2evaluatenew<WS>Numberfor WS ∈ {U+0009 tab, U+000B VT, U+000C FF, U+00A0 NBSP, U+2028, U+2029, …} viaeval(...)and expect0.test/built-ins/Array/prototype/filter/15.4.4.20-5-1.jsPass -> RuntimeError (interp: Pass)test/built-ins/Array/prototype/some/15.4.4.17-5-1.jsPass -> RuntimeError (interp: Pass)test/built-ins/Object/create/15.2.3.5-4-241.jsPass -> RuntimeError (interp: Pass)test/built-ins/Object/create/15.2.3.5-4-242.jsPass -> RuntimeError (interp: Pass)test/built-ins/Object/create/15.2.3.5-4-243.jsPass -> RuntimeError (interp: Pass)test/built-ins/Object/defineProperties/15.2.3.7-5-b-201.jsPass -> RuntimeError (interp: Pass)test/built-ins/Object/defineProperties/15.2.3.7-5-b-202.jsPass -> RuntimeError (interp: Pass)test/built-ins/Object/defineProperties/15.2.3.7-5-b-203.jsPass -> RuntimeError (interp: Pass)test/built-ins/Object/defineProperty/15.2.3.6-3-215.jsPass -> RuntimeError (interp: Pass)test/built-ins/Object/defineProperty/15.2.3.6-3-216.jsPass -> RuntimeError (interp: Pass)test/built-ins/Object/defineProperty/15.2.3.6-3-217.jsPass -> RuntimeError (interp: Pass)test/language/expressions/call/11.2.3-3_8.jsPass -> Fail (interp: Pass)test/language/expressions/new/S11.2.2_A1.1.jsPass -> Fail (interp: Pass)test/language/expressions/new/S11.2.2_A1.2.jsPass -> Fail (interp: Pass)The 48 shared regressions (interp baseline already records these failing)
Lower priority — these mirror behavior already accepted in
interpreted.txtat #802; regenerating the compiled baseline will fold them in. They cluster in the sameObject.*descriptor area, sloppy-thisPromise reaction handlers (*-nonstrict/*-sloppy), sparse-array inherited-accessor iteration (Array/prototype/{reduce,reduceRight,...}), andfor-in/propertyIsEnumerableDontEnum attribute tests (Number.MAX_VALUE/MIN_VALUE,Boolean/Error/Stringprototype).Full list of all 48 shared regressions
test/built-ins/Array/prototype/every/15.4.4.16-7-c-i-22.jsPass -> Fail (interp: RuntimeError)test/built-ins/Array/prototype/filter/15.4.4.20-5-30.jsPass -> RuntimeError (interp: Fail)test/built-ins/Array/prototype/filter/15.4.4.20-9-c-i-22.jsPass -> Fail (interp: RuntimeError)test/built-ins/Array/prototype/forEach/15.4.4.18-5-25.jsPass -> RuntimeError (interp: Fail)test/built-ins/Array/prototype/forEach/15.4.4.18-7-c-i-22.jsPass -> Fail (interp: RuntimeError)test/built-ins/Array/prototype/indexOf/15.4.4.14-9-b-i-21.jsPass -> Fail (interp: RuntimeError)test/built-ins/Array/prototype/lastIndexOf/15.4.4.15-8-b-i-20.jsPass -> Fail (interp: RuntimeError)test/built-ins/Array/prototype/lastIndexOf/15.4.4.15-8-b-i-21.jsPass -> Fail (interp: RuntimeError)test/built-ins/Array/prototype/map/15.4.4.19-8-c-i-22.jsPass -> Fail (interp: RuntimeError)test/built-ins/Array/prototype/reduce/15.4.4.21-8-b-iii-1-18.jsPass -> Fail (interp: RuntimeError)test/built-ins/Array/prototype/reduce/15.4.4.21-8-b-iii-1-20.jsPass -> Fail (interp: RuntimeError)test/built-ins/Array/prototype/reduce/15.4.4.21-8-b-iii-1-22.jsPass -> Fail (interp: RuntimeError)test/built-ins/Array/prototype/reduce/15.4.4.21-8-c-4.jsPass -> RuntimeError (interp: RuntimeError)test/built-ins/Array/prototype/reduce/15.4.4.21-9-c-i-18.jsPass -> Fail (interp: RuntimeError)test/built-ins/Array/prototype/reduce/15.4.4.21-9-c-i-20.jsPass -> Fail (interp: RuntimeError)test/built-ins/Array/prototype/reduce/15.4.4.21-9-c-i-22.jsPass -> Fail (interp: RuntimeError)test/built-ins/Array/prototype/reduceRight/15.4.4.22-8-b-iii-1-22.jsPass -> Fail (interp: RuntimeError)test/built-ins/Array/prototype/reduceRight/15.4.4.22-8-c-4.jsPass -> RuntimeError (interp: RuntimeError)test/built-ins/Array/prototype/reduceRight/15.4.4.22-9-c-i-18.jsPass -> Fail (interp: RuntimeError)test/built-ins/Array/prototype/reduceRight/15.4.4.22-9-c-i-20.jsPass -> Fail (interp: RuntimeError)test/built-ins/Array/prototype/reduceRight/15.4.4.22-9-c-i-22.jsPass -> Fail (interp: RuntimeError)test/built-ins/Array/prototype/some/15.4.4.17-5-25.jsPass -> RuntimeError (interp: Fail)test/built-ins/Array/prototype/some/15.4.4.17-7-c-i-22.jsPass -> Fail (interp: RuntimeError)test/built-ins/Boolean/prototype/S15.6.3.1_A4.jsPass -> RuntimeError (interp: RuntimeError)test/built-ins/Error/prototype/S15.11.3.1_A2_T1.jsPass -> RuntimeError (interp: RuntimeError)test/built-ins/Error/the-initial-value-of-errorprototypemessage-is-the-empty-string.jsPass -> RuntimeError (interp: RuntimeError)test/built-ins/Number/MAX_VALUE/S15.7.3.2_A4.jsPass -> RuntimeError (interp: RuntimeError)test/built-ins/Number/MIN_VALUE/S15.7.3.3_A4.jsPass -> RuntimeError (interp: RuntimeError)test/built-ins/Object/defineProperties/15.2.3.7-6-a-102.jsPass -> RuntimeError (interp: Fail)test/built-ins/Object/defineProperties/15.2.3.7-6-a-92.jsPass -> RuntimeError (interp: Fail)test/built-ins/Object/defineProperties/15.2.3.7-6-a-93.jsPass -> RuntimeError (interp: RuntimeError)test/built-ins/Object/defineProperty/15.2.3.6-4-108.jsPass -> RuntimeError (interp: Fail)test/built-ins/Object/defineProperty/15.2.3.6-4-470.jsPass -> RuntimeError (interp: Fail)test/built-ins/Object/preventExtensions/15.2.3.10-3-10.jsPass -> RuntimeError (interp: RuntimeError)test/built-ins/Object/preventExtensions/15.2.3.10-3-18.jsPass -> RuntimeError (interp: Fail)test/built-ins/Object/preventExtensions/15.2.3.10-3-19.jsPass -> RuntimeError (interp: Fail)test/built-ins/Object/preventExtensions/15.2.3.10-3-20.jsPass -> RuntimeError (interp: RuntimeError)test/built-ins/Object/preventExtensions/15.2.3.10-3-8.jsPass -> RuntimeError (interp: Fail)test/built-ins/Object/preventExtensions/15.2.3.10-3-9.jsPass -> RuntimeError (interp: Fail)test/built-ins/Object/prototype/S15.2.3.1_A2.jsPass -> RuntimeError (interp: RuntimeError)test/built-ins/Promise/executor-call-context-sloppy.jsPass -> Fail (interp: RuntimeError)test/built-ins/Promise/prototype/finally/rejected-observable-then-calls-argument.jsPass -> RuntimeError (interp: RuntimeError)test/built-ins/Promise/prototype/then/rxn-handler-fulfilled-invoke-nonstrict.jsPass -> RuntimeError (interp: RuntimeError)test/built-ins/Promise/prototype/then/rxn-handler-rejected-invoke-nonstrict.jsPass -> RuntimeError (interp: RuntimeError)test/built-ins/String/prototype/S15.5.3.1_A2.jsPass -> RuntimeError (interp: RuntimeError)test/language/expressions/new/spread-err-mult-err-expr-throws.jsPass -> Fail (interp: Fail)test/language/expressions/new/spread-err-sngl-err-expr-throws.jsPass -> Fail (interp: Fail)test/language/expressions/property-accessors/S11.2.1_A4_T1.jsPass -> Fail (interp: RuntimeError)Recommended next steps
this→ globalThis binding in compiled call/codegen, which spans the Array,call, and (in the shared set) Promise-reaction clusters.Object.definePropertydescriptor-extraction path (accessor-without-getter on the attributes object) — contiguous test numbers suggest one codegen path.baselines/compiled.txt(SHARPTS_TEST262_UPDATE_BASELINE=1) so the 108 genuine new passes (incl. parser gains:ParseError88→10) are recorded and the baseline stops being 187 commits stale.Provenance
main@33f266f9; full compiled run: 11,384 tests in 1007.5 s →Pass=7942 Fail=1843 ParseError=10 RuntimeError=690 Timeout=2 HarnessError=18 Skipped=879.config/subset.json(timeout 15 s). Env: win32, .NET 10, 12 logical cores.