fix(meg): correct M100 measurement (standards-only events + in-window peak search) + study_status#13
Merged
Conversation
… peak search) + study_status The meg_validation study reported the auditory M100 peak at ~15 ms (refuted) on the Brainstorm auditory dataset, where the real M100 is ~100-120 ms. Root cause (confirmed by a direct mne probe on the real data): 1. measure_peak searched window_ms +/- 100 ms (~-20..220 ms for an 80-120 ms window), so it grabbed the large early stimulus artifact at ~15 ms instead of the M100. Probe A (all-events + wide search) -> peak 15 ms. 2. epoch did not filter events: it epoched ALL triggers (standards + deviants + button presses), muddying the average. Probe B (standards-only + search within 0.08-0.12 s) -> peak 120 ms, a real in-window response -> supported. Fixes: - Extract a pure, mne-free _peak_search_bounds(window_ms, times_lo, times_hi) helper that keeps the search WITHIN the expected window (small +/-10 ms margin), clamped to available times. measure_peak now calls it. This is the unit that would have caught the +/-100 ms bug. - epoch now filters to a single trigger code: default None -> auto most-frequent (the standard tone); an int via bounds.event_id selects explicitly; "all" is a diagnostic escape hatch. event_id threads through the step result/artifact. - Raise default crop_tmax 30 -> 90 s for enough standard trials for a clean peak; still clearly memory-bounded (channel-subset + resample keep the loaded array small) and per-step checkpointed. - study_status returned an error envelope on a real run because json.dumps emits bare NaN/Infinity for non-finite measured floats, which is invalid JSON. Tool envelopes now sanitize non-finite floats to null and dump with allow_nan=False. Tests (stdlib-only; CI has no mne/numpy): - _peak_search_bounds regression: (80,120) over wide times -> ~(0.08,0.12), not (-0.02,0.22); plus a clamp-to-available-times case. - epoch event-filtering: a fake io records the event_id; the study applies the standards filter by default (not "all"), and bounds.event_id overrides. - study_status: returns strict-JSON-parseable structured result for a completed study whose finding evidence holds NaN/Infinity. Full suite: 136 passed, 7 skipped. No top-level mne/numpy imports added. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
juniperbevensee
added a commit
that referenced
this pull request
Jun 22, 2026
) (#15) * feat(meg): skeptical validation diagnostics + offline golden recipe (#14) Two layers on top of the M100 measurement fix (#13), prompted by the live test where a wrong measurement was nearly accepted because the pipeline ran without error. 1. Skepticism in the study itself. `validate_finding` now withholds a *confident* `refuted` drawn from a noise-dominated average: below MIN_RELIABLE_EPOCHS (or with no measurable amplitude) an out-of-window peak is downgraded to `inconclusive` with a stated `next_step`, never silently accepted. Findings carry the material to sanity-check them — search_window_ms, channel, n_epochs, and a caveats list — so the agent reasons over evidence, not a single number. 2. A shipped, offline golden recipe. SyntheticMegIO + build_golden_steps run the SAME five-step pipeline and verdict logic against a planted in-window M100 — no mne / numpy / network / download. Exposed as study kind 'golden_meg_validation' so the agent can run it live as the self- demonstrating reference for "what correct looks like," and as an offline CI smoke/regression test. docs/golden-validation-recipe.md documents both the supported path and the skeptical-inconclusive path. 3. Methodology, not just tooling. SOUL gains a "be skeptical of your own measurements" principle; SKILL gains a "Reasoning Over Datasets" section: a result contradicting a well-established finding is a red flag to inspect, prefer inconclusive+next-step over a confident refuted on thin evidence, and imitate the golden recipe. Tests: skepticism boundaries (few-epochs out-of-window -> inconclusive; enough-epochs -> still refuted), diagnostics presence, golden recipe supported/inconclusive/no-heavy-deps, and the wired tool kind end-to-end. 143 passed, 7 skipped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(meg): lock the MIN_RELIABLE_EPOCHS boundary (review nit N3) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Juniper Bevensee <juniperbevensee@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Confirmed root cause (live, real data)
The
meg_validationstudy reported the auditory M100 peak at ~15 ms (refuted) on the Brainstorm auditory dataset, where the real M100 is ~100–120 ms. A direct mne probe on the real data isolated two compounding defects:measure_peaksearched too wide. It searchedwindow_ms ± 100 ms(≈ −20 ms to 220 ms in abs mode for an 80–120 ms window), so it grabbed the large early stimulus artifact at ~15 ms instead of the M100.epochdid not filter events. It usedmne.find_events(raw)and epoched all triggers (in a 30 s crop: standards=16, deviants=3, other/button=3), muddying the average.event_id=1) + search within 0.08–0.12 s → peak 120 ms, a real in-window auditory response → would besupported.So the fix is: filter to the standard-tone event and search within the expected window (no ±100 ms widening).
Fixes
_peak_search_bounds(window_ms, times_lo, times_hi) -> (lo, hi)that keeps the search within the expected window (small symmetric ±10 ms margin), clamped to available sample times._MneIO.measure_peakcalls it thenev.get_peak(tmin=lo, tmax=hi, ...). No more ±100 ms widening._MneIO.epochfilters to a single trigger code.event_idsemantics:None→ auto = most-frequent code (the standard tone) [default]; anint(viabounds.event_id) selects explicitly;"all"is a diagnostic escape hatch (legacy unfiltered behavior). The resolvedevent_idthreads through the step result + artifact meta so a fake io can assert filtering is applied.crop_tmax30 → 90 s so the standards-only average has enough trials for a clean peak (a 30 s crop yielded only ~16 standard tones). Still clearly memory-safe: the channel-subset + resample keep the loaded array to a few MB, and per-step checkpointing means an OOM resumes from the last completed step. Documented inline.study_statusenvelope fix. On a real run,study_statusreturned an error becausejson.dumps(..., default=str)emits bareNaN/Infinityfor non-finite measured floats — invalid JSON, which a strict downstream parser rejects (the output began{"study_id":2,...then broke). Tool envelopes (_tool_result/_tool_error) now recursively sanitize non-finite floats tonulland dump withallow_nan=False, so every envelope is strictly valid JSON.New tests (stdlib-only — CI has no mne/numpy)
test_peak_search_bounds_stay_in_window_not_widened_by_100ms— the regression test that would have caught the ±100 ms bug: window (80,120) over wide times → ~(0.08, 0.12), not (−0.02, 0.22); the ~15 ms artifact is outside the search.test_peak_search_bounds_clamped_to_available_times— clamps to available samples.test_epoch_applies_standards_event_filter_by_default— a fake io records theevent_id; the study applies the standards filter by default (not"all").test_epoch_event_id_overridable_via_bounds—bounds.event_idoverrides.test_study_status_returns_strict_json_for_completed_study— completed study withNaN/Infinityin finding evidence returns a strict-JSON-parseable structured result (non-finite →null), not an error envelope.Verification
.venv/bin/python -m pytest -q→ 136 passed, 7 skipped (opt-inMATILDE_LIVE=1test stays skipped in CI).import matilde_plugin.engine.meg_studyworks without mne.Do not merge — left open for independent audit.
🤖 Generated with Claude Code