Skip to content

fix(meg): correct M100 measurement (standards-only events + in-window peak search) + study_status#13

Merged
juniperbevensee merged 1 commit into
mainfrom
dev/juniperbevensee/meg-analysis-fix
Jun 22, 2026
Merged

fix(meg): correct M100 measurement (standards-only events + in-window peak search) + study_status#13
juniperbevensee merged 1 commit into
mainfrom
dev/juniperbevensee/meg-analysis-fix

Conversation

@juniperbevensee

Copy link
Copy Markdown
Collaborator

Confirmed root cause (live, real data)

The meg_validation study reported the auditory M100 peak at ~15 ms (refuted) on the Brainstorm auditory dataset, where the real M100 is ~100–120 ms. A direct mne probe on the real data isolated two compounding defects:

  1. measure_peak searched too wide. It searched window_ms ± 100 ms (≈ −20 ms to 220 ms in abs mode for an 80–120 ms window), so it grabbed the large early stimulus artifact at ~15 ms instead of the M100.
    • Probe A — all-events + wide search → peak 15 ms.
  2. epoch did not filter events. It used mne.find_events(raw) and epoched all triggers (in a 30 s crop: standards=16, deviants=3, other/button=3), muddying the average.
    • Probe B — standards-only (event_id=1) + search within 0.08–0.12 s → peak 120 ms, a real in-window auditory response → would be supported.

So the fix is: filter to the standard-tone event and search within the expected window (no ±100 ms widening).

Fixes

  • In-window peak search. Extracted a pure, mne-free _peak_search_bounds(window_ms, times_lo, times_hi) -> (lo, hi) that keeps the search within the expected window (small symmetric ±10 ms margin), clamped to available sample times. _MneIO.measure_peak calls it then ev.get_peak(tmin=lo, tmax=hi, ...). No more ±100 ms widening.
  • Standards-only epoching. _MneIO.epoch filters to a single trigger code. event_id semantics: None → auto = most-frequent code (the standard tone) [default]; an int (via bounds.event_id) selects explicitly; "all" is a diagnostic escape hatch (legacy unfiltered behavior). The resolved event_id threads through the step result + artifact meta so a fake io can assert filtering is applied.
  • More trials, still bounded. Raised default crop_tmax 30 → 90 s so the standards-only average has enough trials for a clean peak (a 30 s crop yielded only ~16 standard tones). Still clearly memory-safe: the channel-subset + resample keep the loaded array to a few MB, and per-step checkpointing means an OOM resumes from the last completed step. Documented inline.
  • study_status envelope fix. On a real run, study_status returned an error because json.dumps(..., default=str) emits bare NaN/Infinity for non-finite measured floats — invalid JSON, which a strict downstream parser rejects (the output began {"study_id":2,... then broke). Tool envelopes (_tool_result/_tool_error) now recursively sanitize non-finite floats to null and dump with allow_nan=False, so every envelope is strictly valid JSON.

New tests (stdlib-only — CI has no mne/numpy)

  • test_peak_search_bounds_stay_in_window_not_widened_by_100ms — the regression test that would have caught the ±100 ms bug: window (80,120) over wide times → ~(0.08, 0.12), not (−0.02, 0.22); the ~15 ms artifact is outside the search.
  • test_peak_search_bounds_clamped_to_available_times — clamps to available samples.
  • test_epoch_applies_standards_event_filter_by_default — a fake io records the event_id; the study applies the standards filter by default (not "all").
  • test_epoch_event_id_overridable_via_boundsbounds.event_id overrides.
  • test_study_status_returns_strict_json_for_completed_study — completed study with NaN/Infinity in finding evidence returns a strict-JSON-parseable structured result (non-finite → null), not an error envelope.

Verification

  • .venv/bin/python -m pytest -q136 passed, 7 skipped (opt-in MATILDE_LIVE=1 test stays skipped in CI).
  • No top-level mne/numpy imports added; import matilde_plugin.engine.meg_study works without mne.
  • Deterministic sanitization layer clean.

Do not merge — left open for independent audit.

🤖 Generated with Claude Code

… peak search) + study_status

The meg_validation study reported the auditory M100 peak at ~15 ms (refuted)
on the Brainstorm auditory dataset, where the real M100 is ~100-120 ms.

Root cause (confirmed by a direct mne probe on the real data):
1. measure_peak searched window_ms +/- 100 ms (~-20..220 ms for an 80-120 ms
   window), so it grabbed the large early stimulus artifact at ~15 ms instead
   of the M100. Probe A (all-events + wide search) -> peak 15 ms.
2. epoch did not filter events: it epoched ALL triggers (standards + deviants +
   button presses), muddying the average. Probe B (standards-only + search
   within 0.08-0.12 s) -> peak 120 ms, a real in-window response -> supported.

Fixes:
- Extract a pure, mne-free _peak_search_bounds(window_ms, times_lo, times_hi)
  helper that keeps the search WITHIN the expected window (small +/-10 ms
  margin), clamped to available times. measure_peak now calls it. This is the
  unit that would have caught the +/-100 ms bug.
- epoch now filters to a single trigger code: default None -> auto most-frequent
  (the standard tone); an int via bounds.event_id selects explicitly; "all" is a
  diagnostic escape hatch. event_id threads through the step result/artifact.
- Raise default crop_tmax 30 -> 90 s for enough standard trials for a clean
  peak; still clearly memory-bounded (channel-subset + resample keep the loaded
  array small) and per-step checkpointed.
- study_status returned an error envelope on a real run because json.dumps emits
  bare NaN/Infinity for non-finite measured floats, which is invalid JSON. Tool
  envelopes now sanitize non-finite floats to null and dump with allow_nan=False.

Tests (stdlib-only; CI has no mne/numpy):
- _peak_search_bounds regression: (80,120) over wide times -> ~(0.08,0.12), not
  (-0.02,0.22); plus a clamp-to-available-times case.
- epoch event-filtering: a fake io records the event_id; the study applies the
  standards filter by default (not "all"), and bounds.event_id overrides.
- study_status: returns strict-JSON-parseable structured result for a completed
  study whose finding evidence holds NaN/Infinity.
Full suite: 136 passed, 7 skipped. No top-level mne/numpy imports added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@juniperbevensee
juniperbevensee merged commit e079993 into main Jun 22, 2026
1 check passed
@juniperbevensee
juniperbevensee deleted the dev/juniperbevensee/meg-analysis-fix branch June 22, 2026 22:37
juniperbevensee added a commit that referenced this pull request Jun 22, 2026
) (#15)

* feat(meg): skeptical validation diagnostics + offline golden recipe (#14)

Two layers on top of the M100 measurement fix (#13), prompted by the live
test where a wrong measurement was nearly accepted because the pipeline ran
without error.

1. Skepticism in the study itself. `validate_finding` now withholds a
   *confident* `refuted` drawn from a noise-dominated average: below
   MIN_RELIABLE_EPOCHS (or with no measurable amplitude) an out-of-window peak
   is downgraded to `inconclusive` with a stated `next_step`, never silently
   accepted. Findings carry the material to sanity-check them — search_window_ms,
   channel, n_epochs, and a caveats list — so the agent reasons over evidence,
   not a single number.

2. A shipped, offline golden recipe. SyntheticMegIO + build_golden_steps run
   the SAME five-step pipeline and verdict logic against a planted in-window
   M100 — no mne / numpy / network / download. Exposed as study kind
   'golden_meg_validation' so the agent can run it live as the self-
   demonstrating reference for "what correct looks like," and as an offline CI
   smoke/regression test. docs/golden-validation-recipe.md documents both the
   supported path and the skeptical-inconclusive path.

3. Methodology, not just tooling. SOUL gains a "be skeptical of your own
   measurements" principle; SKILL gains a "Reasoning Over Datasets" section:
   a result contradicting a well-established finding is a red flag to inspect,
   prefer inconclusive+next-step over a confident refuted on thin evidence,
   and imitate the golden recipe.

Tests: skepticism boundaries (few-epochs out-of-window -> inconclusive;
enough-epochs -> still refuted), diagnostics presence, golden recipe
supported/inconclusive/no-heavy-deps, and the wired tool kind end-to-end.
143 passed, 7 skipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(meg): lock the MIN_RELIABLE_EPOCHS boundary (review nit N3)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Juniper Bevensee <juniperbevensee@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant