Skip to content

feat(engine): trusted-ingestion parse path + detection cap#37

Merged
sepehr-safari merged 1 commit into
mainfrom
feat/trusted-ingestion
Jul 11, 2026
Merged

feat(engine): trusted-ingestion parse path + detection cap#37
sepehr-safari merged 1 commit into
mainfrom
feat/trusted-ingestion

Conversation

@sepehr-safari

Copy link
Copy Markdown
Member

Summary

Lift Studio past the browser toolkit's ceiling for traces the user opens — the M8
"beyond-browser capacity" differentiator — while keeping the strict caps for
untrusted input.

  • Ingestion trust boundary (ADR-0007). The size/event caps are now a
    parser.Limits value chosen by the source. parseTrace keeps the browser-scale
    untrusted_limits (10 MB / 10k) unchanged; parseTraceTrusted applies
    trusted_limits (256 MB / 2M) for files the user explicitly opened. The
    workspace opens files trusted; future live capture (S5) stays untrusted.
  • 500k / 100 MB capacity, proven. A 500 000-event JSONL trace parses (one line
    at a time — no whole-file JSON tree), correlates, and the timeline's virtual
    window stays viewport-sized. All on the load path is O(n) / O(n·sessions).
  • Detection cap (honest, surfaced). Several detection rules are O(n²) in the
    event count (they mirror the toolkit's algorithms — fine at 10k, a load-time
    stall at 500k). Past detection.max_events_for_detection (50 000) the workspace
    skips detection: the trace still parses, correlates, and is fully
    inspectable, and the status bar shows detection skipped (large trace) instead
    of a misleading empty result. The O(n) rewrite is tracked in perf(engine): O(n) detection rules for dataset-scale traces #36 (with the
    contract-v1 goldens as the bit-exactness safety net).

Verification

  • native test -Dplatform=null69/69 pass (macOS; CI runs Linux too). New
    coverage: the trusted path parses a 500 000-event trace; parseTraceWithLimits
    honors explicit byte/event caps; the untrusted default still rejects >10 MB /

    10k; a trace past the detection cap loads fully with detection_skipped set and
    zero failures; the sample stays under the cap and runs detection. The 15
    conformance goldens still match (the harness parses untrusted, uncapped).

  • native check --strict clean; native build produces the ReleaseFast binary.

Security note

This raises an untrusted-input bound only for trusted sources (files the user
opened), keyed to source, not globally. The untrusted parseTrace path — the one
future live-socket capture will use — is unchanged. The trusted cap is bounded
(256 MB / 2M), so a pathological trusted file still cannot exhaust memory.

Closes #29

Lift Studio past the browser's ceiling for files the user opens (S3):

- Ingestion caps are now a `Limits` value chosen by the source's trust
  level (ADR-0007). `parseTrace` keeps the browser-scale untrusted caps
  (10 MB / 10k); `parseTraceTrusted` raises them to 256 MB / 2M for
  opened files. The workspace opens files trusted; live capture (S5)
  stays untrusted.
- A 500k-event trace parses, correlates, and stays viewport-bounded in
  the timeline — proven headlessly (a 500k-line JSONL parses to 500k
  events; the virtual window stays viewport-sized).
- Failure detection is capped at 50k events: several rules are O(n^2)
  (they mirror the toolkit's algorithms), fine at browser scale but a
  load-time stall at 500k. Past the cap the trace is fully inspectable
  and the status bar shows "detection skipped (large trace)". The O(n)
  detection rewrite is tracked in #36.

Closes #29
@sepehr-safari sepehr-safari added this to the S3 — Inspector UI milestone Jul 11, 2026
@sepehr-safari sepehr-safari added type:feature New capability area:engine Pure Zig OCPP engine labels Jul 11, 2026
@sepehr-safari
sepehr-safari merged commit daf24d8 into main Jul 11, 2026
3 checks passed
@sepehr-safari
sepehr-safari deleted the feat/trusted-ingestion branch July 11, 2026 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:engine Pure Zig OCPP engine type:feature New capability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(engine): trusted-ingestion parse path + streaming (500k / 100 MB)

1 participant