Skip to content

Grammar hygiene: remove two redundant parse paths (ambiguity reduction)#7

Merged
christiankissig merged 2 commits into
masterfrom
method-embedded-redesign
Jun 2, 2026
Merged

Grammar hygiene: remove two redundant parse paths (ambiguity reduction)#7
christiankissig merged 2 commits into
masterfrom
method-embedded-redesign

Conversation

@christiankissig

Copy link
Copy Markdown
Owner

Summary

Removes two genuine duplicate parse paths in the grammar, found while investigating the Earley timeout bucket. Both are safe structural cleanups that shrink the parse forest with zero regressions.

Change Why it was redundant
Drop context as a direct theory alternative context is already reachable via theory → statement → context, so every theory had ≥2 parses at the root
Drop the terminal by in proof_prove by already appears as a repeated proof step, which covers the terminal position; the duplicate gave every by-terminated proof two parses

Effect on ambiguous-node counts (proof snippets): a nested proof drops from 6682 → 2707 (~60%); by simp from 10 → 3.

Honest scope: this is hygiene, not a speed fix

Profiling shows parse wall-clock is dominated by Earley chart breadth (millions of Item.__init__ / symbol __eq__ in predict_and_complete's set-dedup), which scales with grammar breadth × input length — not with the output ambiguity these changes remove. Accordingly, wall-clock is essentially unchanged (e.g. an 8 KB proof-heavy file stays ~11.6 s). These changes are worth keeping because they remove real redundancy and reduce the SPPF/memory, but they do not address the timeout bucket.

The actual speed levers (documented for follow-up): reduce per-proof-step prediction breadth (proof_prove/proof_state/method alternations), the keyword-vs-name and embedded terminal overlaps the dynamic lexer can't prune, or a different parse strategy. The opt-in --timeout from #5 remains the pragmatic guard meanwhile.

Testing

  • Full unit suite: 135 passed, no regressions.
  • Regression-checked against a real-file sample with a strict ok→fail criterion (timeout flakiness excluded): 0 real regressions; 2 sampled timeouts recovered.

🤖 Generated with Claude Code

christiankissig and others added 2 commits June 2, 2026 17:49
`context` was reachable both as a direct alternative of the top-level
`theory` rule and via `theory -> statement -> context`, so every theory
had (at least) two parses at the root, multiplying the Earley chart for
the whole file. `statement` already provides `context`, so the direct
alternative is removed.

Measured on proof snippets: ambiguous-node counts drop ~33% (e.g. a
nested proof 6682 -> 4455), with 0 regressions and 2 of 6 sampled
timeouts recovered. Full unit suite still green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`by` appeared both as a repeated proof step and as a terminal proof step
in `proof_prove`, so any proof ending in `by ...` had two parses,
compounding through nested proofs. The repeated-step `by` already covers
the terminal position, so the terminal copy is removed.

Measured on proof snippets (on top of the previous fix): ambiguous-node
counts drop a further ~40% (nested proof 4455 -> 2707; by simp 7 -> 3),
with 0 real regressions (ok->fail) on the sampled corpus. Full unit suite
still green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@christiankissig christiankissig merged commit 860c472 into master Jun 2, 2026
6 checks passed
@christiankissig christiankissig deleted the method-embedded-redesign branch June 2, 2026 17:27
christiankissig added a commit that referenced this pull request Jul 7, 2026
Grammar hygiene: remove two redundant parse paths (ambiguity reduction)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant