Skip to content

tree-sitter: 13 known parse failures in 3 grammar-gap clusters (incorporates / access-modifier-in-cast / multi-dim array) #453

Description

@lagergren

Summary

After PR #452 re-categorized `treeSitterParseSkipList.intentional.txt`, 13 files moved into `parked.txt` as genuine grammar debt: tree-sitter produces ERROR nodes on them, but the Java reference compiler (`xcc`) parses them cleanly (only emits semantic / verify errors, no `PARSER-NN` messages).

The 13 failures cluster into just three distinct grammar gaps:

Cluster A — `incorporates` clause not parsed (7 files)

`incorporates` mixin clauses fail to parse in several common shapes. Note the variety:

File Line Snippet
`archive/AddressBookDB.x` 11 `incorporates Database`
`archive/addressDB_imdb/module.x` 2 `incorporates imdb_.CatalogMetadata<AddressBookSchema_>`
`archive/addressDB_json/module.x` 2 `incorporates imdb_.CatalogMetadata<AddressBookSchema_>`
`archive/sock-shop/socks-cart.x` 5 `incorporates Database`
`dbTests/PeopleTest.x` 9 `incorporates TerminalApp.Mixin("People DB Test") {`
`jsondb/jsondb_test.x` 13 `incorporates test_db.TestCatalogMetadata {`
`webTests/oauth.x` 7 `incorporates WebApp {`

Three sub-shapes here: bare type name, qualified type with generic args, and the "with arg list"/"with body block" forms.

Cluster B — access modifier inside parenthesized cast / type expression (5 files)

`(protected Type<...>)` (and possibly other access modifiers) inside parens — used as the type argument to `.as(...)` or `.revealAs((...))`. Fails with ERROR nodes around the `protected` keyword position.

File Line Snippet
`jsondb/.../JsonMapStoreConfigTest.x` 24 `schema.getMapStore().as(protected JsonMapStore<String, String>);`
`jsondb/.../JsonMapStoreTest.x` 33 `...as(protected JsonMapStore<String, String>);`
`jsondb/.../test_db/TestClient.x` 51 `&mapData.revealAs((protected DBMapImpl<String, String>));`
`jsondb/.../tx_manager/TxManagerConfigTest.x` 23 `.revealAs((protected TxManager));`
`jsondb/.../tx_manager/TxManagerLogTest.x` 62 `&manager.revealAs((protected TxManager));`

Cluster C — multi-dim array constructor with lambda init (1 file)

File Line Snippet
`manualTests/src/main/x/errors.x` 22 `new Int[7, (i) -> -1];`

(The reference compiler emits `COMPILER-NI: "Multi-dimensional array" is not yet implemented` for this — semantic, not parser. Tree-sitter, however, ERRORs at the `,`.)

How to reproduce

From the project root, on the `master` branch (or any branch that contains PR #452):

```bash

1. Generate parser + scanner artifacts and build the dylib once.

./gradlew :lang:dsl:generateScannerC :lang:dsl:generateTreeSitter \
:lang:tree-sitter:validateTreeSitterGrammar \
-PincludeBuildLang=true -PincludeBuildAttachLang=true
rm -f ~/.cache/tree-sitter/lib/xtc.dylib

2. tree-sitter-cli must be invoked from the generated directory so it

finds the grammar config; otherwise it silently fails to load the

parser and reports zero ERROR nodes (false PASS).

cd lang/tree-sitter/build/generated

3. Parse one of the parked files and grep for ERROR nodes.

/Users/marcus/src/xtclang2/lang/tree-sitter/build/tree-sitter-cli/tree-sitter parse \
/Users/marcus/src/xtclang2/manualTests/src/main/x/errors.x \
| grep -oE 'ERROR \[[0-9]+, [0-9]+\]' | head
```

To verify the Java parser handles the same file cleanly (only semantic errors, no `PARSER-NN`):

```bash
xdk/build/install/xdk/bin/xcc manualTests/src/main/x/errors.x 2>&1 | grep PARSER-

(should print nothing)

```

To re-run the full sweep:

```bash
./gradlew :lang:tree-sitter:testTreeSitterParse \
-PincludeBuildLang=true -PincludeBuildAttachLang=true

Should report "880 passed, 0 failed" with the 13 entries skip-listed.

Removing entries from parked.txt is how you opt them back into the sweep.

```

Suggested triage order

  1. Cluster B (access modifier in cast) — likely the smallest grammar fix; one shape repeated 5×, all in one library.
  2. Cluster A (incorporates) — three sub-shapes, but `incorporates` is a single grammar rule and the fixes probably cluster.
  3. Cluster C (multi-dim array w/ lambda) — one file, and the language semantics for it aren't even implemented (`COMPILER-NI`); lowest priority.

Acceptance criterion

`treeSitterParseSkipList.parked.txt` ends up empty (or only contains entries for which we have an explicit "won't fix in tree-sitter" decision), and `./gradlew :lang:tree-sitter:testTreeSitterParse` reports `893 passed, 0 failed`.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions