feat: add $dynamicRef / $dynamicAnchor schema resolution for OpenAPI 3.1#3889
feat: add $dynamicRef / $dynamicAnchor schema resolution for OpenAPI 3.1#3889aqeelat wants to merge 9 commits into
Conversation
Resolves JSON Schema 2020-12 dynamic reference semantics in the OpenAPI 3.1 parser so generated TypeScript preserves concrete types instead of degrading to unknown. Supported patterns: - Self-referential recursive schemas (BaseCategory.children → BaseCategory[]) - Generic pagination binding via $defs (PaginatedUserResponse.items → User[]) - Sibling scope scan for unique $dynamicAnchor declarations - External $dynamicRef graceful fallback to unknown - Non-identifier schema keys (kebab-case) with $dynamicAnchor Implementation: - Add $dynamicRef, $dynamicAnchor, $defs to spec-type BaseDocument - Add dynamicScope to SchemaState for scope propagation - Add buildDynamicScope() with sibling scan (unique anchors only) - Materialize $ref targets when caller has $defs dynamic bindings - Add $dynamicRef dispatch branch in schemaToIrSchema() - 6 test fixtures + snapshots covering all patterns and edge cases No plugin changes needed — $dynamicRef is resolved to normal $ref in the IR before any plugin sees it.
|
|
|
@aqeelat is attempting to deploy a commit to the Hey API Team on Vercel. A member of the Team first needs to authorize it. |
🦋 Changeset detectedLatest commit: 2248a9b The changes in this PR will be included in the next version bump. This PR includes changesets to release 4 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
Important
The implementation is sound and the test fixtures give good coverage, but the PR description no longer matches what the code does (sibling scope was removed by cb35873), and buildDynamicScope silently drops inline $defs bindings — both worth tightening before merge.
TL;DR — Resolves JSON Schema 2020-12 $dynamicRef / $dynamicAnchor in the OpenAPI 3.1 parser so recursive trees and generic pagination templates produce concrete types instead of unknown. The fix is purely additive — it only triggers when $dynamicRef is present.
Key changes
- Add
dynamicScopetoSchemaState— anchor name → resolved$refmap, propagated through the parser per JSON Schema dynamic-scope rules. buildDynamicScope()— scans the current schema's own$dynamicAnchorand its$defsfor anchor bindings.$dynamicRefdispatch inschemaToIrSchema— resolves the anchor through the active scope to a synthetic$ref, falling back tounknownwhen unresolved.materializeDynamicRefBinding()— when a schema has both$refand a$defsbinding, the referenced template is inlined with the caller's scope so the resulting type isn't a stale alias.$dynamicAnchor-aware inlining insideparseRef— when the referenced component declares a matching$dynamicAnchor, its body is re-parsed under the caller's scope rather than emitted as a name ref.- 6 fixtures + snapshots covering recursive trees, generic pagination (named and inline-route), nested workspace resources, scope isolation, external refs, and non-identifier component keys.
Summary ｜ 25 files ｜ 4 commits ｜ base: main ← feat/dynamicref-support
Inline $defs bindings without $ref are silently dropped
Before: an unsupported, undocumented edge case.
After: still unsupported, but the dropping is silent —buildDynamicScopeonly records bindings whose value is{$dynamicAnchor, $ref}. Inline schemas ({$dynamicAnchor: itemType, type: object, properties: {...}}) are valid JSON Schema 2020-12 bindings but won't enter the scope.
This is a real-world pattern (you don't always factor your bound schema out into components.schemas). Either widen the binding shape to accept inline schemas (materialize them as anonymous IR schemas), or document the limitation alongside the existing inline-route caveat.
packages/shared/src/openApi/3.1.x/parser/schema.ts
PR description has drifted from the implementation
Before: description advertises "Sibling scope — unique anchor" with
BaseFolder.shortcuts → Array<BaseResource>and says the inline paginated-response fixture still producesunknownitems.
After: commitcb35873(fix: avoid sibling dynamic ref bindings) removed sibling-scope resolution, so thenested-workspace-resourcessnapshot now showsshortcuts: Array<unknown>. Meanwhile, thepaginated-responsesnapshot DOES resolve inline-route bindings (the200response is a concrete object withitems: Array<User>), so the "Limitations" bullet about it is stale too.
Worth syncing the PR body and the supported-patterns table with what actually ships — otherwise reviewers and downstream users will be misled about which patterns work.
$dynamicRef anchor parsing is permissive
Before: N/A (new code).
After:schema.$dynamicRef.startsWith('#') ? slice(1) : full string. For a JSON-pointer-form dynamic ref like#/defs/x, the slice yields/defs/xwhich never matches a scope key and falls through tounknown. Functionally fine, but the heuristic conflates plain-name anchors with JSON pointers.
Tightening to "starts with # and contains no /" would make the intent explicit and the failure mode (e.g. logging an unsupported form) easier to diagnose later.
packages/shared/src/openApi/3.1.x/parser/schema.ts
Claude Opus ｜ 𝕏
| if (schema.$defs) { | ||
| for (const [, defSchema] of Object.entries(schema.$defs)) { | ||
| if (defSchema && typeof defSchema === 'object' && !Array.isArray(defSchema)) { | ||
| const defSchemaObj = defSchema as OpenAPIV3_1.SchemaObject; | ||
| if (defSchemaObj.$dynamicAnchor && defSchemaObj.$ref) { | ||
| scope[defSchemaObj.$dynamicAnchor] = defSchemaObj.$ref; | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
$defs bindings are only added to the scope when the inner schema has both $dynamicAnchor AND $ref. Inline-schema bindings — {$dynamicAnchor: itemType, type: object, properties: {...}} — are a valid 2020-12 pattern and currently get silently dropped, leaving the $dynamicRef to fall back to unknown. Either materialize inline bindings as anonymous IR schemas (storing the def path in a parallel map) or call out the limitation explicitly so users know to factor their binding out into components.schemas.
| // Extract the anchor name from the $dynamicRef (e.g., "#itemType" -> "itemType") | ||
| const anchorName = schema.$dynamicRef.startsWith('#') | ||
| ? schema.$dynamicRef.slice(1) | ||
| : schema.$dynamicRef; |
There was a problem hiding this comment.
startsWith('#') accepts JSON-pointer fragments like #/defs/x; slice(1) then yields /defs/x, which silently misses scope lookup and falls back to unknown. Consider tightening to schema.$dynamicRef.startsWith('#') && !schema.$dynamicRef.includes('/') so plain-name anchors and pointers/external refs take visibly different branches.
| !Array.isArray(defSchema) && | ||
| (defSchema as OpenAPIV3_1.SchemaObject).$dynamicAnchor && | ||
| (defSchema as OpenAPIV3_1.SchemaObject).$ref, | ||
| ); |
There was a problem hiding this comment.
hasDynamicRefBindings and buildDynamicScope both walk Object.values(schema.$defs) with the same per-value checks but encode the predicate differently. Factoring this into a single helper (e.g. findDynamicRefBindings(schema) returning the entries once) avoids the two implementations drifting and removes the duplicate inline type cast.
| : { | ||
| circularReferenceTracker: new Set(), | ||
| dynamicScope: buildDynamicScope(schema), | ||
| }; |
There was a problem hiding this comment.
Spreading state here shares circularReferenceTracker (same Set instance) with the parent, which is the intended behaviour, but the asymmetry with dynamicScope (always a fresh object) is worth a comment — otherwise it reads like a shallow-copy mistake and a future refactor could accidentally clone the tracker and silently break circular-ref detection.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3889 +/- ##
==========================================
+ Coverage 37.77% 37.90% +0.12%
==========================================
Files 580 581 +1
Lines 20781 20842 +61
Branches 6031 6070 +39
==========================================
+ Hits 7850 7900 +50
- Misses 10528 10534 +6
- Partials 2403 2408 +5
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
@hey-api/codegen-core
@hey-api/json-schema-ref-parser
@hey-api/nuxt
@hey-api/openapi-ts
@hey-api/shared
@hey-api/spec-types
@hey-api/types
@hey-api/vite-plugin
commit: |
|
@aqeelat still a draft? |
- Extract dynamic ref helpers into separate dynamicRef.ts module - Tighten resolveDynamicRef to reject JSON pointer fragments - Add comment about circularReferenceTracker sharing intent - Document $dynamicRef / $dynamicAnchor support in TypeScript plugin docs - Add changeset for @hey-api/shared and @hey-api/spec-types
|
@mrlubos give me a few minutes. I need to add add more tests. |
|
Not a draft anymore. Latest push adds:
The only remaining failure is the pre-existing flaky timeout in |
dosubot
Bot
added
size:L
There was a problem hiding this comment.
TL;DR — Adds resolution for JSON Schema 2020-12 $dynamicRef / $dynamicAnchor in the OpenAPI 3.1 parser, with the resolution happening in the IR before any plugin runs. Recursive $dynamicRef, generic templates via $defs bindings, and scope isolation between sibling schemas are all covered by integration snapshots, and the new helpers in dynamicRef.ts are unit-tested.
Key changes
- Extend
SchemaStatewithdynamicScope— new field carrying the active anchor-to-$refmap through recursion, with the parentcircularReferenceTrackerSet intentionally shared anddynamicScopealways cloned per call. - New
dynamicRef.tsparser module —buildDynamicScope,buildCurrentDynamicScope,resolveDynamicRef,materializeDynamicRefBinding,shouldInlineDynamicRefTargetcleanly separate the dynamic-scope logic fromschema.ts. schemaToIrSchemadispatch — handles$dynamicRefby rewriting it to a synthetic$ref(when resolvable) or falling back tounknown; materializes{$ref, $defs}template instantiations inline so generic wrappers emit concrete types.parseRefinlining — when a target component carries$dynamicAnchorand the caller's scope binds it elsewhere, the target is parsed inline rather than emitted as a named alias.- Spec-types — adds
$defs,$dynamicAnchor,$dynamicRefto the 2020-12 JSON Schema definitions. - Specs, snapshots, docs, changeset — 6 new spec fixtures + snapshots covering recursive, generic binding, paginated, scope isolation, external ref, and non-identifier keys; user-facing docs under the TypeScript plugin page.
Summary ｜ 29 files ｜ 6 commits ｜ base: main ← feat/dynamicref-support
Resolution model: IR-level rewrite, no plugin changes
Before:
$dynamicRefwas unknown to the parser; downstream plugins sawunknownor stale type aliases.
After: Dynamic scope is computed and consumed entirely in the 3.1 parser, emitting either a synthetic$ref, an inline materialization, orunknown— plugins are unaffected.
The dispatch order in schemaToIrSchema is materializeDynamicRefBinding → $ref → $dynamicRef → other keywords. Materialization fires only when all four guards hold (top-level component target, $ref + $defs, $defs carries an anchor+$ref binding), and unresolved $dynamicRef falls back to parseUnknown rather than leaking the raw keyword.
packages/shared/src/openApi/3.1.x/parser/schema.ts · packages/shared/src/openApi/3.1.x/parser/dynamicRef.ts
Scope shape and shared mutable state
Before:
schemaToIrSchemamutated the caller'sstatedirectly, with$refstashed/restored around recursive calls.
After: Every entry intoschemaToIrSchemaconstructs a freshcurrentState(spread of parent + freshly mergeddynamicScope), but thecircularReferenceTrackerSet is shared by reference so cycle detection still spans the whole tree.
The split — clone dynamicScope per call, share circularReferenceTracker by identity — is the right call: dynamic scope is lexically inherited (children inherit, siblings do not), while cycle detection is structurally global. The existing state.$ref and state.inAllOf mutate/restore patterns in parseRef and parseAllOf remain balanced because the wrapping is shallow and stash-restore happens on the same object that was mutated.
packages/shared/src/openApi/shared/types/schema.ts
Test and snapshot coverage
Before: No
$dynamicReffixtures; recursive schemas degraded toArray<unknown>and generic templates degraded to type aliases.
After: 6 integration fixtures plus a 500-line unit-test file exercise each helper independently and assert the snapshot shape end-to-end.
The scope-isolation snapshot (unboundItem: unknown alongside boundItems: Array<User>) is the most important one — it proves the per-call dynamicScope clone does its job. The recursive-category snapshot confirms BaseCategory.children: Array<BaseCategory> resolves through shouldInlineDynamicRefTarget's self-reference guard. The BaseResource = Document | unknown and BaseFolder.shortcuts: Array<unknown> outputs in the nested-workspace snapshot are the expected "ambiguous anchor → unknown" fallback documented in the plugin page.
packages/shared/src/openApi/3.1.x/parser/__tests__/dynamicRef.test.ts · packages/openapi-ts-tests/main/test/3.1.x.test.ts
Claude Opus ｜ 𝕏
| it('returns undefined for bare #', () => { | ||
| expect( | ||
| resolveDynamicRef({ | ||
| dynamicRef: '#', | ||
| dynamicScope: { '': '#/components/schemas/X' }, | ||
| }), | ||
| ).toBe('#/components/schemas/X'); | ||
| }); |
There was a problem hiding this comment.
The test description says "returns undefined for bare #" but the assertion expects '#/components/schemas/X'. The current implementation does match the empty-string key in scope because '#'.slice(1) === ''. Either tighten resolveDynamicRef to reject dynamicRef === '#' (which the rest of the resolver implies — a bare # has no anchor name to resolve) and assert .toBeUndefined(), or rename the test to reflect the actual behavior.
| it('returns undefined for bare #', () => { | |
| expect( | |
| resolveDynamicRef({ | |
| dynamicRef: '#', | |
| dynamicScope: { '': '#/components/schemas/X' }, | |
| }), | |
| ).toBe('#/components/schemas/X'); | |
| }); | |
| it('resolves bare # against empty-string scope key', () => { | |
| expect( | |
| resolveDynamicRef({ | |
| dynamicRef: '#', | |
| dynamicScope: { '': '#/components/schemas/X' }, | |
| }), | |
| ).toBe('#/components/schemas/X'); | |
| }); |
| // Fallback to preserving the ref if circular | ||
| } | ||
|
|
||
| const refSchema = context.resolveRef<OpenAPIV3_1.SchemaObject>(schema.$ref); |
There was a problem hiding this comment.
context.resolveRef(schema.$ref) is now called unconditionally on every ref before the circular-tracker check, where previously it was only called inside the !circularReferenceTracker.has branch (line 1174). For circular component refs this is a redundant lookup on the hot path. Consider folding the resolution into the existing if (!state.circularReferenceTracker.has(schema.$ref)) block and only calling shouldInlineDynamicRefTarget there — the circular case can't inline anyway.
|
@aqeelat you can ignore the flaky test, that happens only locally as far as I know |
|
@mrlubos quick question on the test types — I'm testing the runtime guard in Currently using
Any preference on the idiomatic approach here? |
|
@aqeelat |
…icRef - Extract getDynamicDefsBindings() helper shared by hasDynamicRefBindings and buildDynamicScope to prevent predicate drift - Reject bare # (empty anchor name) in resolveDynamicRef - Split bare # test into two cases: no scope and empty-string scope key
|
Addressed review feedback in 2248a9b: moved |
2 participants
Closes #3886
What changed
$dynamicRef,$dynamicAnchor,$defsto JSON Schema 2020-12 type definitionsdynamicScopefield for dynamic anchor → type ref propagation through the parserbuildDynamicScope(): Builds dynamic scope from own$dynamicAnchorand$defsbindings$dynamicRefdispatch: New branch inschemaToIrSchema()resolves$dynamicRefthrough dynamic scope to a synthetic$ref$refand$defswith dynamic bindings, the target schema is materialized inline with the caller's scope instead of producing a type alias$dynamicRef/$dynamicAnchorsupport on the TypeScript plugin pagedynamicRef.tshelpers + 6 integration test fixtures with snapshotsSupported patterns
$dynamicAnchor(e.g.BaseCategory.children)Array<unknown>)$defs(e.g.PaginatedUserResponse)$dynamicRefbindingsunknown)Unsupported (falls back to
unknown)$dynamicRef(e.g.other.json#node)@hey-api/json-schema-ref-parser) only processes$ref, not$dynamicRef— external files are never fetched/rewritten. A follow-up issue will be created for this after merge.$dynamicAnchor(multiple same-named anchors)$defsbindings without$ref$defsentries with both$dynamicAnchorand$refare resolvedDesign
$dynamicRefis resolved to normal$refin the IR before any plugin sees it.$dynamicRefis present in the spec. Current behavior for such specs isunknown; any improvement is strictly better.$dynamicRefvalues containing/(JSON pointer fragments) are not resolved.Limitations
components.schemasentry is generated once with its own scope. Full call-site-aware resolution would require generic type parameter synthesis — deferred to a future phase.