Zod schema allocation dominates circuit-JSON generation heap (Zod v3 eager-bind + non-memoized get config())

[itsbalamurali]

Summary

Circuit-JSON generation retains a very large number of Zod schema instances and
their eagerly-bound methods, dominating the heap on mid-size boards. Full
evidence, heap snapshot and retainer analysis are in
tscircuit/tscircuit#4077 — this issue tracks the fix, which is in
@tscircuit/core.

Why this repo

@tscircuit/core is where both halves live:

• it depends on zod ^3.25.67 (Zod v3 eager-binds ~15 methods per schema
  instance in the ZodType constructor), and
• it defines every component's get config(), where the base
  PrimitiveComponent returns a freshly-constructed z.object({}) per access
  rather than a memoized/shared schema.

@tscircuit/props has no direct zod dependency and no get config().

The two code-level causes

1. Zod v3 per-instance method binding. Each schema instance carries ~15
   bound closures (.parse, .safeParse, .parseAsync, .refine,
   .superRefine, .optional, .nullable, .array, …). The snapshot shows
   ~714k native_bind nodes retained via .bound_this across ~47k instances.

2. get config() rebuilds schemas. Net/Trace return module-level
   consts (zodProps: netProps, zodProps: traceProps), but the base
   get config() returns a fresh z.object({}), and config is a plain
   getter with no memo cache. The same schema shape is reconstructed thousands
   of times (~9,952 identical ZodOptional instances in one build).

Suggested fix (pure-TS)

1. Migrate to Zod 4 (https://zod.dev/v4). v4 removes the eager per-instance
   method binding, which eliminates the ~714k native_bind closures wholesale,
   and documents large per-schema memory reductions. Biggest, lowest-risk win.

2. Memoize get config() / hoist schemas. Compute each component's
   zodProps once (static per class or module const) instead of returning a new
   z.object({}) from a getter. The base PrimitiveComponent inline
   z.object({}) is the clearest offender.

3. Reuse parsed props. _parsedProps is read widely; verify props are
   validated once and the parsed result reused rather than re-running
   .parse/.optional() per element.

Verification

Measured transpiler-independent (same wall under tsx/SWC/oxc), and reproduced
with --routing-disabled (not the autorouter). See tscircuit/tscircuit#4077.

• @tscircuit/core 0.0.1493, zod 3.25.76, Node 26.

[itsbalamurali]

@DPS0340 may be you can attempt the zod v4 bump too?

[DPS0340]

Yep — I’ll take a separate pass at the Zod v4 bump. I’ll keep #2814 focused on the config-allocation optimization, inventory the v3-only APIs/types in core, then submit the upgrade with typecheck and full-suite verification.

[DPS0340]

Update: the first prerequisite PR is ready and CI-green: https://github.com/tscircuit/circuit-json/pull/661\n\nIt moves circuit-json to a Zod 4 peer dependency and documents the breaking boundary. Because Zod 3 and 4 schemas cannot compose, I’m keeping the rollout ordered: circuit-json release → @tscircuit/props → core. I’m validating the core compatibility slice against that local v4 stack now.

[itsbalamurali]

Local measurements of the two fixes

I patched locally to quantify each fix before proposing a PR.

Fix 1 — remove Zod v3 eager per-instance method binding (i.e. migrate to Zod 4).
Isolated microbenchmark, 30,000 schemas (z.object({ a: z.string().optional(), b: z.number().optional() })), heapUsed after global.gc():

	heapUsed
Zod v3 as-is (24 .bind(this) per ZodType)	291 MB
same, eager binds removed	61 MB

4.8× less, ~230 MB saved for ~30k schemas — and ~30k is the schema-instance
order seen on the reported board. This is what Zod 4 gives for free (it drops
the eager binding).

Fix 2 — memoize get config() / stop rebuilding schemas per access.
Instrumented ZodObject.create to count constructions during one build of a
small board (~40 primitive instances spanning only ~5 component types):

ZodObject.create calls = 583

That's ~14.6 z.object constructions per primitive instance for a board with
only ~5 distinct component types — schemas are rebuilt per access rather than
once per type. Memoizing get config() (or hoisting the schema to a module/
static const) should collapse this toward the number of distinct types, i.e.
~1–2 orders of magnitude fewer schema instances, which then compounds with
Fix 1's per-instance saving.

Net: the two are multiplicative — Fix 2 cuts the count of schemas, Fix 1 cuts
the per-schema cost.

Happy to open a PR. Fix 2 (memoize get config()) is small and self-contained;
Fix 1 (Zod 3→4) is larger due to v4's API changes across the schema definitions.

[DPS0340]

Thanks for quantifying both axes — those measurements make the split very clear.

I am currently carrying the Zod v4 half as a coordinated package migration rather than changing @tscircuit/core in isolation:

• circuit-json #661 is CI-green and moves its schemas to Zod v4.
• I found that @tscircuit/footprinter also composes Circuit JSON schemas but currently brings Zod v3, so it must migrate immediately after circuit-json before Core can consume the new schemas cleanly.
• Then the dependent @tscircuit/props and @tscircuit/core changes can follow on clean installs.

The Core branch already has the v4 API compatibility work and regression coverage; I am validating it against the locally migrated Footprinter now. I will keep the rollout ordered circuit-json → footprinter → props → core so downstream consumers do not get mixed major Zod instances.

The config-memoization change is orthogonal and a good candidate for a small separate PR; I am keeping this branch focused on the Zod v4 dependency/API migration to avoid overlapping Core changes.

[itsbalamurali]

Follow-up: eager-bind removal measured on a real build (not just isolated)

I patched out the 24 .bind(this) in the bundled ZodType constructor and
re-measured, to separate Fix 1 from Fix 2 in a real generation run.

• Isolated (30k schemas): 291 MB → 61 MB (4.8×) — pure schema allocation.
• Real build, ~40-primitive board: 465 MB → 421 MB (~44 MB, ~9.5%).
• Full board (~50 components, ~90 traces): still does not complete with
  eager-bind removed.

So eager-bind removal (i.e. Zod 4) is real but a modest fraction of a live
build's memory, and it does not on its own unblock the larger board. The
dominant lever is Fix 2 — the schema multiplication: ZodObject.create was
called 583 times building a board with only ~5 distinct component types
(~14.6 constructions per primitive instance), because get config() rebuilds
schemas per access rather than once per type.

Recommended order therefore: Fix 2 (memoize get config()) first — it cuts
the number of schema instances by ~1–2 orders of magnitude — then Fix 1 (Zod 4)
to make each remaining instance cheaper. They compound, but Fix 2 is the one
that changes whether generation completes at all.

(Note for anyone reproducing: the CLI bundles its own zod, so patching
node_modules/zod alone has no effect on build — the bundled copy in the CLI
dist is the one on the hot path.)

[DPS0340]

Follow-up: the Footprinter prerequisite is now tracked in draft at tscircuit/footprinter#726 and is explicitly blocked on the Circuit JSON v4 release in tscircuit/circuit-json#661.

I compared representative Footprinter output from the upstream Zod 3 0.0.386 implementation against the Zod 4 branch (DIP, DFN, passive array, USB-C, and jumper-related footprints): the generated Circuit JSON is byte-for-byte identical.

I also built Core against packed local CJSON/Props/Footprinter artifacts, which avoids worktree-link type duplication: Core DTS build and the focused CJSON/Footprinter regression tests pass. The remaining local SVG snapshot differences come from Core currently resolving Footprinter 0.0.380 while the prerequisite branch is based on 0.0.386, which includes an already-upstream DIP fabrication-label correction; they are not a Zod 4 output change.

[DPS0340]

Agreed on the priority split. The base-config allocation fix is now isolated in core PR #2814 and all of its CI checks are green; it removes the unambiguous per-access schema allocation without claiming that Zod 4 alone unblocks the full board.\n\nI am keeping the Zod 4 work as the separate, ordered package migration: circuit-json #661 is CI-green and awaiting review, then Footprinter #726 and Props #752 can validate against its release before the Core compatibility branch lands. That keeps the two measured levers independently reviewable and avoids mixed Zod-major schemas.

[DPS0340]

Update: Fix 2 is now merged in #2814. It shares the base PrimitiveComponent config instead of constructing a Zod schema on every getter access.\n\nI will keep the Zod 4 work as the separate ordered migration (circuit-json → Footprinter → Props → Core) once circuit-json #661 is released, so the two measured levers remain independently reviewable.