[*as-Code] Reduce Zod schema startup memory pressure

[nickofthyme]

Context

In #268329 we migrated all *-as-Code schemas from @kbn/config-schema to @kbn/zod. Following the docs here, a heap profiling run (3 rounds per side, idle snapshot at startup) measured the memory impact of that change before merging, so the numbers here reflect the gap to close in this follow-up.

Memory Profile Results

Methodology: 3 profiling rounds per side (baseline = main, candidate = zod-ftw), fresh Elasticsearch data each round. Heap snapshots captured after Kibana fully initialised, before any requests. Round 2 snapshots used for heap attribution (consistent with Kibana heap profiling docs).

RSS / Heap — 3-round average

Metric	Baseline (main)	Candidate (zod-ftw)	Delta
Peak sampled RSS	4,796 MB	4,861 MB	+65 MB (+1.4%)
Peak heapUsed	949 MB	943 MB	−6 MB
Avg heapUsed	632 MB	674 MB	+42 MB (+6.6%)
Peak external+ArrayBuffers	1,499 MB	1,544 MB	+45 MB (+3.0%)
Total retained heap	733.8 MB	755.8 MB	+22 MB (+3%)

RSS variance across rounds was high (macOS memory accounting); heap numbers are more reliable.

Zod retained by package (idle, round 2)

Package	Baseline	Candidate	Delta
zod	91.3 MB	109.7 MB	+18.4 MB (+20%)
@kbn/zod	6.4 MB	6.7 MB	+0.4 MB
@kbn/config-schema	10.5 MB	9.9 MB	−0.6 MB

Zod allocation attribution — top new allocators (startup, no requests)

The --filter=zod pass attributes heap bytes back to the @kbn/* package whose module-level constants triggered the allocation.

Package	Baseline	Candidate	Delta
@kbn/lens-embeddable-utils	0 MB	9.94 MB	+9.94 MB
@kbn/maps-plugin	0 MB	3.61 MB	+3.61 MB
@kbn/discover-plugin	0.14 MB	1.37 MB	+1.23 MB
@kbn/dashboard-plugin	0 MB	1.01 MB	+1.01 MB
@kbn/controls-schemas	0 MB	0.83 MB	+0.83 MB
@kbn/as-code-shared-schemas	0 MB	0.59 MB	+0.59 MB
@kbn/as-code-filters-schema	0 MB	0.59 MB	+0.59 MB
@kbn/as-code-data-views-schema	0 MB	0.53 MB	+0.53 MB
@kbn/links-plugin	0 MB	0.48 MB	+0.48 MB
@kbn/aiops-server-schemas	0 MB	0.45 MB	+0.45 MB
@kbn/ml-server-schemas	0 MB	0.44 MB	+0.44 MB
@kbn/lens-plugin	0 MB	0.44 MB	+0.44 MB
@kbn/slo-plugin	0 MB	0.35 MB	+0.35 MB
@kbn/es-query-server	0 MB	0.33 MB	+0.33 MB
(others < 0.3 MB each)			~+1.8 MB
Total new Zod allocs	95.95 MB	116.98 MB	+21.03 MB

Root Cause

All new allocations are module-level const definitions — Zod schema objects constructed eagerly when the module is first require()d. Because Kibana's Node.js process loads these modules during plugin.setup(), every Zod schema object is alive from startup regardless of whether the corresponding route or feature is ever used.

The pattern looks like:

// Eagerly constructed at import time — Zod objects live in heap from startup
export const mySchema = z.object({ ... });

This is the same root cause that was fixed for APM in PR #279910.

Approaches / Tasks (priority order)

1. Lazy-load route validate schemas via thunks (quick win for route-registered schemas)

What: Instead of a top-level const, pass a () => schema thunk to the route validate option. The core non-versioned router (prepareRouteConfigValidation) already wraps these with lodash.once, so the schema is built once on first request and cached.

How (APM pattern, directly applicable to routes in the affected plugins):

// Before (eager — schema built at module load time):
router.post({ path: '/api/...', validate: { body: myBodySchema } }, handler);

// After (lazy — schema built on first request, cached by once()):
router.post({ path: '/api/...', validate: () => ({ body: myBodySchema }) }, handler);

Caveat for versioned routes: core_versioned_route.ts calls validate() per request with no caching. Until that is fixed (see task 3), thunks on versioned routes provide no benefit and should not be used as a memory optimisation there.

Affected routes to audit (the biggest allocators):

• @kbn/lens-embeddable-utils — 9.94 MB new Zod allocs
• @kbn/maps-plugin — 3.61 MB
• @kbn/discover-plugin — 1.23 MB
• @kbn/dashboard-plugin — 1.01 MB

2. Defer module-level schema constants to factory functions

What: Schema packages that export top-level constants (e.g. @kbn/controls-schemas, @kbn/as-code-filters-schema, @kbn/as-code-data-views-schema) cannot be deferred by route-level thunks because the schemas are created when the module is first imported — before any route is called.

How: Move the schema construction inside a function (optionally memoised with lodash.once):

// Before:
export const controlsGroupSchema = z.object({ ... });

// After:
import { once } from 'lodash';
export const getControlsGroupSchema = once(() => z.object({ ... }));

All call sites update from controlsGroupSchema to getControlsGroupSchema(). This is the same pattern used in getDashboardStateSchema in dashboard_state_schemas.ts.

Packages to update (by allocation size):

• @kbn/controls-schemas (+0.83 MB)
• @kbn/as-code-shared-schemas (+0.59 MB)
• @kbn/as-code-filters-schema (+0.59 MB)
• @kbn/as-code-data-views-schema (+0.53 MB)
• @kbn/dashboard-navigation-options-schema (+0.05 MB)

3. Fix versioned router to cache validate thunks

What: extractValidationSchemaFromHandler in core_versioned_route.ts calls validate() on every request with no caching:

// src/core/packages/http/router-server-internal/src/versioned_router/core_versioned_route.ts
if (typeof handler.options.validate === 'function') return handler.options.validate(); // called per-request!

The non-versioned router correctly wraps thunks with once() via prepareRouteConfigValidation. The versioned router should do the same.

Fix: Memoize the thunk result per handler registration, analogous to how the non-versioned router handles it.

4. Fix prepareRouteConfigValidation to short-circuit on function validate (correctness bug)

What: util.ts calls getRequestValidation(validate) before checking whether validate is a function, which triggers the thunk at route-registration time rather than deferring it:

// src/core/packages/http/router-server-internal/src/util.ts (line ~141)
// Bug: calls getRequestValidation(validate) even when validate is a function
const shouldValidateBody = (validate && !!getRequestValidation(validate).body) || !!options.body;

// Fix (from PR #279910):
const shouldValidateBody =
  (validate && (typeof validate === 'function' || !!getRequestValidation(validate).body)) ||
  !!options.body;

This fix is already landed in the APM PR and should be verified as present on main before this follow-up work begins.

5. Lazy-load heavy schema packages via dynamic import() at the plugin level

What: For plugins where the schema package is only needed on the server-side and constitutes a significant share of startup cost, the entire package import can be deferred to first use via await import('@kbn/controls-schemas').

Trade-off: More invasive refactor; async call sites; harder to type. Worth considering only for the two biggest contributors (@kbn/lens-embeddable-utils, @kbn/maps-plugin) if tasks 1–3 don't bring their numbers down sufficiently.

Long-term impact note

The +21 MB is the cold-start overhead — schemas alive from startup with zero requests made. In a long-running stateful deployment where all routes are eventually hit, the per-request schema construction from thunks would eventually converge to the same live set. The real win from lazy-loading is in:

• Serverless short-lived instances that are torn down before all routes are exercised
• Disabled plugins whose schemas are loaded transitively but whose routes are never hit

The module-level constant fix (task 2) has permanent impact regardless of deployment model.

References

• Addressing PR: [* as Code] Migrate to @kbn/zod #268329
• APM lazy Zod loading fix: [APM] lazy loading ZOD schemas #279910
• Kibana heap profiling docs: https://docs.elastic.dev/kibana-dev-docs/key-concepts/performance/heap-profiling

[kibanamachine]

🤖 Automated triage

This issue has been automatically assigned the Team:Core team label.

Reason

The issue describes a problem with increased startup memory pressure due to eager loading of Zod schemas, which are core configuration and validation components. The proposed solutions involve modifying core platform mechanisms such as HTTP routing (core_versioned_route.ts) and schema loading patterns (@kbn/config-schema, @kbn/zod). The Core team owns the Kibana Core runtime, platform services, HTTP routing, and configuration/validation libraries, making them the appropriate team for this platform-level performance optimization.

---

_{This team label was applied automatically by the Kibana automated triage system. If incorrect, remove it and apply the correct team label. To improve future triaging accuracy, consider updating the team description file. Feedback and questions → #kibana-automated-triage Slack channel.}

[infra-vault-gh-plugin-prod]

Pinging @elastic/kibana-visualizations (Team:Visualizations)