Feature request: expose V8 code cache (ScriptCompiler::CachedData) on Context#eval

[ursm]

Motivation

For applications that re-execute the same multi-MB JS bundle on every request
(e.g. SSR / per-visit isolation patterns), Script::Compile parsing dominates
runtime — large bundles add hundreds of ms of cumulative parse cost per visit
that has nothing to do with the script's actual work.

V8 has a built-in remedy in v8::ScriptCompiler::CachedData: compile once,
serialize the bytecode, and feed it back on subsequent compiles to skip parsing.
mini_racer currently always goes through the plain v8::Script::Compile path
(mini_racer_v8.cc:585), so this win isn't reachable from Ruby.

This mirrors the architecture of MiniRacer::Snapshot#dump / Snapshot.load
(#395) — opaque bytes serialized by V8, persisted by the caller, fed back on
the next boot — just at the per-script granularity instead of per-isolate.

Proposed API

Three candidate shapes; happy to take direction on which the maintainers prefer.

(A) kwarg on eval, blob returned alongside result

result, fresh_cache = ctx.eval(src, filename: 'bundle.js', cached_data: prev_cache)

Symmetric with Snapshot#dump / .load. Backward-incompatible return shape
unless gated on the kwarg being present.

(B) kwarg on eval, blob fetched separately

result = ctx.eval(src, filename: 'bundle.js', cached_data: prev_cache)
fresh_cache = ctx.last_compiled_cache  # nil if cache was consumed

Fully backward-compatible signature.

(C) explicit compile step (closer to V8 native shape)

script = ctx.compile(src, filename: 'bundle.js', cached_data: prev_cache)
script.cached_data  # bytes for persistence
script.run          # or implicit run via eval(script)

More surface area, but composable; lets callers warm the cache without running.

Implementation sketch

• Switch v8_eval to v8::ScriptCompiler::Compile with kConsumeCodeCache when
  a blob is supplied; check source.GetCachedData()->rejected and silently
  fall back to a fresh compile on rejection.
• Without a blob, compile normally and call
  ScriptCompiler::CreateCodeCache(unbound_script) to produce the bytes.
• Extend the request / response payloads on the bridge (third optional
  element either way) — same pattern as the snapshot serialization.
• TruffleRuby shim: ignore cached_data, return nil for the cache blob.

Total change is ~150 LoC + specs + CHANGELOG.

Happy to send a PR once there's directional agreement on the API shape.

[nightpool]

I think ctx.compile seems like the best API shapes to start with here. But I'm also interested to hear more about your use-case. For SSR, most applications traditionally keep around the same Context and reuse it to execute the render function multiple times. This is how e.g. SSR using Node.js works. Even with script caching, reevaluating the same multi MB script for each request is always going to be much slower than a single function call.

Is there something specific to your problem domain that means that keeping the same context around between requests isn't an option for you?

That said, even if it doesn't seem like a good fit for your specific use case, this feature seems like the modern replacement for the snapshot API, so it'd be great to be able to persist this to disk and back to support the use cases that. Whatever approach we take, we should make sure to track/serialize CachedDataVersionTag() as well, so that consumers can serialize this data. But clearer insight into the potential use cases is going to help us make a better decision on the final API shape

Finally, why does your implementations plan exclude Truffle Ruby? Is there a blocker to caching compilation results on that platform?

[tisba]

+1 for an explicit API, like the proposed one in c)

[ursm]

Thanks for the thoughtful reply — and to @tisba for the +1.

Use case

The driver here isn't classic SSR — it's per-visit browser simulation
(capybara-simulated, a Capybara driver that runs the app's actual JS
bundle in a JS runtime to simulate real navigations during tests). Each
"visit" models a fresh page load: its own cookies, localStorage, fetch state,
DOM. That isolation is the product, not an optimization choice, so reusing a
single Context across visits collapses exactly the boundary we need.

A test suite can easily do hundreds of independent visits per run, each
re-parsing the same multi-MB app bundle. Concrete numbers from the driver
warming an Avo app's avo.base.js (~6.8 MB) inside a fresh Context (built
from a shared MiniRacer::Snapshot):

metric	value
per-visit ctx.eval(bundle) (median of 30)	~206 ms
cumulative parse overhead at 100 visits	~21 s

That ~21 s is pure parse work — nothing the app did. Code cache is the lever
that makes per-visit isolation affordable. Snapshots help for the
framework-level boilerplate but break down for the app bundle:
SnapshotCreator::CreateBlob() disposes the isolate, and external_references
registration for host-side callbacks (the simulated network layer) doesn't
currently round-trip through mini_racer's attach. Code caching sidesteps
both issues.

(C) it is

Given the consensus, going with the explicit shape:

script = ctx.compile(src, filename: 'bundle.js', cached_data: prev_blob)
prev_blob_for_next_run = script.cached_data  # nil if the supplied cache was consumed and not regenerated
result = script.run                            # safe to call multiple times

One implication worth flagging: separating compile and run means the
bridge needs a handle table for v8::Persistent<v8::Script> (compile RPC
returns a handle + cached_data; run RPC takes the handle; dispose RPC frees
it). That pushes the change closer to ~250 LoC, not the ~150 I estimated for
the kwarg-on-eval shape — still tractable, but worth being upfront about.

If maintainers would rather avoid the handle table and ship a tighter variant
first (result, fresh_cache = ctx.compile_and_run(src, cached_data:), with
explicit compile/run separation added later), happy to take that direction
instead. Otherwise full (C) is the plan.

CachedDataVersionTag() — agreed

Rejection-on-load is only knowable after deserialization, which defeats the
point when sharding many bundles. Proposal: expose as a module-level constant
so callers can mix it into their cache key cheaply.

MiniRacer::V8_CACHED_DATA_VERSION_TAG  # => Integer (uint32 from v8::ScriptCompiler::CachedDataVersionTag())

Kept on the caller side rather than embedded in the blob — keeps the gem out
of long-term blob format invariants.

TruffleRuby

Looked into this more carefully before answering. GraalJS has no Source-grained
persistent cache reachable from Polyglot::InnerContext#eval:

• Source.Builder.cached(true) exists but is in-process / Engine-lifetime
  only and isn't reachable from InnerContext#eval (which takes a String,
  not a Source).
• The only cross-process persistence mechanism is Auxiliary Engine
  Caching, which requires Oracle GraalVM Enterprise + Native Image.
  mini_racer explicitly rejects the Native standalone (truffleruby.rb:81-83),
  so this path is closed by construction.
• Architecturally GraalJS doesn't have a process-stable JS bytecode tier — the
  Truffle AST is specialized at runtime and JIT-compiled to image-bound
  machine code, which is why the only persist story is Engine-grained SVM
  heap snapshotting.

So this would be a no-op shim on TruffleRuby, consistent with how
Snapshot#load and Snapshot#warmup_unsafe! are already implemented there
(truffleruby.rb:378-390): cached_data: silently ignored, script.cached_data
returns nil. I don't think it warrants a TruffleRuby-side implementation
effort because there isn't one to be had — but documenting the gap in the
gem's docs is probably enough.

Happy to start on the PR whenever there's a green light on the
compile/run split (vs the tighter compile_and_run variant).