Run createTestIndexer in-process instead of worker threads - #1467
Merged
Conversation
Replace the per-chain worker + storage message-proxy with an in-process run against an in-memory Persistence.storage, removing the dominant cost (re-evaluating the envio module graph in a fresh isolate per process()). - IndexerState gets an injectable ~onExit; ExitOnCaughtUp resolves it instead of process.exit, so a caught-up run in-process resolves a promise rather than killing the test runner. Production default unchanged. - TestIndexer builds a per-instance in-memory storage (config-derived initial state, never a real DB) and drives IndexerState/IndexerLoop directly; runs bypass Persistence.init and stop the loop on completion. - Registrations are captured once and cloned per run (patchConfig appends a simulate source), so independent createTestIndexer instances run in parallel without shared mutable registration state. - Handlers run inside an AsyncLocalStorage scope so a handler calling indexer.onEvent throws (as in production) without finishing the global registration the test itself uses. - Delete TestIndexerWorker and the proxy's message-channel machinery. - Restore vitest to envio devDependencies (needed to resolve the Vitest binding for local scenario test runs). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fb6Yyif9GZG1DWAiCER4Gh
The worker model serialized entities to JSON to cross the thread boundary. In-process there is no boundary, so store and load entities decoded: - handleLoad filters decoded entities with the already-typed filter and returns them directly (no serialize/parse, no rowsSchema round-trip). - handleWriteBatch takes Persistence.updatedEntity and stores the decoded entities as-is instead of encoding then re-parsing them. - Delete TestIndexerProxyStorage entirely — its serializable types were the only remaining use. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fb6Yyif9GZG1DWAiCER4Gh
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
🚧 Files skipped from review as they are similar to previous changes (3)
📝 WalkthroughWalkthroughThe test indexer now executes in-process with decoded in-memory entities, injectable caught-up exit behavior, per-run persistence, and asynchronous cleanup. Worker-proxy storage is removed, entity boundary immutability and handler errors are tested, and handler type checks and isolation documentation are updated. ChangesTest indexer runtime
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant TestIndexer
participant IndexerLoop
participant InMemoryStorage
participant Handler
TestIndexer->>IndexerLoop: start in-process chain processing
IndexerLoop->>InMemoryStorage: load decoded entities
InMemoryStorage-->>IndexerLoop: return copied entities
IndexerLoop->>Handler: execute event handler
Handler->>InMemoryStorage: write entity updates
InMemoryStorage-->>IndexerLoop: persist decoded updates
IndexerLoop-->>TestIndexer: resolve or propagate processing result
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/envio/src/TestIndexer.res`:
- Line 141: Update the `entityChangeFor(checkpointId)` call to add an explicit
`Utils.magic` function type annotation, matching the annotated casts elsewhere
in `TestIndexer.res`; specify the actual input and output entity types required
by `Array.push`.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: aa123ce4-fc73-40e6-a7b7-c6943dac05b7
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (10)
packages/envio/package.jsonpackages/envio/src/Api.respackages/envio/src/ExitOnCaughtUp.respackages/envio/src/HandlerRegister.respackages/envio/src/HandlerRegister.resipackages/envio/src/IndexerState.respackages/envio/src/IndexerState.resipackages/envio/src/TestIndexer.respackages/envio/src/TestIndexerProxyStorage.respackages/envio/src/TestIndexerWorker.res
💤 Files with no reviewable changes (2)
- packages/envio/src/TestIndexerProxyStorage.res
- packages/envio/src/TestIndexerWorker.res
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1e8aae6325
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| resolve() | ||
| // Each run gets its own copy of the shared base registration so the | ||
| // simulate-source registration it appends stays isolated. | ||
| let registrationsByChainId = cloneRegistrations(await getRegistrations(~config)) |
There was a problem hiding this comment.
When process() imports handlers through getRegistrations here, the in-process runner has not published the per-run persistence or indexer state to Main (and never calls setGlobalPersistence/setIndexerState). The removed worker path went through Main.start, which initialized persistence before HandlerLoader.registerAllHandlers, so handler modules that read the exported indexer.chains at module top level or during a handler saw the test run's startBlock/endBlock/dynamic addresses. With this ordering they instead see only config/static or stale global state, so tests that override the range in process({ chains: ... }) or rely on dynamic addresses via the public indexer API can silently run against the wrong chain metadata.
Useful? React with 👍 / 👎.
| let registrationsRef: ref<option<promise<HandlerRegister.registrationsByChainId>>> = ref(None) | ||
| let getRegistrations = (~config) => | ||
| switch registrationsRef.contents { | ||
| | Some(promise) => promise |
There was a problem hiding this comment.
When a later process() call or a separate createTestIndexer() instance hits this cached promise, it reuses the exact handler and contractRegister closures captured during the first registration. The old worker path imported the handler module in a fresh worker for each run, so module-scoped mutable state in handlers was reset; now any top-level counters, captured callbacks, or caches mutated by one test run leak into the next even though the indexer storage/state is new. This can make repeated or parallel test indexers skip branches or observe stale captured values, so the per-run registration clone needs to avoid reusing mutable handler closures or otherwise reset handler module state.
Useful? React with 👍 / 👎.
|
|
||
| // Silence logs by default in test mode unless LOG_LEVEL is explicitly set. | ||
| switch Env.userLogLevel { | ||
| | None => Logging.setLogLevel(#silent) |
There was a problem hiding this comment.
When createTestIndexer() is constructed without LOG_LEVEL, this now calls Logging.setLogLevel(#silent) in the main test process. The removed worker path only changed the logger inside the worker isolate, so parent-process logging and later tests/indexers were unaffected; with the in-process runner, a single test indexer can silently disable all subsequent Envio logs in the same Vitest process, including logging-focused tests or a real indexer started after the helper. Scope this to the test run or restore the previous log level after the helper finishes.
Useful? React with 👍 / 👎.
* Harden in-process TestIndexer: isolate store, drop dead resume plumbing - Copy entities on the set/get boundary so user code mutating a returned entity (or an object passed to `set`) can't corrupt the in-memory store; add a regression test. - Remove the unused `resumeInitialState`/`currentInitialState` injection — the runner sets `storageStatus = Ready(...)` directly, so it was never reached; make `resumeInitialState` a throwing stub like the other unused storage methods. - Await the in-flight write fiber in cleanup instead of busy-polling it. - Document the loss of per-test isolation (shared module/effect-cache state) in the indexer-testing skill. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DAwNk4gUT3QV6dh7uBtef1 * Replace AsyncLocalStorage guard with plain finished-registration flag The in-process runner used an AsyncLocalStorage scope so a handler calling `indexer.onEvent` at runtime would throw, while still letting tests call the registration API after a run (which required clearing the finished flag). The only thing depending on the API staying callable post-registration was two type-surface tests; those are compile-time only and don't need to run after registration. - Move the `indexer.onEvent`/`contractRegister` type-surface checks into EventHandlers.ts under `if (0)` (type-checked, never executed), and drop the two runtime `it(...)` cases. - Delete the AsyncLocalStorage scope, `runInHandlerScope`/`isInHandlerScope`, and `clearActiveRegistration`; restore `throwIfFinishedRegistration` to the plain `finished` check. A handler registering at runtime now throws via the finished flag, as in production. - getRegistrations no longer reopens the registration API after capturing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DAwNk4gUT3QV6dh7uBtef1 * Tighten test-isolation note in indexer-testing skill Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DAwNk4gUT3QV6dh7uBtef1 --------- Co-authored-by: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f08c14c083
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
84
to
+86
| entityDict | ||
| ->Dict.valuesToArray | ||
| ->Array.forEach(entity => { | ||
| // Cast entity to dict of field values (same approach as InMemoryTable) | ||
| ->Array.filter(entity => { |
There was a problem hiding this comment.
When a handler loads an entity that was already persisted in the test indexer state (for example await context.User.get(id) in a second process() call, or getWhere over a seeded entity), this returns the exact object held in state.entities. The old worker path serialized and parsed the row across the worker boundary, so handler code could not mutate the backing test store unless it called set; now any in-place mutation of the loaded object leaks into later reads/runs even if the handler never saves it. Return copies from handleLoad before they are inserted into the per-run in-memory table.
Useful? React with 👍 / 👎.
Comment on lines
+834
to
+837
| while ( | ||
| indexerState->IndexerState.isProcessing || | ||
| indexerState->IndexerState.writeFiber->Option.isSome | ||
| ) { |
There was a problem hiding this comment.
In cases where process() resolves or rejects while a fetch tick is still parked in SourceManager.dispatch (for example a handler failure after scheduleFetch() has started the next wait-for-new-block poll), this cleanup only waits for processing and writes. The old worker was terminated, but now the in-process poll/query can keep timers or subscriptions alive after the test run and later call back into the stopped state; track/cancel the fetch fibers as part of cleanup so a finished test indexer leaves no background work behind.
Useful? React with 👍 / 👎.
The (~config) => unit => t currying was a dependency-injection seam (it once also threaded ~workerPath); nothing injects a custom config and its only caller invoked it immediately. Fold it into a single createTestIndexer() that loads the memoized config internally. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fb6Yyif9GZG1DWAiCER4Gh
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: da2c77cfa7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| // The store keeps decoded entities so load comparisons (bigint / | ||
| // BigDecimal) work on real values. | ||
| entityDict->Dict.set(entityId, entity) |
There was a problem hiding this comment.
When a handler passes a reusable object to context.Entity.set and then mutates it in a later event or process() run, this stores the same object reference in the test indexer's persisted state. The worker implementation serialized the entity before the parent stored it, so the parent store owned an independent copy; now a mutation after the write can change later indexer.Entity.get() results without another set. Copy the entity before saving it into state.entities.
Useful? React with 👍 / 👎.
* Fix stale TestIndexer comments and drop dead chainId tuple element - copyEntity comment no longer claims all entity fields are immutable scalars; shallow copy still shares array-valued fields. - Rollback storage stubs no longer advise setting rollbackOnReorg to false — the runner already forces it off, making them unreachable. - Drop the unused chainId element from chainEntries (validation kept). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NCpLSmKU3MZaAyGRjmcAVc * Add regression test for handler-throw propagation A synchronous throw in an event handler body must reject the process() promise at the call site, carrying the handler's original message. This path (handler-body throw) was only covered indirectly before, via effect and getOrThrow throws. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NCpLSmKU3MZaAyGRjmcAVc --------- Co-authored-by: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3669f7f4b3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+375
to
+379
| let copyEntity = (entity: Internal.entity): Internal.entity => | ||
| entity | ||
| ->(Utils.magic: Internal.entity => dict<unknown>) | ||
| ->Utils.Dict.shallowCopy | ||
| ->(Utils.magic: dict<unknown> => Internal.entity) |
There was a problem hiding this comment.
For entities with Timestamp fields, this shallow copy still shares the same Date instance between the caller and state.entities. If a test seeds or reads an entity and then mutates that date in place (for example entity.timestamp.setTime(...)) after set, get, getOrThrow, or getAll, the backing test store changes without another set, so later handlers/assertions can observe corrupted state despite the new copy boundary. Use a schema-aware clone or otherwise clone supported mutable field values such as Date.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a2c8d19ff9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| // The store keeps decoded entities so load comparisons (bigint / | ||
| // BigDecimal) work on real values. | ||
| entityDict->Dict.set(entityId, entity) |
There was a problem hiding this comment.
When a JS handler, or TS handler using an escape hatch like as any, calls context.Entity.set with a value that does not satisfy the generated entity schema (for example a bad enum value or bigint string), this path now stores and reports the raw object without the schema round-trip the worker storage used to perform, while production persistence still validates through entityConfig.schema before writing. That lets createTestIndexer tests pass with data that the real indexer rejects at write time, so handler-written entities should be schema-converted/validated before accepting them into the in-memory store.
Useful? React with 👍 / 👎.
DZakh
added a commit
that referenced
this pull request
Jul 23, 2026
* Run createTestIndexer in-process instead of worker threads (#1467) * Run createTestIndexer in-process instead of worker threads Replace the per-chain worker + storage message-proxy with an in-process run against an in-memory Persistence.storage, removing the dominant cost (re-evaluating the envio module graph in a fresh isolate per process()). - IndexerState gets an injectable ~onExit; ExitOnCaughtUp resolves it instead of process.exit, so a caught-up run in-process resolves a promise rather than killing the test runner. Production default unchanged. - TestIndexer builds a per-instance in-memory storage (config-derived initial state, never a real DB) and drives IndexerState/IndexerLoop directly; runs bypass Persistence.init and stop the loop on completion. - Registrations are captured once and cloned per run (patchConfig appends a simulate source), so independent createTestIndexer instances run in parallel without shared mutable registration state. - Handlers run inside an AsyncLocalStorage scope so a handler calling indexer.onEvent throws (as in production) without finishing the global registration the test itself uses. - Delete TestIndexerWorker and the proxy's message-channel machinery. - Restore vitest to envio devDependencies (needed to resolve the Vitest binding for local scenario test runs). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fb6Yyif9GZG1DWAiCER4Gh * Drop JSON round-trip from TestIndexer in-memory storage The worker model serialized entities to JSON to cross the thread boundary. In-process there is no boundary, so store and load entities decoded: - handleLoad filters decoded entities with the already-typed filter and returns them directly (no serialize/parse, no rowsSchema round-trip). - handleWriteBatch takes Persistence.updatedEntity and stores the decoded entities as-is instead of encoding then re-parsing them. - Delete TestIndexerProxyStorage entirely — its serializable types were the only remaining use. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fb6Yyif9GZG1DWAiCER4Gh * Isolate test indexer entity store with defensive copies (#1472) * Harden in-process TestIndexer: isolate store, drop dead resume plumbing - Copy entities on the set/get boundary so user code mutating a returned entity (or an object passed to `set`) can't corrupt the in-memory store; add a regression test. - Remove the unused `resumeInitialState`/`currentInitialState` injection — the runner sets `storageStatus = Ready(...)` directly, so it was never reached; make `resumeInitialState` a throwing stub like the other unused storage methods. - Await the in-flight write fiber in cleanup instead of busy-polling it. - Document the loss of per-test isolation (shared module/effect-cache state) in the indexer-testing skill. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DAwNk4gUT3QV6dh7uBtef1 * Replace AsyncLocalStorage guard with plain finished-registration flag The in-process runner used an AsyncLocalStorage scope so a handler calling `indexer.onEvent` at runtime would throw, while still letting tests call the registration API after a run (which required clearing the finished flag). The only thing depending on the API staying callable post-registration was two type-surface tests; those are compile-time only and don't need to run after registration. - Move the `indexer.onEvent`/`contractRegister` type-surface checks into EventHandlers.ts under `if (0)` (type-checked, never executed), and drop the two runtime `it(...)` cases. - Delete the AsyncLocalStorage scope, `runInHandlerScope`/`isInHandlerScope`, and `clearActiveRegistration`; restore `throwIfFinishedRegistration` to the plain `finished` check. A handler registering at runtime now throws via the finished flag, as in production. - getRegistrations no longer reopens the registration API after capturing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DAwNk4gUT3QV6dh7uBtef1 * Tighten test-isolation note in indexer-testing skill Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DAwNk4gUT3QV6dh7uBtef1 --------- Co-authored-by: Claude <noreply@anthropic.com> * Collapse makeCreateTestIndexer factory into createTestIndexer The (~config) => unit => t currying was a dependency-injection seam (it once also threaded ~workerPath); nothing injects a custom config and its only caller invoked it immediately. Fold it into a single createTestIndexer() that loads the memoized config internally. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fb6Yyif9GZG1DWAiCER4Gh * Test handler errors and improve TestIndexer error messages (#1474) * Fix stale TestIndexer comments and drop dead chainId tuple element - copyEntity comment no longer claims all entity fields are immutable scalars; shallow copy still shares array-valued fields. - Rollback storage stubs no longer advise setting rollbackOnReorg to false — the runner already forces it off, making them unreachable. - Drop the unused chainId element from chainEntries (validation kept). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NCpLSmKU3MZaAyGRjmcAVc * Add regression test for handler-throw propagation A synchronous throw in an event handler body must reject the process() promise at the call site, carrying the handler's original message. This path (handler-body throw) was only covered indirectly before, via effect and getOrThrow throws. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NCpLSmKU3MZaAyGRjmcAVc --------- Co-authored-by: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> * Friendlier event-registration validation and simulate handler checks - Reject duplicate event names per contract at parse time, suggesting the `name` alias for overloads (replaces the never-implemented uniqueness TODO). - Reword the same-dispatch-signature error so it reads clearly (names are unique by the check above, so it never prints "X and X"). - Simulate now applies the same `where:false` drop as finishRegistration and fails loudly when an event has no handler registered, instead of silently fabricating a bare registration. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lcy9XY2kAFMscJRukX7Fqi * Test the new validation and simulate handler behavior ConfigYaml: same-signature rejection with distinct names, byte-identical duplicate, same-name overload (alias suggestion), the SVM hex-casing case, and a success case where a `name` alias resolves an overload. HandlerRegister: simulate returns the handled registration, drops a `where:false` one, and returns nothing for an unhandled event. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lcy9XY2kAFMscJRukX7Fqi * Revert simulate-side changes that broke handler-less simulate The where:false drop and no-handler failure lived in the shared `getSimulateOnEventRegistrations`, which is also used by module-load-time registration probes (`MockConfig.getOnEventRegistration`) that need the bare fallback and the un-dropped registration. And failing on a handler-less event contradicts the framework's intentional pattern of simulating one (e.g. `Noop.EmptyEvent`) as a chain-advance marker. Keeps the parse-time event-name validation (items 1 and 4), which is unaffected. The simulate where:false drop needs to live in the simulate path only; revisiting separately. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lcy9XY2kAFMscJRukX7Fqi * Fail simulate on unhandled events; drop where:false in simulate path only Apply the no-handler failure and the where:false drop in SimulateItems (the user-facing simulate path) instead of the shared getSimulateOnEventRegistrations, so module-load registration probes (MockConfig) keep their bare-fallback behavior. Simulate now fans out only to registrations that would actually run on the chain (a real handler/contractRegister, where not excluded), and throws a clear error when none would. Give Noop.EmptyEvent a no-op handler so the multichain-ordering test still processes an event on chain 1 (its only event) without writing an entity, and point the new no-handler failure test at Gravatar.TestEventWithReservedKeyword, which is defined but genuinely handler-less. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lcy9XY2kAFMscJRukX7Fqi --------- Co-authored-by: Claude <noreply@anthropic.com>
DZakh
added a commit
that referenced
this pull request
Jul 23, 2026
* Allow multiple onEvent registrations per event Registering the same event more than once no longer errors or composes handlers. Each `indexer.onEvent` becomes its own registration; a `contractRegister` merges into a matching handler registration (either registration order, matched on the resolved `where` and wildcard flag), otherwise it stands on its own. Wildcard registrations are no longer capped at one per signature. The routing, buffer dedup (keyed on registration index), and query selection layers already fanned one log out to every matching registration, so this only reshapes the registration layer: - HandlerRegister resolves each call eagerly into a process-global per-chain store (survives an import-cached re-registration cycle; one config per isolate). `finishRegistration` just backfills raw-event-only regs, drops where-empty EVM regs, and assigns the chain-scoped index. The raw persistent slot, per-call resolved store, and the merge/throw paths are gone. - The duplicate-event guard moves from a runtime check to config parse (`Contract::new`), keyed on sighash + indexed-topic count for EVM and the discriminator for SVM. The wildcard-interference guard is dropped. - simulate fans a simulated event out to every registration. raw_events keeps one row per fetched item, so a log matched by N registrations writes N rows (no dedup, by design). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBsVqEGhVkg7EyTjAdcy52 * Don't drop where-empty registrations at finish A `where` resolving to no topic selections already contributes no query terms, so the registration fetches nothing on its own — the explicit drop in finishRegistration wasn't needed here. Keeping the registration also removes its awkward ordering dependency with the raw-event backfill (a filtered-out event still counts as registered, so no bare raw-event reg is added for it). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBsVqEGhVkg7EyTjAdcy52 * Store chain-independent registration intents; fix review findings Registration reverts to storing chain-independent handler/contractRegister intents, resolved per-config at finishRegistration, instead of a per-chain resolved cache. The cache was only populated while handler modules ran, but those are import-cached and registerAllHandlers is called repeatedly with a narrowed config (TestIndexer narrows chainMap per process() call) — so chains absent when handlers first ran got no registrations. Intents are config-independent, so finishRegistration materializes registrations for whatever chains the current config has, and repeated registration is idempotent. contractRegister→handler merge and the where-empty drop now happen at finishRegistration. Also from PR review: - Fuel events participate in the parser-level duplicate-dispatch-key check, keyed on sighash (logId / mint/burn/transfer/call), matching the router. - Restore dropping EVM registrations whose where resolves to no topic selections (per-chain opt-out) at finishRegistration. - Remove the intentional duplicate-handler fixtures (and their two composition tests) that only existed to exercise the removed handler composition; they otherwise inflate every downstream fetch/query count assertion. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBsVqEGhVkg7EyTjAdcy52 * Validate where at call site; drop composition-era test fixtures - Restore call-site `where` validation: setHandler/setContractRegister resolve the intent against the active config before storing it, so an invalid `where` throws at the registration call site again (not deferred to finishRegistration). - Remove the intentional duplicate-registration fixtures that only exercised the removed handler composition / mismatched-options throw: a second Gravatar.FactoryEvent handler+contractRegister pair (captured-add test, whose property is already covered by throwOnHangingRegistration) and a CustomSelection wildcard re-registration. These added extra registrations to the Gravatar contract, inflating fetch/query counts across the E2E and rollback suites. Drop the now-orphaned captured-add test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBsVqEGhVkg7EyTjAdcy52 * Say "dispatch key" not "signature" in duplicate-event error The per-contract collision key is a dispatch key (EVM sighash + indexed count, Fuel logId/receipt kind, SVM discriminator), not strictly a signature, so the diagnostic wording is more accurate this way. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBsVqEGhVkg7EyTjAdcy52 * Validate intent `where` across all matching chains `validateIntentWhere` used `Array.some`, stopping at the first chain that defines the event. A `where` that resolves fine on the first chain but is structurally invalid on a later one slipped past the call-site check and only threw at `finishRegistration`. Iterate every matching chain so the error surfaces at the registration call site regardless of which chain's resolution is invalid. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBsVqEGhVkg7EyTjAdcy52 * Capture raw events for where:false events; dedupe raw events per log - finishRegistration: when raw events are enabled, an event whose `where` opts out of a chain (empty topic selections) is no longer fully dropped — its handler still doesn't run there, but a bare (handler-less) registration is added so the event's logs are captured for `raw_events`. The backfill now keys on surviving registrations; a `where: false` event still isn't reported as handler-less (tracked via intentKeys). - PgStorage.writeBatch: a single log fans out to one item per matching registration, but `raw_events` records the log, so dedupe rows by log coordinate (chain, block, logIndex) — one row per log. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBsVqEGhVkg7EyTjAdcy52 * Exclude where:false events from raw events; keep handler-less backfill Correct the raw-events behavior: a `where: false` event has a handler that opted out of the chain, so it gets no registration there at all — not even a raw-events one. Event configs with no explicit handler still get a bare raw-events registration when raw events are enabled. This reverts the finishRegistration change from the previous commit back to keying the backfill on the resolved registrations; the raw-events row dedup (PgStorage) stays. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBsVqEGhVkg7EyTjAdcy52 * Simplify duplicate-event message; move dup tests to ReScript Reword the same-contract duplicate-event error to drop the "dispatch key" jargon. Normalize SVM discriminators (lowercase) before keying so hex-casing variants collide, matching the router. Replace the two Rust `#[test]` cases with parseYaml tests in the ReScript suite, plus an SVM casing-collision case. Replay pre-registered handlers in FIFO source order so multi-handler dispatch order matches registration order, with a regression test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBsVqEGhVkg7EyTjAdcy52 * Resolve onEvent `where` once per chain; add dispatch-order test Cache each registration intent's per-chain resolution on the intent, so the user's `where` callback runs exactly once per chain instead of once at the registration call site and again at every `finishRegistration`/simulate. This restores the "invoked exactly once per chain" invariant. Add an end-to-end test: two handlers on one event (Gravatar.MultiHandlerOrder) both run, ordered by (blockNumber, logIndex, registration index), plus a unit test asserting a `where` callback is invoked once per chain across repeated finishes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBsVqEGhVkg7EyTjAdcy52 * Add MultiHandlerOrder to Indexer_test expected ABI The new Gravatar.MultiHandlerOrder event shifts the generated contract ABI, which `Indexer_test`'s full chain-config deep-equal asserts on. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBsVqEGhVkg7EyTjAdcy52 * Resolve registrations at onEvent into one reused registration Replace the chain-independent intent array + per-intent resolved-where cache with a single persistent `activeRegistration` that handlers resolve into at `onEvent`/`onBlock` call time (the `where` callback runs once per chain there, at the call site). `startRegistration` is idempotent, so the import-cached handlers register once and every later `finishRegistration` reuses the result; `finishRegistration` reads the store and builds a fresh per-config output (merge, raw-events backfill, index) without mutating it, so a run appending a simulate/mock source stays isolated. onEvent + onBlock now share the `chainRegistrations` type in the session store. onBlock validation runs against the config start block (registration sees the full, un-narrowed config); the persistence-derived resume block overrides downstream without re-validation. MockIndexer registers once against the full config and narrows per run. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBsVqEGhVkg7EyTjAdcy52 * Match exact duplicate-event error messages after base merge Base changed `expectParseError` to compare the full parse-error string exactly, so the duplicate-event assertions need the complete message (including the `Config parse error:` prefix and, for the EVM case, the `Failed parsing globally defined contract` context) instead of a substring. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBsVqEGhVkg7EyTjAdcy52 * Eliminate worker thread from TestIndexer, store entities decoded (#1476) * Run createTestIndexer in-process instead of worker threads (#1467) * Run createTestIndexer in-process instead of worker threads Replace the per-chain worker + storage message-proxy with an in-process run against an in-memory Persistence.storage, removing the dominant cost (re-evaluating the envio module graph in a fresh isolate per process()). - IndexerState gets an injectable ~onExit; ExitOnCaughtUp resolves it instead of process.exit, so a caught-up run in-process resolves a promise rather than killing the test runner. Production default unchanged. - TestIndexer builds a per-instance in-memory storage (config-derived initial state, never a real DB) and drives IndexerState/IndexerLoop directly; runs bypass Persistence.init and stop the loop on completion. - Registrations are captured once and cloned per run (patchConfig appends a simulate source), so independent createTestIndexer instances run in parallel without shared mutable registration state. - Handlers run inside an AsyncLocalStorage scope so a handler calling indexer.onEvent throws (as in production) without finishing the global registration the test itself uses. - Delete TestIndexerWorker and the proxy's message-channel machinery. - Restore vitest to envio devDependencies (needed to resolve the Vitest binding for local scenario test runs). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fb6Yyif9GZG1DWAiCER4Gh * Drop JSON round-trip from TestIndexer in-memory storage The worker model serialized entities to JSON to cross the thread boundary. In-process there is no boundary, so store and load entities decoded: - handleLoad filters decoded entities with the already-typed filter and returns them directly (no serialize/parse, no rowsSchema round-trip). - handleWriteBatch takes Persistence.updatedEntity and stores the decoded entities as-is instead of encoding then re-parsing them. - Delete TestIndexerProxyStorage entirely — its serializable types were the only remaining use. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fb6Yyif9GZG1DWAiCER4Gh * Isolate test indexer entity store with defensive copies (#1472) * Harden in-process TestIndexer: isolate store, drop dead resume plumbing - Copy entities on the set/get boundary so user code mutating a returned entity (or an object passed to `set`) can't corrupt the in-memory store; add a regression test. - Remove the unused `resumeInitialState`/`currentInitialState` injection — the runner sets `storageStatus = Ready(...)` directly, so it was never reached; make `resumeInitialState` a throwing stub like the other unused storage methods. - Await the in-flight write fiber in cleanup instead of busy-polling it. - Document the loss of per-test isolation (shared module/effect-cache state) in the indexer-testing skill. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DAwNk4gUT3QV6dh7uBtef1 * Replace AsyncLocalStorage guard with plain finished-registration flag The in-process runner used an AsyncLocalStorage scope so a handler calling `indexer.onEvent` at runtime would throw, while still letting tests call the registration API after a run (which required clearing the finished flag). The only thing depending on the API staying callable post-registration was two type-surface tests; those are compile-time only and don't need to run after registration. - Move the `indexer.onEvent`/`contractRegister` type-surface checks into EventHandlers.ts under `if (0)` (type-checked, never executed), and drop the two runtime `it(...)` cases. - Delete the AsyncLocalStorage scope, `runInHandlerScope`/`isInHandlerScope`, and `clearActiveRegistration`; restore `throwIfFinishedRegistration` to the plain `finished` check. A handler registering at runtime now throws via the finished flag, as in production. - getRegistrations no longer reopens the registration API after capturing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DAwNk4gUT3QV6dh7uBtef1 * Tighten test-isolation note in indexer-testing skill Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DAwNk4gUT3QV6dh7uBtef1 --------- Co-authored-by: Claude <noreply@anthropic.com> * Collapse makeCreateTestIndexer factory into createTestIndexer The (~config) => unit => t currying was a dependency-injection seam (it once also threaded ~workerPath); nothing injects a custom config and its only caller invoked it immediately. Fold it into a single createTestIndexer() that loads the memoized config internally. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fb6Yyif9GZG1DWAiCER4Gh * Test handler errors and improve TestIndexer error messages (#1474) * Fix stale TestIndexer comments and drop dead chainId tuple element - copyEntity comment no longer claims all entity fields are immutable scalars; shallow copy still shares array-valued fields. - Rollback storage stubs no longer advise setting rollbackOnReorg to false — the runner already forces it off, making them unreachable. - Drop the unused chainId element from chainEntries (validation kept). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NCpLSmKU3MZaAyGRjmcAVc * Add regression test for handler-throw propagation A synchronous throw in an event handler body must reject the process() promise at the call site, carrying the handler's original message. This path (handler-body throw) was only covered indirectly before, via effect and getOrThrow throws. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NCpLSmKU3MZaAyGRjmcAVc --------- Co-authored-by: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> * Friendlier event-registration validation and simulate handler checks - Reject duplicate event names per contract at parse time, suggesting the `name` alias for overloads (replaces the never-implemented uniqueness TODO). - Reword the same-dispatch-signature error so it reads clearly (names are unique by the check above, so it never prints "X and X"). - Simulate now applies the same `where:false` drop as finishRegistration and fails loudly when an event has no handler registered, instead of silently fabricating a bare registration. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lcy9XY2kAFMscJRukX7Fqi * Test the new validation and simulate handler behavior ConfigYaml: same-signature rejection with distinct names, byte-identical duplicate, same-name overload (alias suggestion), the SVM hex-casing case, and a success case where a `name` alias resolves an overload. HandlerRegister: simulate returns the handled registration, drops a `where:false` one, and returns nothing for an unhandled event. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lcy9XY2kAFMscJRukX7Fqi * Revert simulate-side changes that broke handler-less simulate The where:false drop and no-handler failure lived in the shared `getSimulateOnEventRegistrations`, which is also used by module-load-time registration probes (`MockConfig.getOnEventRegistration`) that need the bare fallback and the un-dropped registration. And failing on a handler-less event contradicts the framework's intentional pattern of simulating one (e.g. `Noop.EmptyEvent`) as a chain-advance marker. Keeps the parse-time event-name validation (items 1 and 4), which is unaffected. The simulate where:false drop needs to live in the simulate path only; revisiting separately. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lcy9XY2kAFMscJRukX7Fqi * Fail simulate on unhandled events; drop where:false in simulate path only Apply the no-handler failure and the where:false drop in SimulateItems (the user-facing simulate path) instead of the shared getSimulateOnEventRegistrations, so module-load registration probes (MockConfig) keep their bare-fallback behavior. Simulate now fans out only to registrations that would actually run on the chain (a real handler/contractRegister, where not excluded), and throws a clear error when none would. Give Noop.EmptyEvent a no-op handler so the multichain-ordering test still processes an event on chain 1 (its only event) without writing an entity, and point the new no-handler failure test at Gravatar.TestEventWithReservedKeyword, which is defined but genuinely handler-less. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lcy9XY2kAFMscJRukX7Fqi --------- Co-authored-by: Claude <noreply@anthropic.com> * Point merged tests at InternalTestIndexer.fromUserApi The merged main renamed the config helper (`MockIndexerConfig` -> `InternalTestIndexer.fromUserApi`); update the multi-registration and duplicate-event tests accordingly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBsVqEGhVkg7EyTjAdcy52 * Pin Noop.EmptyEvent handler to chain 1; drop MultiHandlerOrder fixture Noop.EmptyEvent's no-op handler registered on every chain that lists the Noop contract, including chain 137 where Noop has an address — adding a fetch partition that the rollback/reorg tests (one partition per chain) don't expect. Pin the handler to chain 1, its only intended chain. Drop the MultiHandlerOrder event + its two handlers and the end-to-end dispatch-order test: adding a new event to the shared chain-1337 Gravatar contract perturbs partition snapshots in the rollback suite. Dispatch ordering by (blockNumber, logIndex, registration index) stays covered by the FetchState buffer-order test and the HandlerRegister index-order tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EBsVqEGhVkg7EyTjAdcy52 --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes
createTestIndexerrun entirely in-process, removing the per-chain worker thread and its storage message-proxy.Why
Each
process()call spawned a worker thread per chain, which re-evaluated the entire envio module graph in a fresh isolate and round-tripped every load/write over a message channel. That startup cost dominated test time.Result: ~12× faster. On
test_codegen/test/EventHandler.test.ts(72 tests):Independent
createTestIndexer()instances can also run theirprocess()calls in parallel (verified) — each has its own in-memory storage andIndexerState; registrations are captured once and cloned per run, so nothing mutable is shared.How
~onExitonIndexerState—ExitOnCaughtUpcalls it instead ofprocess.exit(Success), so a caught-up run resolves a promise in-process rather than killing the test runner. Production default is unchanged (still exits).Persistence.storageover the entity store, with a config-derived initial state (resumeInitialState) — runs bypassPersistence.initand never touch a real database.IndexerState/IndexerLoopdirectly with the simulate source (SimulateItems.patchConfig), then stops the loop on completion.HandlerRegister;registerAllHandlersruns once and eachprocess()run gets its own clone so the simulate-source registration stays isolated.handleLoad/handleWriteBatchwork on decoded entities directly.TestIndexerWorkerandTestIndexerProxyStorage(the whole message-channel layer).vitestto envio's devDependencies (needed to resolve envio'sVitestbinding for local scenario test runs).Behavior change
Because handler registration is now finished in the test process itself (previously each worker isolate registered independently), calling
indexer.onEvent/indexer.contractRegisterat runtime — e.g. inside a test body — throws "Cannot callindexer.onEventafter the indexer has started" once registration has completed. The runtime type-surface tests inEventHandler.test.tswere moved to compile-only checks insrc/handlers/EventHandlers.tsfor this reason. Tests that register handlers dynamically per test need the same treatment.Verification
pnpm rescript+ vitest on all createTestIndexer consumers intest_codegen(EventHandler.test.ts,CustomSelection.test.ts,WildcardSimulate_test,SimulateDynamicAddress_test,OptionalBlockParams_test,CustomSelection_test— 92 tests) all pass, plus a parallel-process()test. Core regression set (MockIndexer loop, rollback, E2E, SourceManager, HandlerRegisterLifecycle, Indexer) also green. Fuel/SVM scenarioindexer.test.tsare covered by CI (unchanged API + ecosystem-agnostic simulate path).🤖 Generated with Claude Code
https://claude.ai/code/session_01Fb6Yyif9GZG1DWAiCER4Gh
Summary by CodeRabbit
New Features
Bug Fixes
Documentation