Support chain IDs above int32 max with configurable storage mode - #1507
Conversation
Chain ids beyond 2^31-1 (Tron Shasta 2494104990, Nile 3448148188) were rejected by the `S.int` config schema and would not fit the INTEGER columns the internal tables declare. The CLI now derives a `ChainIdMode` from the maximum active chain id (`<= i32::MAX` -> Int32, otherwise Int64) and emits it as `chainIdMode` in the public config, which is also the persisted envio_info fingerprint. It sits in its own diff tier, so a resume against a schema built for the other mode fails with the standard incompatible-config message instead of silently truncating ids. Ids above Number.MAX_SAFE_INTEGER are rejected at parse time. At runtime a new `ChainId` module carries the float-backed representation plus the validating schema, which also normalizes the strings Postgres BIGINT and ClickHouse UInt64 columns return. The internal tables declare their chain-id columns with a `ChainId` field type that resolves to INTEGER/Int32 or BIGINT/UInt64 from the mode, so they stay module-level constants and small-id projects keep generating identical DDL. Generated APIs are unchanged for small ids; a wide config falls back to `type chainId = ChainId.t` in ReScript (integer polyvariants are int32-bound) while TypeScript keeps its numeric literal union. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECqzHdVS4cLxfCXPw8zpb6
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughChain IDs now resolve to Int32 or Int64 modes, flow through public configuration and generated indexer code, use validated runtime normalization, and select matching Postgres and ClickHouse column and query types. ChangesChain ID mode support
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant ConfigParser
participant PublicConfig
participant CodeGenerator
participant Runtime
participant Storage
participant Database
ConfigParser->>ConfigParser: Resolve ChainIdMode from configured chains
ConfigParser->>PublicConfig: Serialize chainIdMode
PublicConfig->>CodeGenerator: Provide Int32 or Int64 mode
CodeGenerator->>Runtime: Generate matching chain ID types and lookups
Runtime->>Storage: Pass ChainId values and chainIdMode
Storage->>Database: Create schemas and queries using mode-specific types
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/envio/src/PgStorage.res (1)
600-621: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winScope cached batch SQL by
chainIdMode.
setQueryCacheis keyed only bytable, butnewQueryembeds mode-specific casts such asinteger[]orbigint[]. A laterPgStorage.makeusingInt64can reuse anInt32query for the same table and reject wide IDs. Scope this cache to the storage instance, or includepgSchemaandchainIdModein its key.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/envio/src/PgStorage.res` around lines 600 - 621, Update setQueryCache usage in setOrThrow so cached batch SQL is scoped by pgSchema and chainIdMode, or move the cache to the storage instance. Ensure makeTableBatchSetQuery results using mode-specific casts are never reused across different schemas or chain-ID modes for the same table.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/cli/src/hbs_templating/codegen_templates.rs`:
- Around line 1340-1353: Update the chain ID generation around chain_id_type and
the exhaustive switch to derive Int32 cases only from active, non-skipped
chains, matching the filtering used during mode resolution. Ensure skipped wide
IDs are excluded from both generated lists, and add a regression test covering
an active id: 1 with skipped id: 2494104990.
In `@packages/envio/src/bindings/ClickHouse.res`:
- Line 132: Update the ChainId branch in the ClickHouse value schema to apply
the same nullable and array wrapping logic used by the Date and UInt52 branches,
based on f.isNullable and f.isArray, while retaining ChainId.intSchema as the
underlying scalar schema.
In `@packages/envio/src/ChainId.res`:
- Around line 31-45: Update the string-handling branch in the schema preprocess
parser so it validates that the entire input string is a numeric representation
before calling Float.parseFloat. Reject strings containing trailing non-numeric
characters, such as "2494104990junk", while preserving the existing safe-integer
and non-negative range checks for valid values.
In `@packages/envio/src/Config.res`:
- Around line 1276-1286: Normalize both configuration values in the
compatibility diff flow so an omitted chainIdMode is treated as the
Int32/"int32" mode before diffPaths compares them. Update the logic around the
visible chainIdMode diff tier, preserving explicit modes and ensuring legacy
configs without the field match equivalent configs that specify int32.
---
Outside diff comments:
In `@packages/envio/src/PgStorage.res`:
- Around line 600-621: Update setQueryCache usage in setOrThrow so cached batch
SQL is scoped by pgSchema and chainIdMode, or move the cache to the storage
instance. Ensure makeTableBatchSetQuery results using mode-specific casts are
never reused across different schemas or chain-ID modes for the same table.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 2a3cdb8c-1e76-4abe-9239-7c262f7b3bd0
⛔ Files ignored due to path filters (7)
packages/cli/src/hbs_templating/snapshots/envio__hbs_templating__codegen_templates__test__internal_config_json_code_generated_for_evm.snapis excluded by!**/*.snappackages/cli/src/hbs_templating/snapshots/envio__hbs_templating__codegen_templates__test__internal_config_json_code_generated_for_fuel.snapis excluded by!**/*.snappackages/cli/src/hbs_templating/snapshots/envio__hbs_templating__codegen_templates__test__internal_config_json_code_generated_for_svm.snapis excluded by!**/*.snappackages/cli/src/hbs_templating/snapshots/envio__hbs_templating__codegen_templates__test__internal_config_json_code_with_all_options.snapis excluded by!**/*.snappackages/cli/src/hbs_templating/snapshots/envio__hbs_templating__codegen_templates__test__internal_config_json_code_with_lowercase_contract_name.snapis excluded by!**/*.snappackages/cli/src/hbs_templating/snapshots/envio__hbs_templating__codegen_templates__test__internal_config_json_code_with_multiple_contracts.snapis excluded by!**/*.snappackages/cli/src/hbs_templating/snapshots/envio__hbs_templating__codegen_templates__test__internal_config_json_code_with_no_contracts.snapis excluded by!**/*.snap
📒 Files selected for processing (14)
packages/cli/src/config_parsing/public_config.rspackages/cli/src/config_parsing/system_config.rspackages/cli/src/hbs_templating/codegen_templates.rspackages/envio-tests/test/lib_tests/ChainIdMode_test.respackages/envio/src/ChainId.respackages/envio/src/ChainId.resipackages/envio/src/ChainMap.respackages/envio/src/Config.respackages/envio/src/PgStorage.respackages/envio/src/Sink.respackages/envio/src/bindings/ClickHouse.respackages/envio/src/db/InternalTable.respackages/envio/src/db/Table.resscenarios/test_codegen/test/lib_tests/PgStorage_test.res
| // ReScript integer polyvariants (`#137`) are int32-bound, so a config | ||
| // with a wider id falls back to the opaque runtime representation. | ||
| // TypeScript keeps its numeric literal union either way. | ||
| let chain_id_type = match cfg.chain_id_mode { | ||
| ChainIdMode::Int64 => "type chainId = ChainId.t".to_string(), | ||
| ChainIdMode::Int32 => format!( | ||
| "type chainId = [{}]", | ||
| chain_id_cases | ||
| .iter() | ||
| .map(|chain_id_case| format!("#{}", chain_id_case)) | ||
| .collect::<Vec<_>>() | ||
| .join(" | "), | ||
| ), | ||
| }; |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Exclude skipped chains from generated Int32 cases.
Mode resolution ignores skipped chains, but chain_id_cases and the exhaustive switch include them. An active id: 1 plus skipped id: 2494104990 selects Int32 then emits #2494104990, which ReScript cannot compile. Build both generated Int32 lists from active chains, and add this skipped-wide-ID regression case.
Also applies to: 1485-1521
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/cli/src/hbs_templating/codegen_templates.rs` around lines 1340 -
1353, Update the chain ID generation around chain_id_type and the exhaustive
switch to derive Int32 cases only from active, non-skipped chains, matching the
filtering used during mode resolution. Ensure skipped wide IDs are excluded from
both generated lists, and add a regression test covering an active id: 1 with
skipped id: 2494104990.
| dateSchema | ||
| } | ||
| } | ||
| | ChainId => ChainId.intSchema->S.toUnknown |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Handle nullable and array ChainId fields in the ClickHouse value schema.
DDL uses f.isNullable and f.isArray, but this branch always uses scalar ChainId.intSchema. Nullable or array ChainId entity values will fail conversion before insertion. Apply the same wrapping used for Date/UInt52.
Proposed fix
- | ChainId => ChainId.intSchema->S.toUnknown
+ | ChainId => {
+ let chainIdSchema = ChainId.intSchema->S.toUnknown
+ if f.isNullable {
+ S.null(chainIdSchema)->S.toUnknown
+ } else if f.isArray {
+ S.array(chainIdSchema)->S.toUnknown
+ } else {
+ chainIdSchema
+ }
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| | ChainId => ChainId.intSchema->S.toUnknown | |
| | ChainId => { | |
| let chainIdSchema = ChainId.intSchema->S.toUnknown | |
| if f.isNullable { | |
| S.null(chainIdSchema)->S.toUnknown | |
| } else if f.isArray { | |
| S.array(chainIdSchema)->S.toUnknown | |
| } else { | |
| chainIdSchema | |
| } | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/envio/src/bindings/ClickHouse.res` at line 132, Update the ChainId
branch in the ClickHouse value schema to apply the same nullable and array
wrapping logic used by the Date and UInt52 branches, based on f.isNullable and
f.isArray, while retaining ChainId.intSchema as the underlying scalar schema.
| let schema: S.t<t> = S.float->S.preprocess(s => { | ||
| parser: value => { | ||
| let number = switch value->typeof { | ||
| | #string => value->(Utils.magic: unknown => string)->Float.parseFloat | ||
| | _ => value->(Utils.magic: unknown => float) | ||
| } | ||
| if !isSafeInteger(number) || number < 0. { | ||
| s.fail( | ||
| `Expected a chain id between 0 and ${maxSafe->Float.toString}, received ${value->( | ||
| Utils.magic: unknown => string | ||
| )}`, | ||
| ) | ||
| } | ||
| number | ||
| }, |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🌐 Web query:
In ReScript 12, does Float.parseFloat("123junk") accept the numeric prefix, and what documented API is recommended for strict numeric-string validation?
💡 Result:
In ReScript 12 (as in previous versions), Float.parseFloat("123junk") does accept the numeric prefix and returns 123.0 [1][2]. This function maintains the underlying behavior of JavaScript's global parseFloat, which parses the string from left to right, stops at the first character that cannot be part of a valid numeric literal, and ignores all subsequent characters [3][4][5]. The official ReScript documentation explicitly recommends using fromString (available in @rescript/core or Belt) for scenarios where you need to ensure the input represents a valid float and avoid the lenient, potentially error-prone behavior of parseFloat [1][2][6]. Unlike parseFloat, fromString returns an option (e.g., option<float>), providing None if the input cannot be fully or correctly parsed as a number [6][7]. For strict validation requirements where you must ensure a string is formatted entirely as a valid number, the community standard is to use a schema validation library such as rescript-struct [8] or rescript-schema [9]. These libraries allow you to define stricter rules, such as using regular expressions (e.g., S.string->S.pattern(%re("/^-?\d+(\.\d+)?$/"))) to validate that the entire input matches a strictly numeric format before proceeding with conversion [5][9][8].
Citations:
- 1: https://rescript-lang.org/docs/manual/api/stdlib/float/
- 2: https://v11.rescript-lang.org/docs/manual/v11.0.0/api/core/float
- 3: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global%5FObjects/parseFloat
- 4: https://jsguides.dev/reference/number/parse-float/
- 5: https://runebook.dev/en/docs/javascript/global_objects/number/parsefloat
- 6: https://github.com/rescript-lang/rescript-core/
- 7: https://www.npmjs.com/package/@rescript/core
- 8: https://github.com/DZakh/rescript-struct/blob/v5.1.0/docs/js-usage.md
- 9: https://github.com/DZakh/rescript-schema/blob/main/docs/rescript-usage.md
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== packages/envio/src/ChainId.res =="
wc -l packages/envio/src/ChainId.res
sed -n '1,120p' packages/envio/src/ChainId.res
echo
echo "== parseFloat behavior probe =="
node - <<'JS'
for (const v of ["2494104990junk", "2494104990", "12.3abc", "abc", "9007199254740992junk"]) {
const n = parseFloat(v);
console.log(JSON.stringify({v, n, isSafeInteger: Number.isSafeInteger(n), nonNegative: n >= 0}));
}
JSRepository: enviodev/hyperindex
Length of output: 2929
Reject strings with trailing junk before parsing. Float.parseFloat accepts numeric prefixes, so values like "2494104990junk" still pass the range check and normalize to a chain id. Require a full-string numeric check first.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/envio/src/ChainId.res` around lines 31 - 45, Update the
string-handling branch in the schema preprocess parser so it validates that the
entire input string is a numeric representation before calling Float.parseFloat.
Reject strings containing trailing non-numeric characters, such as
"2494104990junk", while preserving the existing safe-integer and non-negative
range checks for valid values.
| // chainIdMode sits right after version: it decides the physical type of | ||
| // every chain-id column, so a change to it is reported on its own rather | ||
| // than buried under the chain diffs that always accompany it. | ||
| let tiers = [ | ||
| ["version"], | ||
| ["chainIdMode"], | ||
| ["name"], | ||
| ["storage"], | ||
| ["evm", "fuel", "svm"], | ||
| ["entities"], | ||
| ] |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Treat a missing legacy mode as int32 during compatibility checks.
fromPublic defaults an absent field to Int32, but diffPaths compares raw JSON. Existing persisted configs without chainIdMode will therefore fail resume against a new equivalent config containing "chainIdMode": "int32". Normalize an omitted mode to int32 before diffing.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/envio/src/Config.res` around lines 1276 - 1286, Normalize both
configuration values in the compatibility diff flow so an omitted chainIdMode is
treated as the Int32/"int32" mode before diffPaths compares them. Update the
logic around the visible chainIdMode diff tier, preserving explicit modes and
ensuring legacy configs without the field match equivalent configs that specify
int32.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f067f9ab80
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| fn resolve(chains: &ChainMap) -> Result<Self> { | ||
| let max_id = chains | ||
| .values() | ||
| .filter(|chain| !chain.skip) |
There was a problem hiding this comment.
When an active chain fits in int32 but a skipped chain has an ID such as 2494104990, this filter selects Int32; however, code generation intentionally includes skipped chains and emits that ID as a polyvariant pattern in getChainById. ReScript rejects the resulting #2494104990 value with an int32-range error, so the generated project cannot compile even though skip is documented as not affecting code generation. The mode selection (and maximum-safe-ID validation) therefore needs to account for every chain represented in generated code.
Useful? React with 👍 / 👎.
| // than buried under the chain diffs that always accompany it. | ||
| let tiers = [ | ||
| ["version"], | ||
| ["chainIdMode"], |
There was a problem hiding this comment.
When an existing envio_info row predates chainIdMode but its version and remaining config otherwise match, the current generated config now contains "chainIdMode": "int32", and this tier treats the missing/present pair as incompatible. Resume consequently throws and asks the user to reset compatible data whose chain columns are already INTEGER. The fromPublic default does not help because diffPaths compares the raw JSON objects, so the stored side should be normalized to int32 before this comparison.
Useful? React with 👍 / 👎.
| // The same runtime schema, typed for the modules that still annotate chain ids | ||
| // as `int`. Safe because ReScript's `int` is a JS number at runtime — chain ids | ||
| // are only ever compared and stringified, never used in int32 arithmetic. | ||
| let intSchema = schema->(Utils.magic: S.t<t> => S.t<int>) |
There was a problem hiding this comment.
For any active chain ID above 2147483647, this schema exposes the wide runtime number as a value typed int, but production code still reparses chain-ID dictionary keys with ReScript's int32-limited Int.fromString. In EventProcessing.processEventBatch, both the started and finished logging paths call chainId->Int.fromString->Option.getUnsafe, so the first progressed batch for a wide chain throws before normal handler processing; TestIndexer and Internal.EffectCache.parseChainId contain the same conversion. These paths need a chain-ID-specific safe-number parser rather than relying on the int type cast.
Useful? React with 👍 / 👎.
`ChainId.t` was only used at the config/table boundary; every module in between still annotated chain ids as `int`, which is exactly the type that can't represent them. Those annotations are now `ChainId.t`, so the compiler — not a comment — is what keeps a chain id from being treated as an int32. `ChainMap.Chain` is backed by it directly, and chain-keyed dictionaries go through `ChainId.Dict` instead of the int-keyed `Utils.Dict` helpers (which remain for the block-number-keyed dicts in FetchState and ReorgDetection). `ChainId.intSchema` is gone — `schema` is the only one left. Two boundaries deliberately stay `int` so no user code changes: `context.chain.id` (`Internal.chainInfo`) and `Envio.effectChain.id`. `fromInt`/`toInt` are the identity at runtime and mark those crossings, along with the int literals that construct chain ids in configs and tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECqzHdVS4cLxfCXPw8zpb6
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/envio/src/TestIndexer.res`:
- Around line 313-314: Update the chain ID handling in the block-range
configuration flow around parseBlockRange to parse each key once with
ChainId.normalizeOrThrow, avoiding Int.fromString and its range limitation.
Retain the parsed ChainId.t values for sorting and validation, and preserve the
existing invalid-ID error behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: d213ed63-3eb0-416a-83b3-4b318369f5a7
📒 Files selected for processing (80)
packages/envio-tests/test/ClientAddressFilter_test.respackages/envio-tests/test/RateLimit_test.respackages/envio-tests/test/ReorgDetection_test.respackages/envio-tests/test/SvmHyperSyncSource_test.respackages/envio-tests/test/UserApiValidation_test.respackages/envio-tests/test/lib_tests/ChainState_materialize_test.respackages/envio-tests/test/lib_tests/EffectCache_test.respackages/envio-tests/test/lib_tests/Metrics_test.respackages/envio/src/Batch.respackages/envio/src/ChainFetching.respackages/envio/src/ChainId.respackages/envio/src/ChainId.resipackages/envio/src/ChainMap.respackages/envio/src/ChainMap.resipackages/envio/src/ChainMetadata.respackages/envio/src/ChainState.respackages/envio/src/Config.respackages/envio/src/ContractRegisterContext.respackages/envio/src/CrossChainState.respackages/envio/src/EventConfigBuilder.respackages/envio/src/EventProcessing.respackages/envio/src/FetchState.respackages/envio/src/HandlerRegister.respackages/envio/src/HandlerRegister.resipackages/envio/src/IndexerState.respackages/envio/src/Internal.respackages/envio/src/LoadLayer.respackages/envio/src/LogSelection.respackages/envio/src/Main.respackages/envio/src/Metrics.respackages/envio/src/Persistence.respackages/envio/src/PgStorage.respackages/envio/src/Rollback.respackages/envio/src/RollbackCommit.respackages/envio/src/SafeCheckpointTracking.respackages/envio/src/SimulateDeadInputTracker.respackages/envio/src/SimulateItems.respackages/envio/src/TestIndexer.respackages/envio/src/UserContext.respackages/envio/src/bindings/ClickHouse.respackages/envio/src/db/EntityHistory.respackages/envio/src/db/InternalTable.respackages/envio/src/sources/Evm.respackages/envio/src/sources/Fuel.respackages/envio/src/sources/HyperSync.resipackages/envio/src/sources/RpcSource.respackages/envio/src/sources/SourceManager.respackages/envio/src/sources/SourceManager.resipackages/envio/src/sources/Svm.respackages/envio/src/tui/Tui.respackages/envio/src/tui/components/CustomHooks.resscenarios/fuel_test/test/FuelHyperSyncSourceHeight_test.resscenarios/test_codegen/test/ChainMeta_test.resscenarios/test_codegen/test/E2E_test.resscenarios/test_codegen/test/EventBlockFilter_test.resscenarios/test_codegen/test/EventFilters_test.resscenarios/test_codegen/test/HandlerRegisterLifecycle_test.resscenarios/test_codegen/test/IndexerStateStall_test.resscenarios/test_codegen/test/IndexerState_test.resscenarios/test_codegen/test/LoadLayer_test.resscenarios/test_codegen/test/RpcSourceContract_test.resscenarios/test_codegen/test/RpcSource_test.resscenarios/test_codegen/test/SourceBlockHashes_test.resscenarios/test_codegen/test/__mocks__/MockConfig.resscenarios/test_codegen/test/__mocks__/MockEvents.resscenarios/test_codegen/test/helpers/MockIndexer.resscenarios/test_codegen/test/helpers/RpcSourcePins.resscenarios/test_codegen/test/lib_tests/ChainState_test.resscenarios/test_codegen/test/lib_tests/CrossChainState_test.resscenarios/test_codegen/test/lib_tests/DynamicContractsStartupSize_test.resscenarios/test_codegen/test/lib_tests/EntityIdType_test.resscenarios/test_codegen/test/lib_tests/FetchState_onBlock_test.resscenarios/test_codegen/test/lib_tests/FetchState_test.resscenarios/test_codegen/test/lib_tests/HyperSyncDecoder_test.resscenarios/test_codegen/test/lib_tests/IndexerLoop_test.resscenarios/test_codegen/test/lib_tests/PgStorage_test.resscenarios/test_codegen/test/lib_tests/SameSignatureEventDecode_test.resscenarios/test_codegen/test/lib_tests/SourceManager_test.resscenarios/test_codegen/test/rollback/ChainMocking.resscenarios/test_codegen/test/rollback/Rollback_test.res
🚧 Files skipped from review as they are similar to previous changes (5)
- packages/envio/src/ChainMap.res
- packages/envio/src/Config.res
- packages/envio/src/db/InternalTable.res
- packages/envio/src/bindings/ClickHouse.res
- packages/envio/src/PgStorage.res
`Chain.t` was an alias for `ChainId.t`, and `makeUnsafe`/`toChainId` were both `%identity` — a second name for the same type, with a constructor that no longer constructed anything. Callers now pass `ChainId.t` directly; `ChainId.fromInt` is the one way to make a chain id from an int literal. ChainMap keeps its Belt.Map wrapper, keyed on ChainId.t. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECqzHdVS4cLxfCXPw8zpb6
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/envio/src/Internal.res (1)
367-368: 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy liftUse
ChainId.tforInternal.chainInfo.id
handlerContext.chain.idis read by generated handlers, and Int64 mode already allows chain ids above the ReScriptintrange. Keeping this field asintwill truncate wide ids.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/envio/src/Internal.res` around lines 367 - 368, Change the `Internal.chainInfo.id` field from `int` to `ChainId.t`, and update any directly associated construction or access code to preserve the full chain ID without narrowing or truncation. Keep the handler-facing `context.chain.id` behavior intact while using the existing `ChainId.t` representation.Source: MCP tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@packages/envio/src/Internal.res`:
- Around line 367-368: Change the `Internal.chainInfo.id` field from `int` to
`ChainId.t`, and update any directly associated construction or access code to
preserve the full chain ID without narrowing or truncation. Keep the
handler-facing `context.chain.id` behavior intact while using the existing
`ChainId.t` representation.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 3ebe4276-fd35-49e5-a94b-025c67fa03b4
📒 Files selected for processing (48)
packages/envio-tests/test/ClientAddressFilter_test.respackages/envio-tests/test/RateLimit_test.respackages/envio-tests/test/SvmHyperSyncSource_test.respackages/envio-tests/test/lib_tests/ChainIdMode_test.respackages/envio-tests/test/lib_tests/ChainState_materialize_test.respackages/envio/src/Batch.respackages/envio/src/ChainFetching.respackages/envio/src/ChainMap.respackages/envio/src/ChainMap.resipackages/envio/src/ChainState.respackages/envio/src/Config.respackages/envio/src/ContractRegisterContext.respackages/envio/src/CrossChainState.respackages/envio/src/CrossChainState.resipackages/envio/src/IndexerState.respackages/envio/src/IndexerState.resipackages/envio/src/Internal.respackages/envio/src/Main.respackages/envio/src/RawEvent.respackages/envio/src/Rollback.respackages/envio/src/SimulateDeadInputTracker.respackages/envio/src/SimulateItems.respackages/envio/src/TestIndexer.respackages/envio/src/sources/Evm.respackages/envio/src/sources/EvmHyperSyncSource.respackages/envio/src/sources/Fuel.respackages/envio/src/sources/FuelHyperSyncSource.respackages/envio/src/sources/RpcSource.respackages/envio/src/sources/SimulateSource.respackages/envio/src/sources/Source.respackages/envio/src/sources/SourceManager.respackages/envio/src/sources/Svm.respackages/envio/src/sources/SvmHyperSyncSource.resscenarios/fuel_test/test/FuelHyperSyncSourceHeight_test.resscenarios/test_codegen/test/IndexerStateStall_test.resscenarios/test_codegen/test/IndexerState_test.resscenarios/test_codegen/test/RpcSourceContract_test.resscenarios/test_codegen/test/RpcSource_test.resscenarios/test_codegen/test/SourceBlockHashes_test.resscenarios/test_codegen/test/__mocks__/MockConfig.resscenarios/test_codegen/test/helpers/MockIndexer.resscenarios/test_codegen/test/helpers/RpcSourcePins.resscenarios/test_codegen/test/lib_tests/CrossChainState_test.resscenarios/test_codegen/test/lib_tests/FetchState_onBlock_test.resscenarios/test_codegen/test/lib_tests/FetchState_test.resscenarios/test_codegen/test/lib_tests/HyperSyncDecoder_test.resscenarios/test_codegen/test/lib_tests/PgStorage_test.resscenarios/test_codegen/test/rollback/ChainMocking.res
🚧 Files skipped from review as they are similar to previous changes (32)
- packages/envio/src/ContractRegisterContext.res
- scenarios/fuel_test/test/FuelHyperSyncSourceHeight_test.res
- scenarios/test_codegen/test/SourceBlockHashes_test.res
- packages/envio-tests/test/lib_tests/ChainState_materialize_test.res
- scenarios/test_codegen/test/lib_tests/FetchState_onBlock_test.res
- scenarios/test_codegen/test/IndexerStateStall_test.res
- scenarios/test_codegen/test/RpcSourceContract_test.res
- packages/envio-tests/test/SvmHyperSyncSource_test.res
- packages/envio/src/sources/Evm.res
- scenarios/test_codegen/test/lib_tests/HyperSyncDecoder_test.res
- packages/envio/src/sources/Svm.res
- scenarios/test_codegen/test/lib_tests/FetchState_test.res
- scenarios/test_codegen/test/rollback/ChainMocking.res
- packages/envio-tests/test/RateLimit_test.res
- scenarios/test_codegen/test/helpers/RpcSourcePins.res
- packages/envio/src/Main.res
- packages/envio/src/sources/Fuel.res
- packages/envio/src/IndexerState.res
- packages/envio/src/Batch.res
- packages/envio/src/ChainFetching.res
- packages/envio/src/CrossChainState.res
- scenarios/test_codegen/test/helpers/MockIndexer.res
- packages/envio/src/sources/SourceManager.res
- packages/envio/src/ChainState.res
- packages/envio-tests/test/lib_tests/ChainIdMode_test.res
- packages/envio/src/SimulateDeadInputTracker.res
- packages/envio/src/Config.res
- scenarios/test_codegen/test/IndexerState_test.res
- scenarios/test_codegen/test/lib_tests/PgStorage_test.res
- packages/envio/src/Rollback.res
- scenarios/test_codegen/test/lib_tests/CrossChainState_test.res
- packages/envio/src/TestIndexer.res
Dead since the ChainId migration (or before it): `ChainId.toFloat`, `ChainId.equal` (`===` works on the opaque type and is what callers use), `ChainId.maxSafe`'s export, `ChainMap.set`/`entries`/`map`/`size`/ `update`, and `Utils.Dict.incrementByInt`. `ChainId.compare` now returns `int`, which is what Belt's `cmp` wants — the only caller was undoing a float. Naming: a `ChainId.t` is now called `chainId` everywhere internal. `Internal.item` spelled its field `chain` while `onBlockRegistration` in the same file spelled it `chainId`, and `getItemChainId` existed to bridge the two; sources, IndexerState, ChainFetching and CrossChainState each picked their own. `IndexerState.chain`, an alias for `ChainId.t`, is gone. `context.chain` and `Config.chain` are untouched — the first is the handler-facing API, the second is a record, not an id. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECqzHdVS4cLxfCXPw8zpb6
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cbba022b6c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ~pgSchema, | ||
| ~table, | ||
| ~itemSchema=itemSchema->S.toUnknown, | ||
| ~chainIdMode, |
There was a problem hiding this comment.
When the same Node process writes through an Int32 storage and later reinitializes the same schema/table in Int64 mode, setQueryCache is still keyed only by the module-level table object. Because chainIdMode is consulted only on a cache miss, the second write reuses the first query's INTEGER[] cast; a wide value such as 2494104990 is then rejected by PostgreSQL as out of range instead of being written to the new BIGINT column. Include the mode in the cache key or maintain separate cached queries per mode.
Useful? React with 👍 / 👎.
Int32 is what every config predating the field implies, so serializing it changes the public config JSON — and with it the persisted envio_info fingerprint — for every existing small-id project, which would demand a reindex on upgrade for no reason. Skipping it on the default restores the six config-JSON snapshots to byte-identical with main. The generated ReScript chainId type was covered by a Rust unit test reaching into ProjectTemplate. `from_user_api` now returns the generated Indexer.res alongside the .d.ts it already returned — same parse, same `with_indexer_types` flag — so the assertion lives with the rest of the chain-id coverage in ChainIdMode_test.res and runs against the real NAPI boundary. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECqzHdVS4cLxfCXPw8zpb6
Two defects from review. Mode resolution filtered out skipped chains while codegen emits a `chainId` case for every chain in config.yaml. An active chain 1 beside a skipped 2494104990 resolved Int32 and then emitted `#2494104990` — an int polyvariant ReScript can't represent — and a skipped id above MAX_SAFE_INTEGER skipped validation entirely. Resolution now covers every configured chain, which also keeps the physical column types stable when a chain is skipped and unskipped. `context.chain.id` was `int`, reached through an identity cast from the float-backed representation. ReScript ints are 32-bit, so that type was wrong for exactly the ids this branch adds support for — a handler can't even write `chain.id == 2494104990`, the literal is out of range. The generated Indexer.res now declares its own `handlerChain` using the generated `chainId`, matching `contractRegisterChain` and `indexerChain`, and `Internal.chainInfo` goes back to the internal `ChainId.t`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECqzHdVS4cLxfCXPw8zpb6
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a8dd631c2e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // Escapes for the boundaries that stay `int`: the handler-facing | ||
| // `context.chain.id` / `Envio.effectChain.id`, and int literals in configs and | ||
| // tests. Both are the identity at runtime — an `int` is already a JS number. |
There was a problem hiding this comment.
For an Int64 config, generated handlers expose chain IDs as ChainId.t, but this boundary deliberately remains int and Envio.effectChain is still declared as {id: int} in packages/envio/src/Envio.res:194. Consequently, a chain-scoped effect cannot pass or compare context.chain.id with generated indexer.chainIds or use ChainId APIs without an explicit unsafe identity conversion, so the ReScript API remains int32-oriented for precisely the wide-chain scenario this change supports. Change effectChain.id to ChainId.t instead of retaining the int boundary.
Useful? React with 👍 / 👎.
| // with a wider id falls back to the opaque runtime representation. | ||
| // TypeScript keeps its numeric literal union either way. | ||
| let chain_id_type = match cfg.chain_id_mode { | ||
| ChainIdMode::Int64 => "type chainId = ChainId.t".to_string(), |
There was a problem hiding this comment.
When envio contract-import generates ReScript handlers for an Int64 project, event.chainId now has this opaque ChainId.t type, but Event::get_entity_id_code in packages/cli/src/hbs_templating/contract_import_templates.rs:546 still emits (event.chainId :> int)->Belt.Int.toString. ReScript rejects that generated code because ChainId.t is not a subtype of int, so contract import produces a project that cannot compile for any wide chain. Generate the entity ID with ChainId.toString in Int64 mode, or use a representation-independent conversion.
Useful? React with 👍 / 👎.
…tart-at-head-recovery The base picked up main, bringing configurable chain-id storage (#1507) and generic mock indexer entity queries (#1509). One conflict, in makeSetReadyAtQuery: #1507 turned it into a one-row-at-a-time update because the id column is INTEGER or BIGINT depending on ChainId.mode, so `= ANY($2::int[])` no longer holds for every configuration. Took that shape and kept the `IS NULL` guard on top; the caller loops the chain ids, so the guard applies per row exactly as it did to the array form. Source.make's `~chain` became `~chainId` in #1509; renamed at the call sites in the resume tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DL4RDYUpZcK2YyydfSvqLc
Reconciles main's ChainId.t migration (#1507) with the address-store work. Source options keep main's `chainId: ChainId.t` alongside this branch's `addressStore` handle, and SimulateSource takes both — the branch builds it from ChainState (via SimulateSourceConfig) so it can pass the store, rather than from SimulateItems as main still does. Fetch-state fixtures take main's ChainId.fromInt call sites and drop the removed `contractConfigs` field. ClientAddressFilter_test keeps this branch's trimmed form: main's buildAddressFilterBody / filterByClientAddress cases cover a mechanism this branch replaces with the Rust routing gate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011kjxcBCs4nHF5FNNB8FaEu
2 participants
Summary
Add support for blockchain networks with chain IDs above the int32 maximum (2,147,483,647), such as Tron Shasta (2,494,104,990). The system now automatically detects the widest chain ID in the config and selects appropriate storage modes (int32 or int64) for database columns and generated code.
Key Changes
ChainId module (
packages/envio/src/ChainId.res): New runtime representation for chain IDs as floats (JS numbers) with amodetype (Int32|Int64) that tracks the widest scalar needed. Includes validation againstNumber.MAX_SAFE_INTEGER(9,007,199,254,740,991) and schema for parsing BIGINT columns returned as strings from PostgreSQL.Config resolution (
packages/cli/src/config_parsing/system_config.rs):ChainIdMode::resolve()inspects all active chains and selectsInt32if all IDs fit in i32, otherwiseInt64. Rejects IDs above MAX_SAFE_INTEGER. The mode is serialized into the public config so resume operations against incompatible schemas are rejected.Database schema generation:
Table.res: NewChainIdfield type that resolves toIntegerorBigIntbased on modePgStorage.res: All table creation and insert queries now accept~chainIdModeparameterClickHouse.res: Chain ID columns map toInt32orUInt64based on modeInternalTable.res: Chains and Checkpoints tables useChainIdfield type; address/event reads normalize BIGINT strings throughChainId.normalizeOrThrowGenerated indexer code (
packages/cli/src/hbs_templating/codegen_templates.rs):#chainId) with exhaustive pattern matching ingetChainByIdChainId.t(JS number) since ReScript polyvariants are int32-bound;getChainByIdlooks up chains by stringified ID on the recordConfig compatibility (
Config.res): AddedchainIdModefield to public config;diffPathsandthrowIfIncompatibledetect mode mismatches and reject resume with standard incompatible-config error.Tests (
packages/envio-tests/test/lib_tests/ChainIdMode_test.res): Comprehensive test suite covering mode resolution, schema generation for Postgres/ClickHouse, runtime representation, TypeScript surface, and config compatibility checks.Notable Implementation Details
intto avoid truncation. TheChainId.intSchemaallows modules still typed asintto work safely since chain IDs are only compared and stringified, never used in int32 arithmetic.ChainId.normalizeOrThrowvalidates and converts them to the runtime representation.chainIdModefield default toInt32(all IDs they can express fit in INTEGER).https://claude.ai/code/session_01ECqzHdVS4cLxfCXPw8zpb6
Summary by CodeRabbit
chainIdModeto the public indexer configuration to control 32-bit vs 64-bit chain ID handling.chain_idpersistence now adapt to the selected mode (Postgres + ClickHouse).indexerCode).chainIdModeto prevent incorrect restarts.