chore: prepare 2.3.0#755
Draft
matiwinnetou wants to merge 39 commits into
Draft
Conversation
* chore: migrate to JDK 25 LTS and update dependencies Migrate the project from JDK 24 to JDK 25 LTS and update all stable dependencies across the multi-module build. JDK 25 migration required adapting the StructuredTaskScope preview API (ShutdownOnFailure replaced by Joiner-based open() API) in both LedgerBlockServiceImpl and AddressHistoryServiceHibernate. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: use explicit types instead of var for StructuredTaskScope Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Mateusz Czeladka <mateusz.czeladka@cardanofoundation.org> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
💥 Preprod Tests: DEPLOYMENT FAILEDTests run against preprod network with live blockchain data |
feat: Spike: replace 'apply indexes' with a native solution
update code
update code
update code
- Replace fail-fast `return` with `failedCount` counter so all 16 indexes
are attempted even when one fails; overall state is READY only if
failedCount == 0, otherwise FAILED (Tier 1.1)
- Resolve current schema at init() and qualify all DROP INDEX statements
as `<schema>.<index>` to avoid cross-schema matches (Tier 1.3)
- Add guardrail comment: executeIndexCreation always re-queries pg_index
per-index so any future retry path inherits drop-before-rebuild (Tier 1.2)
- Update tests: mock current_schema(), fix DROP expectations, add
continuesAfterSingleFailure test verifying both CREATE calls fire
despite first failure
…exes natively Deletes all artifacts that existed solely to support the external index-applier container: the Docker Compose file, the apply-indexes.sh shell script, the Helm Job template, the ConfigMap template and its bundled db-indexes.yaml, and the two dead _helpers.tpl helpers (indexApplierName / dbIndexesName). Also removes jq/yq from the Postgres Dockerfile (only needed for apply-indexes.sh YAML parsing) and drops the indexApplier resource blocks from all three Helm hardware profiles and the top-level indexApplier values section. Documentation updated across boot-sequence, docker, kubernetes deployment/helm-values/overview, and index-management pages to reflect the native yaci-indexer approach. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
❌ Preprod Tests: FAILEDTests run against preprod network with live blockchain data |
Contributor
✅ Preprod Tests: PASSEDTests run against preprod network with live blockchain data |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
✅ Preprod Tests: PASSEDTests run against preprod network with live blockchain data |
#752) * fix(search-service): reject lovelace as currency symbol in /search/transactions * test(data-endpoints): reject lovelace currency in search transactions --------- Co-authored-by: Sotatek-DucPhung <duc.phung@sotatek.com> Co-authored-by: Lincon Vidal <lincon.vidal@cardanofoundation.org>
* fix: validation for payload endpoints max_val_size fix: validation for payload endpoints max_val_size * update code update code * update code update code * fix: add error 5061 to network options golden example snapshot * test: add golden example test cases for max val size validation * test: update max_val_size skip test cases --------- Co-authored-by: Sotatek-DucPhung <duc.phung@sotatek.com> Co-authored-by: matiwinnetou <mateusz.szczap@gmail.com> Co-authored-by: Sotatek-HenryDo <henry.do2@sotatek.com>
Contributor
❌ Preprod Tests: FAILEDTests run against preprod network with live blockchain data |
Contributor
✅ Preprod Tests: PASSEDTests run against preprod network with live blockchain data |
…ion (#735) * fix: payloads endpoint be validate min ada value during txn construction fix: payloads endpoint be validate min ada value during txn construction * update code update code * test: fix broken unit tests and add min ADA validation test coverage * test: update payloads golden examples to pass min ADA validation * test: add error code 5060 to network/options golden example * test: add golden example test cases for the min ADA validation * test: fix pool registration golden example format (cold key hash, stake_test1, relay type) --------- Co-authored-by: Sotatek-DucPhung <duc.phung@sotatek.com> Co-authored-by: Sotatek-HenryDo <henry.do2@sotatek.com>
Contributor
✅ Preprod Tests: PASSEDTests run against preprod network with live blockchain data |
#751) * fix(token-registry): convert CIP-26 logo from hex to base64 before returning * test: update behavioral tests for hex-to-base64 logo conversion * test: add positive and negative integration test cases for CIP-26 logos --------- Co-authored-by: Sotatek-DucPhung <duc.phung@sotatek.com> Co-authored-by: Sotatek-HenryDo <henry.do2@sotatek.com>
Contributor
✅ Preprod Tests: PASSEDTests run against preprod network with live blockchain data |
Contributor
❌ Preprod Tests: FAILEDTests run against preprod network with live blockchain data |
Contributor
✅ Preprod Tests: PASSEDTests run against preprod network with live blockchain data |
# feat: replace 'apply indexes' with a native solution Closes #736. Moves the required Rosetta index lifecycle (create, validate, track progress) from the external `index-applier` sidecar container into `yaci-indexer` itself. After blockchain sync reaches tip, the indexer automatically creates all 16 Rosetta-required indexes, gates readiness on completion, and exposes per-index lifecycle state via a new actuator endpoint. --- ## Architecture: Before vs After ``` BEFORE ────────────────────────────────────────────────────────────── ┌─────────────────┐ ┌──────────────────┐ │ yaci-indexer │ │ index-applier │ │ │ │ (sidecar) │ │ sync chain │ │ │ │ expose data │ │ apply-indexes │ │ │ │ .sh │ └─────────────────┘ └──────────────────┘ │ │ │ │ psql + shell loop └──────────┬─────────────┘ ▼ PostgreSQL DB (16 indexes) AFTER ────────────────────────────────────────────────────────────── ┌───────────────────────────────────────────┐ │ yaci-indexer │ │ │ │ sync chain │ │ expose data │ │ PgIndexService │ │ └─ creates indexes when synced │ │ └─ gates readiness probe │ │ └─ exposes /actuator/rosetta-indexes │ └───────────────────────────────────────────┘ │ │ JDBC (autoCommit=true) ▼ PostgreSQL DB (16 indexes) ``` --- ## Lifecycle State Machine ```mermaid stateDiagram-v2 [*] --> PENDING : startup (init) PENDING --> PENDING : sync not yet at tip PENDING --> APPLYING : node reaches tip\n(compareAndSet guard) APPLYING --> READY : all 16 indexes created\nsuccessfully APPLYING --> FAILED : ≥1 index failed\n(remaining still attempted) READY --> [*] FAILED --> [*] note right of PENDING Re-queries pg_index on startup. Skips to READY if all indexes already exist and are valid. end note note right of APPLYING Single background thread. Continue-on-failure: all 16 indexes are always attempted. end note note right of FAILED Per-index errorMessage available via /actuator/rosetta-indexes. Recovery requires pod restart. end note ``` --- ## Index Creation Flow ```mermaid flowchart TD A([triggerIndexing called]) --> B{state ==\nPENDING?} B -- No --> Z([return — no-op]) B -- Yes --> C[CAS: PENDING → APPLYING] C --> D[submit to RosettaIndexBuilder thread] D --> E[for each of 16 indexes] E --> F[queryIndexState from pg_index\nschema-filtered] F --> G{state?} G -- READY --> N[skip — already done] G -- INVALID or BUILDING --> H[DROP INDEX IF EXISTS\nschema.indexname] G -- MISSING --> I[CREATE INDEX CONCURRENTLY\nautoCommit=true] H --> I I --> J{success?} J -- Yes --> K[mark READY\nupdate lastProgressAt] J -- No --> L[mark FAILED\nlog error\nfailedCount++] K --> M{more indexes?} L --> M N --> M M -- Yes --> E M -- No --> O{failedCount == 0?} O -- Yes --> P[state = READY] O -- No --> Q[state = FAILED\nlog warning] ``` --- ## Health Probe Behaviour ```mermaid flowchart LR subgraph Readiness [Readiness Probe — /actuator/health/readiness] R1{connection\nalive?} -- No --> RD1[DOWN\nConnection] R1 -- Yes --> R2{synced\nto tip?} R2 -- No --> RD2[DOWN\nSyncing] R2 -- Yes --> R3{index lifecycle\n== READY?} R3 -- No --> RD3[DOWN\nApplying Indexes] R3 -- Yes --> RU[UP] end subgraph Liveness [Liveness Probe — /actuator/health/liveness] L1{state ==\nAPPLYING?} -- No --> LU[UP] L1 -- Yes --> L2{lastProgressAt\n< 15 min ago?} L2 -- Yes --> LU L2 -- No --> LD[DOWN\nStalled] end ``` > **Note**: FAILED state → Readiness DOWN, Liveness **UP**. This is intentional — > a stalled build warrants a pod restart (liveness kill), but a permanently failed > index does not. Operators diagnose via `/actuator/rosetta-indexes`. --- ## Actuator Endpoint `GET /actuator/rosetta-indexes` ```json { "overallState": "APPLYING", "lastProgressAt": "2026-04-22T10:31:05Z", "totalRequired": 16, "totalReady": 9, "totalMissing": 6, "totalFailed": 1, "indexes": [ { "name": "idx_address_utxo_amounts_gin", "state": "READY", "errorMessage": null }, { "name": "idx_tx_input_tx_hash", "state": "BUILDING", "errorMessage": null }, { "name": "idx_block_hash", "state": "FAILED", "errorMessage": "permission denied for table block" }, { "name": "idx_stake_address_hash_raw", "state": "MISSING", "errorMessage": null } ] } ``` --- ## Key Implementation Details ### Why `autoCommit=true`? `CREATE INDEX CONCURRENTLY` is rejected by PostgreSQL inside a transaction block. The service uses a raw JDBC connection with explicit `autoCommit=true` to guarantee no transaction wrapping. ### Why continue-on-failure? The 16 indexes span independent tables. All indexes are always attempted so a single permission error on one table does not block the remaining 15. Per-index failure detail is available via `/actuator/rosetta-indexes`. ### Schema-qualified DROP Both `pg_index` queries and `DROP INDEX` statements are scoped to `current_schema()` to avoid cross-schema matches in multi-schema PostgreSQL deployments. --- ## Infrastructure Changes | Area | Before | After | |------|--------|-------| | Docker Compose | `index-applier` sidecar container | Removed — `yaci-indexer` handles indexes natively | | `apply-indexes.sh` | Shell script (psql + yq loop) | Deleted | | Postgres Dockerfile | Bundled `jq`, `yq`, `apply-indexes.sh` | Removed (no longer needed) | | Helm chart | `index-applier` Job + ConfigMap templates | Removed — `indexApplier` section dropped | | Helm profile resources | `indexApplier` blocks in entry/mid/advanced | Removed | | Healthcheck retries | 360 × 10 s = 1 hour | 1080 × 10 s = **3 hours** (mainnet GIN build time) | | Documentation | References to `index-applier` throughout | Updated to reflect native approach | --- ## Package & Class Naming (yaci-indexer) Classes were renamed for clarity as part of this PR: | Old | New | |-----|-----| | `indexmanagement` package | `indexes` package | | `RosettaIndexLifecycleService` | `IndexService` | | `PostgreSQLRosettaIndexLifecycleService` | `PgIndexService` | | `NoOpRosettaIndexLifecycleService` | `NoOpIndexService` | | `RosettaIndexConfig` | `IndexCatalog` | | `RosettaIndexesEndpoint` | `IndexesEndpoint` | | `RosettaIndexProgressSnapshot` | `IndexProgress` | | `RosettaIndexStallIndicator` | `IndexStallIndicator` | --- ## API module — cleanup Removed dead `getIndexCommands()` from `RosettaIndexConfig` in the api module. The api never creates indexes — that is `PgIndexService`'s responsibility. The method was unused and has been deleted. --- ## Test Coverage 39 unit tests across 5 classes: | Class | Scenarios | |-------|-----------| | `PgIndexServiceTest` | Init states, sync trigger, INVALID drop, BUILDING drop, continue-on-failure, failure handling, double-trigger guard | | `IndexesEndpointTest` | APPLYING snapshot, READY (all pre-exist) snapshot | | `IndexStallIndicatorTest` | Stall detection, terminal state distinction | | `YaciSyncHealthIndicatorTest` | Readiness states including APPLYING_INDEXES | | `IndexCatalogParityTest` | Both module YAML files are byte-identical | > `CREATE INDEX CONCURRENTLY` with `autoCommit=true` must be verified against a real > PostgreSQL instance — unit mocks cannot catch transaction-block rejections.
Contributor
✅ Preprod Tests: PASSEDTests run against preprod network with live blockchain data |
* fix: block data endpoints when indexer is syncing * test: add unit and integration tests for blocking data endpoints when syncing * chore: update indexer not ready error message * fix: verify network request before checking sync status * test: fix unit and integration tests setup for sync checks * test: retry retriable indexer readiness errors
Contributor
✅ Preprod Tests: PASSEDTests run against preprod network with live blockchain data |
Contributor
✅ Preprod Tests: PASSEDTests run against preprod network with live blockchain data |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
2.3.0 is a maintenance release for the 2.x line. It merges
developintomain, rolling up bug fixes, dependency upgrades, and tooling/infra improvements. No hard-fork or breaking API changes are expected in this release.Currently part of this PR (already merged to
develop)max_val_size) validation for payload endpoints/search/transactionsTOKEN_REGISTRY_BASE_URLpom.xmlrevision,RELEASE_VERSIONenv files, HelmreleaseVersion, docs, golden examples)Upcoming — to be merged before the release is cut
Open PRs expected to land in
developbefore 2.3.0 is cut: