Skip to content

Entity updates broken since es2022 target bump: nightly onchain sync aborts on dup-key, transaction_lightning silently accumulates duplicate rows #205

Description

@Danswar

Symptom

Since 2025-11-11, the nightly onchain sync (processSyncOnchainTransactions, 02:00 UTC) fails every night with:

duplicate key value violates unique constraint "UQ_8a233973c49afbbc14a01ce076b"

The count is suspiciously constant — exactly 55 duplicate-key errors per night on PRD, 3 on DEV, every single night. That constant is the tell: 55/3 is the entire row count of transaction_onchain in each environment. The job collides with every row it just loaded, Promise.all rejects, the @Lock() wrapper logs Error during processSyncOnchainTransactions, and the job aborts. There are zero syncOnchainTransactions: runtime= success lines in the logs — the job has effectively never completed since November.

Root cause

Commit e18f05a (PR #84, 2025-11-11) bumped tsconfig.json target from es2017 to es2022:

-    "target": "es2017",
+    "target": "es2022",

With target ≥ es2022, TypeScript's useDefineForClassFields default flips to true. Class-field declarations (like the entity base class's id) then become own properties initialized to undefined on every instance — including entities produced by repo.create({...}).

The update helpers merge a freshly-created entity into the loaded DB row with Object.assign:

private async doUpdateOnchainTransaction(
updateOnchainTransactionEntity: TransactionOnchainEntity,
): Promise<TransactionOnchainEntity> {
let dbOnchainTransactionEntity = await this.transactionOnchainRepo.findOneBy({
transaction: Equal(updateOnchainTransactionEntity.transaction),
});
if (!dbOnchainTransactionEntity) {
dbOnchainTransactionEntity = updateOnchainTransactionEntity;
} else {
Object.assign(dbOnchainTransactionEntity, updateOnchainTransactionEntity);
}
return this.transactionOnchainRepo.save(dbOnchainTransactionEntity);
}

let dbOnchainTransactionEntity = await this.transactionOnchainRepo.findOneBy({
  transaction: Equal(updateOnchainTransactionEntity.transaction),
});

if (!dbOnchainTransactionEntity) {
  dbOnchainTransactionEntity = updateOnchainTransactionEntity;
} else {
  Object.assign(dbOnchainTransactionEntity, updateOnchainTransactionEntity);  // copies id: undefined!
}

return this.transactionOnchainRepo.save(dbOnchainTransactionEntity);

Pre-es2022, the fresh entity had no own id property, so Object.assign left the loaded row's id intact and save() issued an UPDATE. Post-es2022, the fresh entity carries id: undefined as an own property, Object.assign wipes the loaded row's id, and save() issues an INSERT — colliding with the very row the code just found.

The worse, silent half

transaction_onchain has UNIQUE ("transaction") (migration 1697006086810-Transactions.js), so its corruption is loud — the INSERT fails.

transaction_lightning has no unique constraint on (type, transaction) — only the PK. The identical pattern in doUpdateLightningTransaction:

private async doUpdateLightningTransaction(
updateTransactionLightningEntity: TransactionLightningEntity,
): Promise<TransactionLightningEntity> {
let dbTransactionLightningEntity = await this.transactionLightningRepo.findOneBy({
type: Equal(updateTransactionLightningEntity.type),
transaction: Equal(updateTransactionLightningEntity.transaction),
});
if (!dbTransactionLightningEntity) {
dbTransactionLightningEntity = updateTransactionLightningEntity;
} else {
Object.assign(dbTransactionLightningEntity, updateTransactionLightningEntity);
}
return this.transactionLightningRepo.save(dbTransactionLightningEntity);
}

…therefore succeeds in inserting duplicates. Consequences, ongoing since November 2025:

  • The 03:00 UTC processSyncLightningTransactions re-inserts its trailing-window candidate set as new rows every night (its logged entries= count has been growing, e.g. 369 → 1250/night over one recent week).
  • The websocket live-update path turns every state transition (e.g. invoice OPEN → SETTLED) into a new row instead of an update, so a single invoice exists multiple times in different states.

Same mechanism likely affects the other Object.assign-onto-loaded-entity sites: user-boltcard.service.ts:62 and lightning-wallet.service.ts:286 — worth auditing all of them in the fix.

Note user_transaction.lightningTransactionId has an FK onto transaction_lightning.id, so any dedupe cleanup has to remap references, not just delete rows.

Suggested fix

Two options, not mutually exclusive:

  1. "useDefineForClassFields": false in tsconfig — one line, restores the pre-November semantics for the entire codebase and kills the whole bug class at once.
  2. Replace the Object.assign(dbEntity, freshEntity) merge sites with an id-safe merge (repo.merge() with an explicit field list, repo.update(id, partial), or explicit per-field assignment).

Option 1 is the smallest safe change; option 2 is more future-proof if a later target bump is ever wanted. Either way a regression test pinning "update does not change the row's id / does not insert a second row" would prevent a silent re-flip.

Optional hardening while in there: add the missing UNIQUE (type, transaction) constraint on transaction_lightning after the cleanup below, so this failure mode can never again be silent.

Follow-up: data cleanup (separate task)

After the code fix, transaction_lightning needs a dedupe pass (keep the newest row per (type, transaction), remap user_transaction.lightningTransactionId FKs to the surviving row, delete the rest) and the onchain sync needs one clean full run to verify. Sizing the duplicate population first (SELECT type, transaction, COUNT(*) ... GROUP BY ... HAVING COUNT(*) > 1) would tell us how big that task is.

Verification after deploy

  • 02:00 UTC job: Error during processSyncOnchainTransactions disappears; a syncOnchainTransactions: runtime= success line appears.
  • Zero UQ_8a233973 duplicate-key lines in the DB logs.
  • transaction_lightning row count stops growing at the nightly sync rate; invoice state transitions update rows in place.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions