Add append-only double-entry ledger (monitoring-only)#3880
Merged
Conversation
4 tasks
TaprootFreak
force-pushed
the
feat/accounting-ledger
branch
from
9b07c3f to
f027f60
Compare
- new ledger_account/ledger_tx/ledger_leg tables (new-tables-only migration, integer-cent balance check, append-only with explicit reversals) - chart-of-accounts bootstrap and idempotent booking engine with consumers for bank, exchange, payin, payout, buy/sell, liquidity and trading sources - valuation from already-persisted CHF fields and snapshot marks, no new external or pricing calls (static isolation gate + DB write isolation test) - one-time cutover opening, daily mark-to-market and per-account reconciliation with staleness guards, transit-age and suspense alarms - read-only ADMIN endpoints for account balances, ledger detail, reconciliation status, suspense and realized-margin reporting - all crons gated by dedicated process flags (kill switch); existing financial log and safety module remain unchanged and authoritative
- bound cutover opening queries with lookback window and settled filters - correct check-constraint name to match TypeORM naming strategy - paginate mark-to-market candidate selection across all open accounts - align cutover watermark init with per-consumer settled semantics - harden isolation gates and fix vacuous setting stub in isolation test
- drop unused imports in cutover spec, use synthetic source id in mapper spec - add multi-page pagination test for mark-to-market candidate selection - scope isolation-gate allowlist marker to write blocks only, with self-test
- add controller, module-compile, DTO, repository and entity-relation tests (structural files to 100%) - cover consumer and service error/fallback/skip branches; module branch coverage 81.5% -> 97%, 619 tests - replace 4 weak tests with real behaviour assertions (fee amount/account, dex txId filter, bootstrap idempotency re-run, equity-parity arithmetic) - fix: openUnattributed passed -0 instead of undefined amountChf for needsMark opening legs (found via coverage)
…spec Satisfies @typescript-eslint no-unsafe-function-type (CI lint).
…reation race-safe - ledger_leg.amountChfCents / ledger_tx.amountChfSum become bigint: the int4 ceiling (~21.4m CHF per leg) would permanently stall a source chain on the first oversized transfer. A fail-loud transformer converts the PG string back to number and throws beyond Number.MAX_SAFE_INTEGER instead of rounding. - findOrCreate now treats a concurrent UNIQUE(name) violation (23505) as the documented no-op by re-fetching the winning row; any other error propagates.
…oundary Card buy-crypto rows are excluded from the per-row cutover opening (their seq0 is owned by the forward consumer), and the consumer card gate now blocks seq0/seq1 for rows whose immutable outputDate shows they had settled at the cutover snapshot - their value is already inside the aggregate opening. The cutover snapshot date is pinned as a setting so the gate has a stable anchor.
…ix reconciliation units - checkTransitAge joined only leg.account but selected MIN(leg.bookingDate), a column that lives on ledger_tx - every reconciliation run would crash on real Postgres. Now joins leg.tx and selects MIN(tx.bookingDate); the spec asserts the exact join/select so a regression fails. - checkEquityParity was log-only. It now alarms (new LEDGER_EQUITY_PARITY mail context) above a runtime threshold, compares against the median of the last 5 valid FinancialDataLog snapshots (immune to transient snapshot skew) and folds the open referral-credit liability in explicitly, logging every component so the difference stays attributable. - Per-asset reconciliation compared a NATIVE quantity diff against a CHF-denominated tolerance; the diff is now marked to CHF first, and assets without a mark are reported as unverified instead of silently passing. - getEquityComparison required no lower bound and re-read feeds per period; from is now mandatory (92-day cap) and the per-day aggregates collapse into one grouped query.
…eGuard develop changed RoleGuardClass to hold an entryRoles array (multi-role OR support); the spec still probed the removed entryRole field.
TaprootFreak
force-pushed
the
feat/accounting-ledger
branch
from
0552fc9 to
b673ec2
Compare
…tency, conventions) - buy-fiat scans now load fiatOutput.bank.asset - the settlement leg read it unloaded and threw on the first tracked-bank row, wedging the consumer - content-change scans preload marks with a 2-day lookback; the zero-width window left every non-CHF row unmarkable and stalled the cursor - liquidity-order-dex sourceId includes the row id: (context, correlationId) is not unique, so a second settled swap was silently dropped - conventions: English-only comments, PascalCase enum values, logger colon format, shared cent transformer, no redundant try/catch in cron runners
…walk, recon relations) - forward scans preload marks with the same 2-day lookback as the content-change path; the earliest batch row previously fell exactly on the window start and could never be marked, wedging the consumer - activeTx walks the reversal chain via the loaded reversalOfId scalar and requires the re-book at reversal.seq + 1; the unloaded relation made the null-check vacuous and a seq gap could pick a foreign tx as active - reconcileAssets loads asset.bank so custody classification stops falling back to the on-chain default for every account (bank accounts were marked stale over weekends and their diff reconciliation silently skipped) - conventions: import order, section comment style, dead snapshot parameter, bank lookup and native-balance aggregation without N+1
…ic rebook, dto enums) - receivedOpened now recognizes bank-funded buy-cryptos (gates on the bank_tx opening booking); a post-cutover bank-funded completion was silently skipped in the content-change scan and wedged the forward scan - the content-change callback no longer ignores a gate-blocked completion - it throws like buy-fiat so the cursor retries instead of skipping the row - reverseAndRebookIfChanged wraps reversal and re-book in one transaction; a crash between the two left a flat-reversal state that unique-collided on every retry - admin recon status marks the native diff to CHF before the tolerance check, matching the reconciliation job; feedless accounts report as unverified - conventions: DTO status unions become PascalCase enums (endpoints ship with this PR, no consumer exists), cutover import order fixed - test substance: exact spread-leg amounts, real re-run idempotency case, discriminating ambiguous-sweep assertions
…books, cutover fail-loud) - every status-filtered source now has content-change coverage: a row the forward watermark skipped while unsettled is forward-booked idempotently once it settles (payout, trading, liq-mgmt, dex; crypto-input and exchange-tx callbacks now forward-book instead of only reversing) - gate-blocked rows distinguish never-bookable (settled before the cutover snapshot - value lives in the aggregate opening - skip and advance) from not-yet-bookable (throw and retry); a flag bump on an old row no longer wedges the scan - value-coupled seq chains reverse and re-book as one interleaved transaction, so a late amount change cannot leave the liability open - cutover openings and owedReferenceRate fail loud on missing marks instead of booking silent zeros; dead injection and any-casts removed
… conventions) - openManualDebt and openBuyFiatOwed abort the cutover run loudly on a missing mark instead of silently dropping the CHF value or skipping the opening; already-booked openings are skipped idempotently on the retry - buy-crypto recognizes the cutover owed-opening marker: a row whose owed was opened at the cutover skips seq1 (and the card seq0) instead of wedging on the received gate or double-counting the card inflow - conventions: existsBy for existence checks, drop redundant eager:false, spell out LedgerBookingLiquidityManagement, move the valid-financial-log query into the repository, capitalize client-facing error messages
…ard, conventions) - every asset consumer now carries the settled-before-cutover guard, not just buy-crypto/buy-fiat: a pre-cutover-settled row re-selected by an updated bump can no longer re-book its seq0 on top of the aggregate opening (wallet double-count + phantom liability); the cutover persists a per-source boundary id as the durable signal - conventions: English-only comment leftovers, unused logger, capitalized plain-prose error messages, NotFoundException for unknown accounts, import order, Promise.all for independent reads, single ORed query for the unattributed opening
…ut native units, custody classes) - the cutover pins boundary ids and hole ids set-once right after pinning the snapshot, before any opening: a fail-loud retry hours later can no longer widen the settled-classification window and double-book rows whose updated was bumped in between; the remaining seconds-wide residual is documented instead of claimed impossible - Checkout custody accounts carry one native convention (gross in card currency, CHF mark-consistent): the card seq0 books inputReferenceAmount and the CheckoutLtd settlement books gross native, so mark-to-market no longer flushes the acquirer fee as phantom fx pnl double-counted in the margin report - custody classification uses the objective blockchain lists (XT/MEXC were falling back to on-chain thresholds, frozen bank assets to 24h): BankDead maps to OlkyFrozen; the unreachable order-driven/feedless exchange classes are removed with their thresholds
- ladder 2 requires an explicit non-null charge currency matching the custody account unit (a null==null match could smuggle an unverified unit in); mismatches fall through to the mark bridge - document that a fee not yet CHF-priced books net-as-gross temporarily and self-heals via the content-change rebook once priced - mid-loop resume test: a crash between boundary write and watermark write re-derives the watermark from the pinned boundary, never a fresh MAX(id)
…idge, stable fill ranking) - open-at-cutover card rows get a per-row received opening: the Checkout collateral feed already carries their gross, so the forward seq0 gates on the cutover marker instead of double-booking onto the aggregate opening - asset openings book under per-asset source ids; a retry after a partial crash can no longer shift sequence slots when the chart of accounts grew in between - exchange trade source ids carry the venue prefix and the fill ranking is status-agnostic, so an ok-to-failed flip neither collides nor reshuffles surviving fills - mixed transactions with a missing historical mark value the open leg at the latest available mark (warn-logged, needsMark kept) or defer the row loudly - bookTx never receives an unbalanceable leg set and the head of the line can no longer wedge - fail-loud guards for null trade cost, null output amounts and CHF-derived forward fees; isolation gate covers upsert/soft deletes and fails closed on an empty scan
… the plug, transit age) - checkoutLtdLegs returns through withFxPlug so a missing historical mark bridges the two asset legs instead of throwing a CHF imbalance that wedged the whole bank_tx forward scan - the dex network-fee expense leg derives its CHF from the same bridged mark as its asset counter-leg, keeping mixed swaps balanced - transit-age alarms measure since the balance last left zero instead of the first booking ever, so churning routes stop reporting fresh residuals as ancient
… pinning, bridge routing) - straddling paymentLink rows open per-row on LIABILITY/paymentLink and the completion gates accept that marker; they no longer wedge the scan or land in the wrong bucket - rows open and unpriced at the cutover are pinned: the scans skip them with an alarm instead of throwing forever, and the card seq0 cannot re-book gross that the collateral feed already carried into the aggregate opening - paymentLink-funded buy-cryptos clear LIABILITY/paymentLink instead of letting both liabilities drift equity-neutrally - the Scrypt trade branch, payment seq0 and payout liability counter route through the mark bridge: no leg set reaches bookTx unbalanceable and no unvalued leg lands on a CHF-denominated liability
…usive with input seq0) An open buy row whose funding crypto input had not settled at the snapshot no longer gets a per-row received/paymentLink opening: the input's value is not in the aggregate asset opening yet, so the forward crypto_input seq0 is the single opener and the pair would have double-credited the liability. Settled inputs, card and bank funded rows open per-row as before; a counter-check confirms the new tests fail without the guard.
…ned boundary) The received/paymentLink per-row cutover opening now keys on the same immutable at-snapshot boundary as the forward seq0 suppression instead of the mutable live status: an input settling inside the retry window can no longer be opened per-row AND booked by its forward seq0. The opening is the exact complement of the suppression by construction.
…ary complement) openBuyFiatOwed now carries the same pinned-boundary coverage guard as the received openings, so its paymentLink and owed per-row openings can no longer pair with a forward seq0 of an input that settled inside the retry window; stale comments describing the old live-status logic rewritten.
…ndary complement) openBuyCryptoOwed carries the same pinned-boundary coverage guard as its three siblings, closing the last opener/suppression divergence cell in the snapshot-to-pin residual window; the buy-fiat owed relations comment now documents the coverage read.
Collaborator
Author
|
Rebased onto current develop and hardened through 15 full review passes (correctness + conventions) until both review tracks came back clean. Highlights of what the passes surfaced and fixed:
Ledger test suite grew from 332 to 745+ assertions across 34 suites; full jest, type-check, lint and format gates green on every push. |
1 task
Danswar
added a commit
that referenced
this pull request
Jul 15, 2026
… break circular dependency (#4220) AccountingModule (#3880) is now the first module in core.module.ts to require LiquidityManagementModule, which transitively re-enters ReferralModule (via BankModule -> UserModule) before finishing its own export. ReferralModule's own unwrapped LiquidityManagementModule import then captures undefined, crashing Nest bootstrap ("Nest cannot create the ReferralModule instance") on every restart in DEV.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Introduces a pure-observer double-entry journal as a reconciliation source of truth for all value movements.
ledger_account/ledger_tx/ledger_legtables — the migration creates new tables only, no ALTER or data change on any existing tableTest plan
npm run format:check,npm run type-check,npm run lintclean