fix(clock-skew): restrict per-node skew to self-originated adverts (#1816, #1818) - #1820
Conversation
…pa-clawbot#1816, Kpa-clawbot#1818) byNode is an involvement index (indexResolvedPathHops, store.go:1696-1705, Kpa-clawbot#1558/Kpa-clawbot#1352): a transmission is indexed under every relay-hop pubkey found in an observation's resolved_path, not just its originator. getNodeClockSkewLocked iterated all ADVERT transactions under a pubkey without checking who actually signed them, so a relay inherited the skew of every broken-clock node it forwarded as if it were its own. This produced fleet-wide false no_clock/bimodal_clock classifications, bit-identical RecentMedianSkewSec 'clusters' across unrelated relays that forwarded the same broken originator (Kpa-clawbot#1816), and single relays showing a multi-day skew from just 1-2 relayed adverts landing in the tail of their small recent-window sample despite healthy self-adverts (Kpa-clawbot#1818). Add txOriginatedBy(tx, pubkey): ADVERTs are self-signed, so decoded pubKey is the originator per protocol (case-insensitive compare as a defensive measure). Apply it as a guard in both the main skew aggregation loop and the per-hash evidence loop in getNodeClockSkewLocked. byNode itself is untouched; Kpa-clawbot#1558/Kpa-clawbot#1352 still rely on the broader involvement index for other consumers. Existing clock_skew tests built synthetic ADVERT fixtures without a pubKey field and relied on directly seeding s.byNode, bypassing the normal indexByNode path where every real ADVERT carries pubKey. Add pubKey to each fixture so it reflects a self-originated advert, which is what these tests were already meant to represent. Co-Authored-By: Claude <noreply@anthropic.com>
|
Automated polish review — parallel personas (carmack / munger / kent-beck). No blocking issues; minor nits only. Verdict🟢 Merge-worthy on code + tests. CI shows Findings
Nits (non-blocking, optional)
|
Closes #1816. Closes #1818 (confirmed duplicate of #1816 by the triage bot).
Root cause
byNodeis an involvement index (indexResolvedPathHops,store.go:1696-1705, #1558/#1352): a transmission is indexed under every relay-hop pubkey found in an observation'sresolved_path, not just its originator.getNodeClockSkewLocked(clock_skew.go:489) iterated every ADVERT transaction under a pubkey without checking who actually signed it, so a relay inherited the clock skew of every broken-clock node it forwarded as if it were its own.This produced:
no_clock/bimodal_clockclassifications on healthy relays whose only "bad" samples were adverts they merely relayed.RecentMedianSkewSec"clusters" across unrelated relays that all forwarded the same broken-clock originator.Fix
Add
txOriginatedBy(tx, pubkey): ADVERTs are self-signed, sodecoded["pubKey"]is the originator per protocol (case-insensitive compare as a defensive measure). Apply it as a guard in both the main skew-aggregation loop and the per-hash evidence loop ingetNodeClockSkewLocked.byNodeitself is untouched — #1558/#1352 still rely on the broader involvement index for other consumers.Tests
clock_skew_test.go/clock_skew_issue1094_test.go/clock_skew_issue1285_test.gofixtures built synthetic ADVERT transactions without apubKeyfield and seededs.byNodedirectly, bypassing the normalindexByNodepath where every real ADVERT carriespubKey. AddedpubKeyto each fixture so it reflects a self-originated advert, which is what these tests already intended to represent. All pre-existing tests pass unchanged in behavior.clock_skew_issue1816_test.go:TestTxOriginatedBy— unit coverage of the new guard (self, foreign, missing pubKey, case-insensitivity).TestIssue1816_RelayDoesNotInheritOriginatorSkew— a relay with healthy self-adverts plus relayed adverts from a broken-clock originator (matching the report's +100.5k s band) must reportokseverity based only on its own adverts.TestIssue1816_PureRelaysReportNoSkew_NoBitIdenticalCluster— five relay pubkeys that only ever forward a broken originator's advert (never self-advert) must reportnil, not a bit-identical copy of the originator's skew.TestIssue1818_TwoForeignAdvertsDoNotPoisonIslandNode— reproduces the cwichura island scenario: 8 healthy self-adverts + 2 foreign adverts at ~10 days skew must not flip severity or polluteRecentMedianSkewSec.Full suite:
go test ./...passes (one pre-existing, unrelated flaky test —TestHandleNodePaths_PrefixCollision_1352, an index-loading race — reproduces intermittently on unmodifiedmastertoo).Operator context: running CoreScope for SaarMesh (SaarLorLux, DE/FR/LU, 800+ nodes); this bug was surfacing as fleet-wide clock-skew false positives on our infra nodes.
Co-Authored-By: Claude noreply@anthropic.com