Skip to content

fix(audit): hash created_at at the precision Postgres stores#2638

Open
shani-singh1 wants to merge 2 commits into
block:mainfrom
shani-singh1:fix/audit-chain-timestamp-precision
Open

fix(audit): hash created_at at the precision Postgres stores#2638
shani-singh1 wants to merge 2 commits into
block:mainfrom
shani-singh1:fix/audit-chain-timestamp-precision

Conversation

@shani-singh1

Copy link
Copy Markdown

Fixes #2637 — full analysis and reproduction there.

Problem

Audit entries are stamped and hashed with Utc::now() (nanoseconds), then stored in a TIMESTAMPTZ column (microseconds). compute_hash covers created_at.to_rfc3339(), and chrono emits 0/3/6/9 fractional digits depending on the value — so the digest written at service.rs:103 is computed over …T12:00:00.123456789+00:00 while verify_chain recomputes over the …T12:00:00.123456+00:00 that Postgres hands back.

Every hash chain backed by a real database therefore fails verification at its first entry, on untampered data. That is not just a broken feature — it means a genuinely forged row is indistinguishable from the permanent baseline failure, so HashMismatch carries no signal.

It is invisible in CI because all six chain tests are #[ignore = "requires Postgres"], and the in-process hash.rs tests use a fixture timestamp of 2026-01-01T00:00:00Z — zero sub-seconds, the one value where the bug cannot appear.

Solution

Reduce created_at to the stored precision before hashing, so the in-memory entry and the row are byte-identical:

pub fn to_storage_precision(created_at: DateTime<Utc>) -> DateTime<Utc> {
    created_at.trunc_subsecs(6)
}

log_inner is the only place that assigns created_at — every caller goes through NewAuditEntry, which carries no timestamp — so this is a single choke point. It is wrapped in a log_timestamp() helper purely so the invariant is assertable without a database.

I chose truncation at the write path over the alternative (hashing a precision-independent encoding such as timestamp_micros().to_be_bytes()). Both fix the mismatch, but truncating keeps the existing hash preimage format and gives the stronger invariant: the AuditEntry returned from log() is now exactly what a later read returns.

Truncation matches what actually happens on the wire — sqlx encodes DateTime<Utc> as microseconds since the Postgres epoch, truncating — so the value hashed is the value stored.

Validation

Toolchain note: built on Windows with the x86_64-pc-windows-gnu toolchain (no MSVC linker locally).

Before, against Postgres 17 with migrations/* applied:

$ cargo test -p buzz-audit --lib -- --ignored --test-threads=1

test service::tests::chain_links_within_one_community ... FAILED
test service::tests::chains_are_independent_per_community ... FAILED
test service::tests::community_chain_starts_at_seq_1_with_null_prev ... ok
test service::tests::cross_community_row_does_not_verify ... ok
test service::tests::verify_detects_tampering_within_a_community ... FAILED
test service::tests::verify_empty_range_is_false ... ok

test result: FAILED. 3 passed; 3 failed

with HashMismatch { seq: 2 } / HashMismatch { seq: 1 } on untampered chains.

After, same database:

test result: ok. 6 passed; 0 failed

verify_detects_tampering_within_a_community is the one to look at: it asserts HashMismatch lands on the tampered entry's seq. It was failing because verification already blew up on an earlier untampered row — so the assertion proving tamper detection works had never actually been exercised. It passes now.

Also:

  • cargo test -p buzz-audit --lib (no Postgres) — 12 passed, 0 failed.
  • cargo clippy -p buzz-audit --all-targets -- -D warnings — clean.
  • cargo fmt -p buzz-audit -- --check — clean.

New tests

Three in hash.rs, none needing Postgres:

  • storage_precision_drops_sub_microsecond_digits — the helper's contract, and that it is idempotent so a re-read value is unchanged.
  • nanosecond_timestamps_cannot_survive_a_database_round_trip — asserts the digests differ. This is the trap itself, written down so the next person changing the hash preimage sees why the precision reduction is load-bearing.
  • storage_precision_timestamps_survive_a_database_round_trip — the invariant the write path must hold.

Plus log_timestamp_carries_no_sub_microsecond_digits in service.rs, deliberately not #[ignore]d, so a regression on the write path is caught by just test-unit instead of only by Postgres-gated tests that normally never run.

Compatibility

Rows written before this stay unverifiable — they always were — so there is no migration. An operator relying on an existing chain has to re-anchor.

Relationship to #2620

#2620 proposes a shared verify_entries walk (anchoring, seq contiguity, tail-truncation detection) plus a buzz-admin audit verify command. Its Postgres-free unit tests build entries in memory and would pass regardless, but its #[ignore] Postgres tests and the operator command itself would fail on every real chain until this lands. Worth taking this first so that work has a verifiable baseline — the two changes don't overlap in code.

@shani-singh1
shani-singh1 requested a review from a team as a code owner July 23, 2026 22:22
`log_inner` stamped entries with `Utc::now()` (nanoseconds) and hashed the
value via `to_rfc3339()`, whose sub-second digit count follows the value.
`audit_log.created_at` is `TIMESTAMPTZ` — microsecond resolution — so
`verify_chain` recomputed the digest over a different byte string than the
one that was written and returned `HashMismatch` on untampered data.

Every chain backed by a real database failed verification at its first
entry, which also made a genuinely forged row indistinguishable from the
baseline failure.

Reduce `created_at` to the stored precision before hashing, so the in-memory
entry and the row are byte-identical. `log_inner` is the only place that
assigns it — every caller goes through `NewAuditEntry`, which carries no
timestamp — so this is a single choke point.

Tests: three Postgres-free tests in `hash.rs` pinning the invariant, the trap
that caused it, and the round trip; plus a non-ignored `service.rs` test that
the write path's timestamp carries no sub-microsecond digits, so a regression
is caught by `just test-unit` rather than only by the `#[ignore]` chain tests.

Rows written before this stay unverifiable — they always were — so no
migration is needed, but an operator relying on an existing chain has to
re-anchor.

Fixes block#2637

Signed-off-by: Shani Singh <teamdeveloperworld@gmail.com>
@shani-singh1
shani-singh1 force-pushed the fix/audit-chain-timestamp-precision branch from 09d3868 to ebfcb93 Compare July 23, 2026 22:28
@dophsquare

Copy link
Copy Markdown

P0 triage — merge candidate. 🐝

Reviewed the diff. The root cause is real and the fix is correct: Utc::now() yields nanosecond resolution on Linux (clock_gettime), compute_hash covers created_at.to_rfc3339() whose sub-second digit count is value-dependent (chrono emits 0/3/6/9 digits), and Postgres TIMESTAMPTZ stores microseconds. So the write-time preimage ≠ the read-time preimage and verify_chain fails at entry #1 on untampered data — tamper-evidence is effectively off in any real deployment.

to_storage_precisiontrunc_subsecs(6) before hash+store is the right normalization, and it's idempotent so a re-read row is stable. Test coverage is exactly what I'd want: the negative case (nanosecond_timestamps_cannot_survive_a_database_round_trip) proves the trap, and the positive case proves the invariant holds after truncation.

One thing worth confirming before merge: that every write path routes created_at through to_storage_precision (not just the one at service.rs:103) — a single unnormalized call site reintroduces the break. If there's a linked-in enforcement point (e.g. hashing only accepts already-truncated timestamps) that'd be even more robust. LGTM otherwise.

Truncating at the write path fixes the live break, but leaves the invariant
as a convention: a future write path that stamps `created_at` without
`to_storage_precision` reintroduces the write/read preimage split.

Normalize at the single point that consumes the value instead, so the
enforcement is structural. Truncation is idempotent and the write path
already truncates, so no digest changes — verified by writing a chain with
the previous commit and validating those same rows under this one.

The `assert_ne!` test that pinned the trap moved down a level: it now asserts
on the RFC-3339 preimage (`.123456789` vs `.123456`), which is the property
that actually causes the split, and a new test asserts `compute_hash` yields
the same digest either way.

Signed-off-by: Shani Singh <teamdeveloperworld@gmail.com>
@shani-singh1

Copy link
Copy Markdown
Author

Both points addressed — the second one by taking your suggestion.

Every write path. Confirmed: there is exactly one. log_inner (service.rs:112) is the only place in the workspace that assigns AuditEntry.created_at. Every producer goes through NewAuditEntry, which deliberately carries no timestamp — the two external call sites (buzz-relay/src/api/media.rs:426, buzz-relay/src/handlers/event.rs:558) construct that struct, and its doc comment already states seq/prev_hash/hash/created_at are assigned by AuditService::log.

The enforcement point. You were right that convention isn't enough, so I did it: 08bcce9 moves the normalization into compute_hash itself, which is the single point that consumes the value.

hasher.update(
    to_storage_precision(entry.created_at)
        .to_rfc3339()
        .as_bytes(),
);

A future write path that forgets to truncate now can't split the preimage — write-time hashes trunc(ts), read-time hashes trunc(trunc(ts)), and truncation is idempotent. The write-path truncation stays, because it carries a second invariant worth keeping: the AuditEntry returned from log() is byte-identical to the row a later read returns.

Digest stability, verified rather than argued. Since the write path already truncates, normalizing again should be a no-op on real chains — but "should be" isn't good enough for a hash function, so I checked it against Postgres: wrote a 3-entry chain with the previous commit, then verified those same rows with this one via a throwaway integration test.

verifying 3 pre-existing rows (seq 1..=3)
test preexisting_chain_still_verifies ... ok

So this commit is digest-compatible with the one before it — no re-anchoring needed on top of what the first commit already required.

Test restructure. nanosecond_timestamps_cannot_survive_a_database_round_trip had to go, since its assert_ne! is now false by construction. It's replaced by two tests that split the concern properly:

  • rfc3339_sub_second_width_follows_the_value — pins the trap one level down, on the preimage itself (…20.123456789+00:00 vs …20.123456+00:00). That's the property that actually causes the split, and it stays true regardless of what compute_hash does with it.
  • compute_hash_normalizes_sub_microsecond_timestamps — pins the new guarantee: handed an untruncated created_at, compute_hash still produces the storage-precision digest.

Re-ran everything after the change: 13 unit tests pass without Postgres, 6/6 chain tests pass against Postgres 17, clippy -D warnings and fmt --check clean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

buzz-audit: every hash chain fails verification — created_at is hashed at nanosecond precision but stored at microsecond precision

2 participants