fix(activation): orphan diagnostic (observational) + eviction never-stamped (2a) + sort reload-after-promote (2b) - #319
Merged
Conversation
…tamped (2a) + sort reload-after-promote (2b)
Residual activation-orphan work for v1.1.46. Root cause reframed by code-trace:
the recorded "competing ForcePublish publish" lead is refuted (ForcePublish runs
on the flush thread; every lazy-load apply preserves diffs). Because the EAGER
isPublished field also loses its bit, the whole-batch loss is a NON-APPLY or an
explicit REVERT, not a load clobber. The v1.1.43 backstop still recovers all —
zero user impact; this lands the probe to pin the true cause plus two real
deterministic bug fixes found along the way.
1. Diagnostic (STRICTLY OBSERVATIONAL): on a verifier orphan, do a post-publish
re-read (force_publish_blocking barrier + re-query) and dump the flush
thread's recent remove-ops for the slot from a bounded ring. Classifies each
orphan PUBLISH-LAG / NON-APPLY / REVERT. Logs only — the re-drive still fires
exactly as before, so a genuine orphan is never masked. Ring recording is
gated on the verifier being enabled and records removes only (not the hot
insert path). Barrier runs on the WAL-reader thread, targets the flush
thread, holds no lock, is 500ms-capped and sequential -> cannot self-deadlock.
2a. Idle eviction over-eviction fix: extracted a pure idle_evict_decision(); a
never-query-stamped clean value now seeds a stamp and keeps (a full idle
window) instead of evict-blind (the old unwrap_or(true)). Prerequisite for
enabling eviction on single_value per_value_lazy fields (postId RSS fix).
2b. Sort reload-after-promote fix: replace_base_preserve_diff now guards on
is_loaded && !base.is_empty(), so a reload with a stale disk snapshot cannot
clobber a base holding promoted-but-unpersisted bits, while the fresh-boot
empty placeholder (SortField::new seeds new_empty) still adopts disk.
Tests: 3 idle_evict_decision unit tests, 3 replace_base_preserve_diff tests
(incl. a failing->passing regression guard), 1 diagnostic ring test. Full lib
suite green except the pre-existing query_stream_full_channel_drops_oldest.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Two deterministic write-path correctness fixes + a diagnostic for the residual activation-orphan. Part of the v1.1.46 bundle. NON-URGENT — the v1.1.43 backstop recovers every orphan (zero data loss, standby-only); these are hardening + the probe that will pin the specimen's true cause.
Changes (3 files, +404/−13)
Diagnostic (strictly observational). On a verifier orphan detection, logs a classification —
PUBLISH-LAG/NON-APPLY/REVERT— from a post-publish re-read (aforce_publish_blockingbarrier + re-query) plus a dump of a bounded remove-op ring (RemoveRecord/recent_removes_for_slot, cap 16384, recorded flush-thread-only, removes-only — off the hot insert path). The re-drive still fires unconditionally on every orphan (no early-continue) — a genuine orphan is never masked. This pins whether the observed whole-batch loss is a non-apply or an explicit revert on the next prod orphan. Barrier is safe: WAL-reader→flush thread, bounded(1) channel, 500ms cap, no lock held across the wait, sequential orphan processing.2a — idle eviction never-stamped fix (
concurrent_engine.rs). The idle-eviction sweep'sunwrap_or(true)made never-query-stamped values evictable as soon as they merged clean. Extractedidle_evict_decision(): a clean-but-never-stamped value now seeds a stamp and keeps (full idle window) instead of evicting blind. Prerequisite for enabling single_value eviction (the RSS #318 fix is gated on this).2b — sort reload-after-promote fix (
versioned_bitmap.rs/sort.rs).replace_base_preserve_diffnow guardsis_loaded && !base.is_empty(), so a base holding promoted-but-unpersisted bits can't be clobbered by a stale-disk reload, while a fresh-boot empty placeholder still adopts disk. (First attempt withis_loaded-only broke boot restore — the!base.is_empty()term is load-bearing; 3 sort-restore tests confirm.)Honesty note
2a and 2b are real correctness bugs but neither explains the observed specimen (postId=single_value / isPublished=boolean+eager / publishedAt=sort — none eviction-enabled, and eager isPublished is immune to reload-clobber). The prior load-clobber theory was refuted at code level. So this branch ships the two fixes + the diagnostic that will actually pin the specimen (non-apply vs revert) on the next prod orphan. Follow-up fix once classified.
Verification
idle_evict_decision×3 (2a);replace_base_preserve_diff×3 incl. the guard-bypass panic (2b);recent_removes_ring_filters_by_slot_and_window(diagnostic).--features server,pg-sync, skipping the deadlock test): 1287 passed, 1 failed = pre-existingquery_stream_full_channel_drops_oldest(Windows-timing).🤖 Generated with Claude Code