feat(reader): crawl-version tabs in the reader bookmark#953
feat(reader): crawl-version tabs in the reader bookmark#953FagnerMartinsBrack wants to merge 2 commits into
Conversation
|
@claude Resolve conflicts. Use your Read tool to read the local file Context
|
|
Claude encountered an error —— View job Resolving merge conflicts for PR #953
Rebasing surfaced conflicts in 2 files. Investigating now. |
|
Claude encountered an error —— View job I'll analyze this and get back to you. |
2 similar comments
|
Claude encountered an error —— View job I'll analyze this and get back to you. |
|
Claude encountered an error —— View job I'll analyze this and get back to you. |
|
@claude Resolve conflicts. Use your Read tool to read the local file Context
|
|
Claude finished @FagnerMartinsBrack's task in 27m 53s —— View job Resolved merge conflicts for PR #953 ✅
Rebased 1. 2. 3. Verification — • Branch: |
The reader's crawl bookmark now lists dated crawl versions of an article
across all three reader surfaces (public /view, owner reader, admin recrawl)
for every visitor, anonymous included. Each content-changing crawl is
snapshotted into a per-minute S3 folder and appended to a capped crawlVersions
log on the article row; the reader renders the newest 10 newest-first, the
first as the current version (badge) and the rest greyed and aria-disabled.
Write side records a version inline at the three promotion sites (select,
recrawl, refresh) only when the canonical content actually changed: CopyObject
of the winning tier source to content-versions/{enc}/{minute}/content.html,
then a compare-and-swap append of the minute id. Same-minute re-records are
idempotent overwrites.
Also fixes a pre-existing stale-canonical bug: refresh now rewrites the
canonical object when the readable text changed even if the winner tier did
not flip, so version snapshots stay consistent with what readers see.
Read side threads a new findArticleCrawlVersions provider through the reader,
falling back to the single contentFetchedAt instant for pre-feature articles.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B1UAavvi4Casx8YNAibjp6
FagnerMartinsBrack
force-pushed
the
claude/crawl-version-tabs-reader-qrhr45
branch
from
d39e841 to
94f0ffb
Compare
FagnerMartinsBrack
temporarily deployed
to
staging
GitHub Actions
Inactive
|
@claude Review this PR. Use your Read tool to read the local file Context
|
|
Claude finished @FagnerMartinsBrack's task in 7m 37s —— View job Review — crawl-version tabs in the reader bookmarkReviewed the write side (record-crawl-version + the three promotion handlers), the read side (article-reader merge, providers, store), the reader components/CSS/HTML, and the E2E/route tests. The design is clean: a pure High Priority Issues (Must Fix)None found Medium Priority Issues1. Version minute-id is derived from
|
High Priority Issues (Must Fix)None found Medium Priority Issues1. Version minute-id is derived from
|
|
@claude Fix the review issues. Use your Read tool to read the local file Context
Review to AddressSee the review comment above this one for the issues to fix. |
|
Claude finished @FagnerMartinsBrack's task in 32m 8s —— View job Fixed the review issue — crawl-version idempotency ✅Pushed Medium #1 — duplicate crawl versions on SQS retry. Approach (as the review suggested — mirror
Why Tests: updated the 6 emitter suites for the new field; added an explicit redelivery-idempotency test to each consumer (handler invoked twice with Verification: Not done (intentionally): the Low-priority stale |
…ff a stable extractedAt The version minute-id was derived from now() in the select and recrawl handlers, but recordCrawlVersion runs before transitionAndPersist. When the persist throws (DDB throttle, optimistic-lock conflict, effect-dispatch error), SQS redelivers the record; the retry lands in a later wall-clock minute with the old hash still on the row, so contentChanged stays true and a second S3 snapshot plus a second crawlVersions entry are recorded for a single content-changing crawl. That broke the "version = a crawl that changed the canonical content" invariant and the documented idempotency of the recording step. Thread an extractedAt timestamp through TierContentExtractedEvent and RecrawlContentExtractedEvent, stamped once by each upstream emitter at extraction time, and use it as recordCrawlVersion's crawledAt — mirroring how the refresh handler already keys off detail.contentFetchedAt. A redelivery now re-copies to the same minute key and appendCrawlVersion dedupes to a no-op. The field is optional: messages in flight across the deploy that adds it still parse, and the select/recrawl DLQ handlers (which need only url to terminalize a stuck crawl) keep working; the handlers fall back to now() when it is absent. Co-authored-by: Fayner Brack <FagnerMartinsBrack@users.noreply.github.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@claude Review this PR. Use your Read tool to read the local file Context
|
|
Claude finished @FagnerMartinsBrack's task in 7m 39s —— View job Review — crawl-version tabs in the reader bookmark (commit
|
High Priority Issues (Must Fix)None found Medium Priority IssuesNone found The previous round's Medium (duplicate crawl versions on SQS retry) is properly resolved. Low Priority Suggestions1. Stale seed comment in
|
2 participants
Context
Crawler Events 1 (#871) gave the reader a right-side crawl bookmark showing a single "last crawled at «time»" tab. This change makes it list dated crawl versions of an article on all three reader surfaces (public
/view/*, owner/queue/:id/view, admin recrawl) for every visitor, anonymous included.Before this, no crawl history existed anywhere — every crawl overwrote
content/{enc(url)}/content.htmland the scalarcontentFetchedAt. This introduces a per-version content snapshot in S3 plus a version log the reader can render.Behaviour
2026-07-10T09:41Z); a second content-changing crawl in the same minute is an idempotent overwrite.aria-disabled(not selectable yet).Write side (inline at the three promotion sites)
Recording is wired inline at the select / recrawl / refresh promotion handlers rather than as a new event consumer (
CanonicalContentChangedover-fires on recrawl and never fires on refresh). No new events/commands/subscriptions, no new infra.CopyObjectfrom the winner tier sourcearticles/{enc}/sources/{tier}.html→content-versions/{enc(url)}/{minuteId}/content.html. Same-minute → same key → idempotent overwrite.crawlVersions(newest-first minute-ids, cap 10) on the existinghutch-articlesrow. Write = GetItem projection → pure prepend/dedupe/slice → CAS UpdateItem (attribute_not_exists(crawlVersions) OR crawlVersions = :old); a CAS loser throws so SQS redelivers and the idempotent re-run converges.transitionAndPersist. A persist failure retries with the old hash still on the row, so recording re-runs idempotently.crawlVersions; the reader uses a read-sidecontentFetchedAtfallback instead, so "row entry ⇒ snapshot exists" holds for the future serve-a-version feature.Companion fix (approved in the plan)
Refresh with changed content but an unchanged winner tier now rewrites the canonical object (the
refresh-content-extracted-handlergate becomesexistingTier !== winnerTier || contentChanged). This fixes a pre-existing stale-canonical bug and keeps version snapshots consistent with what readers see.Read side
New
findArticleCrawlVersionsprovider contract; the reader merges provider versions with thecontentFetchedAtfallback (versions→ datedLocalTime[], newest first; else single fallback; else[]).ReaderState.lastCrawledAtis replaced bycrawlVersions: LocalTime[], threaded through the view / owner-reader / admin-recrawl pages (the iOS chromeless reader keeps omitting the bookmark). The bookmark component just renders the list — first entry current (badge), others--disabled.Testing
pnpm checkis green across all 40 checked projects (lint, types, unit/integration/E2E, 100% coverage, unused-css, knip).appendCrawlVersion(prepend/dedupe/cap),record-crawl-versionfake-client tests, and the three handlers (records on change, skips on unchanged, refresh rewrites canonical on content change)./viewanonymous +/queue/:id/viewowner): pre-feature article → single current tab + badge; seeded 3-version log → ordered keys +aria-disabledon the older two.crawl-bookmark-visualE2E now seeds a 3-version capsule; all four Linux PNG baselines were regenerated and the capsule/grip geometry assertions pass.Notes
-chromium-darwinvisual baselines could not be regenerated here (Linux-only environment); they will refresh on the next macOS run. Only the-chromium-linuxbaselines gate CI, and those are updated.ios-readplaceis unaffected (no iOS files touched); it has nochecktarget and its Swift codegen (xcodegen) isn't available in the Linux CI container, so it is not part ofpnpm check.🤖 Generated with Claude Code
https://claude.ai/code/session_01B1UAavvi4Casx8YNAibjp6
Generated by Claude Code