Skip to content

fix(virality): score last completed day and floor acceleration at 0#79

Merged
Naroh091 merged 5 commits into
mainfrom
fix/acceleration-scores-completed-day
Jul 15, 2026
Merged

fix(virality): score last completed day and floor acceleration at 0#79
Naroh091 merged 5 commits into
mainfrom
fix/acceleration-scores-completed-day

Conversation

@PabloPerezBenavente

Copy link
Copy Markdown
Contributor

Problem

Narratives were being badged early_surge while showing essentially zero acceleration. Investigation found two stacked issues:

  1. Acceleration was ~0 for almost the entire cohort. In prod, 22,208 of 22,256 narratives had acceleration exactly 0.0. The analysis-indicators pipeline defaulted calc_date to date.today(), but its scheduled cron fires just after midnight UTC (observed 00:04:31Z). Acceleration compares carried-forward video stats "as of calc_date" against "as of calc_date - 1", so the delta is driven entirely by videos scraped on calc_date. At 00:04 the current day has barely been scraped → current == prev → acceleration 0.

  2. The percentile classifier promoted that noise into the surge band. With the switch to percentile-based alert levels, EARLY_SURGE requires acceleration percentile ≥ 0.95. When 99.8% of the field is 0.0 and a handful are negative (decliners), any microscopically-positive narrative sorts to the top percentile — so flat narratives with accel0.0001 were classified as surging.

Changes

  • Default calc_date to yesterday (the last completed scraping day) in run_narrative_analysis_indicators_pipeline, so both comparison windows cover a full day of scraping. On prod data this lifts nonzero accelerations from 48 → ~2,288.
  • Add a --calc-date YYYY-MM-DD CLI option for backfills and re-runs.
  • Floor acceleration_rate at 0. Acceleration measures growth, so a declining narrative reads as "not surging" (0), not a negative rate. This also collapses all decliners and flats into one 0.0 block at the bottom of the percentile ranking, so only genuinely growing narratives rank high — killing the flat-narrative-as-EARLY_SURGE noise.

Verification

  • Read-only replay of the comparison logic for a completed day (2026-07-14 vs 2026-07-13) returned 2,288 narratives with nonzero acceleration vs 48 from the buggy 00:04 run.
  • mypy clean on changed files.

Notes / follow-ups

  • The cron that schedules this pipeline at 00:04 UTC lives outside this repo (not in deployment/). The default-to-yesterday change fixes it regardless, but if that external job passes an explicit date it would override the default — worth confirming where it's defined.
  • No unit test added yet for the calc_date default / --calc-date override.

🤖 Generated with Claude Code

PabloPerezBenavente and others added 5 commits July 15, 2026 10:42
The analysis-indicators pipeline defaulted calc_date to date.today(), but
its scheduled cron fires just after midnight UTC. Acceleration compares
carried-forward video stats "as of calc_date" against "as of calc_date - 1",
so the delta is driven entirely by videos scraped on calc_date. At 00:04 the
current day has barely been scraped, leaving current == prev and
acceleration == 0 for ~99.8% of narratives (22,208 / 22,256 in prod). The
percentile-based alert classifier then promoted near-zero noise into the
surge band, producing "early_surge" badges with no real acceleration.

Default calc_date to yesterday (the last completed scraping day) so both
comparison windows cover a full day. On prod data this lifts nonzero
accelerations from 48 to ~2,288. Also add a --calc-date option to the CLI
command for backfills and re-runs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Acceleration measures growth, so a declining narrative should read as "not
surging" (0), not as a negative rate. Negative rates also poison the
percentile ranking: a merely-flat (0.0) narrative sorts above every decliner
and lands in a high acceleration percentile, which is exactly what mislabels
flat narratives as EARLY_SURGE. Flooring collapses all decliners and flats
into one 0.0 block so only genuinely growing narratives rank high.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The pipeline default was changed to date.today() - 1 (the last completed
scraping day) so acceleration compares two fully-scraped days. Update the
test, which still asserted date.today(), to match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The by-expected-views ranking multiplies each video's growth rate by time
since its last stats update. With 0.1s sleeps, a ~60ms scheduling stall
between video2's create and patch was enough to widen its measurement
window, depress its computed rate, and let the slower video rank first
(assert 1 < 0 flake in CI). Widening the sleeps to 1s pushes the stall
needed to flip the order from ~0.06s to ~0.56s, making the ordering robust
to normal jitter while keeping the test structure unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up to the previous commit. The ordering flake is fixed by giving a
long measurement window to the video with the SMALLER view increase
(video1, 100->200): over a 0.1s window its bump became a spurious
~1000 views/s that could outrank video2's genuine 400 views/s. A 1s window
drops video1 to a sane ~100/s, so video2 wins ~20x nominally and still wins
under a ~0.4s stall. video2's sleeps go back to 0.1s to keep the test fast;
widening video2's own window would instead depress its rate and break the
assertion.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Naroh091 Naroh091 merged commit 8333e2b into main Jul 15, 2026
3 checks passed
@Naroh091 Naroh091 deleted the fix/acceleration-scores-completed-day branch July 15, 2026 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants