Skip to content

P5-calendar: animedex season + animedex schedule (multi-source with fallback) #18

Description

@narugo1992

Goal

Land Phase 5 calendar commands: animedex season [<year>] [<season>] and animedex schedule [--day <day>]. Both are multi-source (AniList + Jikan), with graceful per-source fallback so that when one upstream is rate-limited or returns 5xx, the command still returns whatever the other upstream had — never crashes silently into "no output". The fallback shape is the §0 inform-not-gate principle applied to multi-source aggregation: degrade visibly, do not hide.

Refs #1 §7 (Phase 5 checklist).

Why this slice

The calendar commands are the isolated half of Phase 5: they do not need a prefix:id parser, do not need a type→backend mapping, do not need a cross-source ID map. They are wrapper commands over two upstreams' calendar endpoints, fanned out and merged with source attribution. They share zero source files with the search / show / crossref track, so this slice can run truly in parallel with P5-search-show (the only point of meeting is the trivial registration line in animedex/entry/__init__.py).

This is also the first time the project does multi-source fan-out with per-source failure handling. The fan-out helper that lands here should be generic enough that the entity track (P5-search-show) can reuse it; if the abstraction needs to be promoted to a project-level helper, that's a substrate-touch question covered in the proposal step below.

Scope (minimum required)

animedex season [<year>] [<season>] [--source <backends>] [--limit N]

  • <year> defaults to the current calendar year (local system time).
  • <season> ∈ {winter, spring, summer, fall}. Default: the season that contains the current local month (month 1–3 → winter, 4–6 → spring, 7–9 → summer, 10–12 → fall).
  • --source is a comma-separated allowlist (default: all). E.g. --source anilist or --source jikan.
  • --limit N is per source, defaulting to 25 (matches Jikan's natural page size). To globally cap, the caller uses --jq 'limit(N; .)' downstream.
  • Multi-source fan-out: AniList GraphQL Page query (media(season: WINTER, seasonYear: 2026, type: ANIME)) + Jikan REST /seasons/{year}/{season}?limit=25.
  • Output: a flat list of anime records, each carrying mandatory source attribution. Cross-source merging is the default: AniList and Jikan records that identify the same anime are grouped into one merged entry that exposes every upstream's contributing record under a sources / records map so no upstream-visible field disappears. Matching uses external IDs first (MAL ID, AniList ID, AniDB ID etc.), then a deterministic fuzzy comparison over title variants, season, year, format, episode count, and aired-from date. Single-source rows that fail to match anything on the other backend remain as their own entry with their single source attribution. The output's _source / sources fields are the user-visible contract that the same anime can carry data from multiple upstreams without losing track of which upstream contributed what.

animedex schedule [--day <day>] [--source <backends>]

  • --day ∈ {monday..sunday, today, tomorrow, all}. Default: all (whole week, returns 7 days' worth of airing rows).
  • --source as above.
  • Multi-source fan-out: AniList GraphQL AiringSchedule query (airingSchedules(airingAt_greater: ..., airingAt_lesser: ...)) + Jikan REST /schedules/{day_name} (or /schedules?filter={day_name}).
  • Output: airing rows ordered by airingAt (UTC epoch), each row carrying source attribution. The schedule path does not merge across sources (an airing row is a per-episode broadcast event, and AniList and Jikan model these differently enough that grouping them would distort the timeline); merging is the season path's job.

Fan-out fallback contract (the load-bearing piece)

For both commands, each source is dispatched independently. When a source fails:

  • HTTP 429 / 5xx: capture the upstream response, do not raise, register the source as failed.
  • Network error / timeout: same — capture the exception, register the source as failed.
  • Parse error on the response: same.
  • Auth error (if any source requires auth in future — none do today): same.

The command's return shape carries both successes and failures:

{
  "items": [ /* successful rows from all healthy sources, with _source */ ],
  "sources": {
    "anilist": {"status": "ok",   "items": 25, "duration_ms": 312},
    "jikan":   {"status": "failed", "reason": "rate-limited",
                "message": "anilist: rate-limited (HTTP 429); 25 items still returned from jikan",
                "http_status": 429, "duration_ms": 87}
  }
}

(Field names are author's choice; this JSON is illustrative. The proposal step below settles the exact schema.)

The CLI's behaviour on partial failure:

  • Successful items render normally on stdout (TTY or JSON).
  • Stderr emits one line per failed source: "source 'anilist' failed: rate-limited (HTTP 429); continuing with other sources".
  • Exit code 0 when at least one source succeeded.
  • Exit code non-zero (suggest 1 or the standard "upstream-error" mapping) only when every source failed. The stderr message in that case names every failure and the command's body output is the empty-list envelope (so jq does not crash on .items).

This is the §0 inform-not-gate principle applied to multi-source aggregation: degrade visibly, give the user whatever data is available, never silently produce empty output and never crash mid-pipeline.

Encouraged exploration

The minimum scope is a floor. Sensible extensions to consider (each lands in this PR only if it does not balloon the scope):

  • --year-range 2020-2024 or --seasons winter,spring for season: list multiple seasons.
  • Filters: --studio ghibli, --genre action, --format TV if the upstream endpoints accept them cleanly. AniList GraphQL accepts most of these; Jikan exposes them through query params.
  • A --concurrent / --sequential flag if the fan-out helper supports both modes (defaulting to concurrent for calendar — it's typically 2 sources at most).
  • Cache TTL: 1 hour for schedule, 24 hours for season (matches existing TTL table at animedex/cache/sqlite.py).

These are not required for the PR to land. If you add them, surface them in the PR body.

API documentation entry points

The high-level helpers that this PR's calendar commands fan out to should already exist from earlier phases; the work is composing them, not re-implementing per-backend endpoint logic.

Substrate touch points (read carefully)

This PR introduces the project's first multi-source fan-out helper. Before writing implementation code, post an abstraction proposal as a comment on this issue and wait for an explicit external sign-off before implementing — sign-off is a recorded maintainer reply, not a self-reply (AGENTS §15.5). The proposal should answer at least:

  1. Where does the fan-out helper live? Options:

    • (a) animedex/agg/_fanout.py — a top-level helper module that both this PR and P5-search-show will consume.
    • (b) Inline inside animedex/agg/calendar.py for now, refactor when P5-search-show lands.
    • (a) is cleaner if the abstraction is right on the first try; (b) is safer if you're not sure. Pick one; state the reason. Either is acceptable.
  2. Concurrent or sequential fan-out? Two healthy sources = 2 calls. Concurrent (via concurrent.futures.ThreadPoolExecutor with max_workers=2) means total latency = max(anilist, jikan). Sequential means anilist + jikan. Concurrent is faster but adds threading to a code path that has so far been single-threaded; rate-limit buckets are per-backend so there's no contention. Propose one with rationale.

  3. The failure envelope schema. The illustration above ({items, sources: {anilist: {...}, jikan: {...}}}) is one shape. Two design questions: (i) Do per-source failures show under sources only, or also as inline annotations on the items list (e.g. _failed_sources: [...] next to items: [])? (ii) Should the failure envelope reuse the existing RawResponse.firewall_rejected pattern, or define a new AggregateResult model under animedex/models/? I lean toward a new top-level model since this is the first aggregate-shape envelope in the project — settle in the proposal.

  4. Default season inference. Local month → season mapping is the obvious choice (1–3 winter, 4–6 spring, 7–9 summer, 10–12 fall). One nuance: some communities use Dec/Jan/Feb as winter rather than Jan/Feb/Mar. AniList itself uses the Mar-boundary convention. Default to that and document it in the docstring; do not invent a different boundary.

  5. Empty-day on schedule. If a day has zero airing rows from both sources, the output should still be a valid envelope (items: []) with both source statuses ok, not a failed envelope. Confirm the proposal handles this.

If the proposal you write differs materially from the above, that is fine; the proposal exists to surface the trade-off in writing before code lands.

Fixture capture

Three fixture scenarios per command (across tools/fixtures/run_anilist.py and tools/fixtures/run_jikan.py):

  1. Happy path — both sources return data normally for a recent season (e.g. 2024 winter). Capture page 1 from each.
  2. Partial failure — capture a 429 response from AniList (the project's degraded-tier limit makes this realistic). The fan-out helper should still emit jikan data and inform on stderr. You do not need to cause a 429 against the live AniList; fabricate it at the YAML level by editing one captured fixture's response.status and response.body_text, leave a response.captured_from: synthetic-429 note in the YAML metadata.
  3. Total failure — both sources synthetic 5xx. Same hand-edit technique; document the synthesis in the YAML metadata.

Capture date and any proxy notes go in the PR body, per AGENTS §15.5. The captured-from-live fixtures are real; the synthetic ones are documented as such.

Verification checklist (self-check before requesting review)

  • animedex season (no args) returns the current year's current season from both AniList and Jikan, each row carrying _source.
  • animedex season 2024 winter returns 2024 winter from both sources.
  • animedex season --source jikan only fans out to Jikan; the envelope's sources map contains only jikan.
  • animedex season --limit 5 returns at most 5 records per source.
  • animedex schedule (no args) returns the full 7-day airing list from both sources.
  • animedex schedule --day monday returns Monday-only rows from both sources.
  • animedex schedule --day today resolves correctly against local system date.
  • Partial failure: replay the AniList-429 fixture; assert the command exits 0, stdout contains jikan rows, stderr contains an inform line naming AniList and the rate-limit reason.
  • Total failure: replay both-source-5xx fixtures; assert the command exits non-zero, stdout is an empty-items envelope, stderr names both failed sources.
  • CLI tested in both --json (forced JSON) and the default TTY path (isatty()=True forced via the existing force_tty fixture). See AGENTS §9bis.6 for the rule.
  • HTTP is the only mock seam — responses.RequestsMock against captured fixtures; no monkeypatch.setattr(animedex.backends.<x>, ...) above-the-wire shortcuts (§9bis.1).
  • No new entry in animedex/transport/read_only.py is needed (no new backends introduced).
  • _BACKEND_POLICY in animedex/entry/_cli_factory.py gains entries for the two aggregate command groups if they're registered as Click subcommands at the top level (or directly on animedex_cli); the docstring format follows §10 with the \f cutoff convention.
  • make rst_auto regenerates docs/source/api_doc/agg/ (or wherever the new module lives) and the diff is committed.
  • Tutorial entries under docs/source/tutorials/ cover at least one runnable season and schedule example each.
  • _SELFTEST_TARGETS in animedex/diag/selftest.py registers the new animedex.agg.* modules.
  • Pre-flight (AGENTS §15.4): make format && make test && make rst_auto green; make build && make test_cli green; python -m animedex.policy.lint animedex/ green; grep -rE 'Phase [0-9]|AGENTS[. ]§|Reviewer review' animedex/ tools/ returns zero matches.

Load-bearing reminders

  • Lossless source attribution under merging. When a season row groups AniList and Jikan into one entry, the rich-model fields from each upstream are preserved under the merged record's sources / records map. §13 still applies: no upstream-visible field disappears. The merged row's compact TTY rendering may project a common shape for the eye, but the JSON path keeps each upstream's full rich record.
  • Merging is the user-visible default, not an opt-in. Matching is conservative: external IDs first, then a deterministic fuzzy comparison; a single threshold cutoff determines when two upstreams' rows are merged into one entry. Rows that do not meet the threshold remain as their own single-source entries. The merge is the value the multi-source command exists to produce.
  • §0 inform-not-gate on degraded sources. A failed source is never the reason the command crashes; the user gets the data the other source returned, plus a clear stderr line about what failed. The empty-when-everything-fails path still emits a structurally valid envelope on stdout.
  • HTTP-only mock seam. Tests load real captured fixtures (and the documented synthetic 429 / 5xx fixtures) through responses.RequestsMock. No project-internal monkeypatch.setattr.

Parallelism

This issue can be picked up by codex while the P5-search-show issue runs in parallel on another codex. They share zero source files; the only conflict point is the registration line where animedex_cli learns about the new top-level commands, which is a trivial textual merge.

The P5-crossref issue (cross-source ID translation) depends on this PR's fan-out helper and on P5-search-show's prefix:id parser, so it waits for both.

PR body template (what to include)

When you open the PR, please include:

  • Summary — one-paragraph overview of the new commands, the fan-out approach, and the fallback contract.
  • Demos — one TTY GIF per command (vhs / ttyd per the terminal-capture-workflow skill). For schedule, ideally a captured run that exercises the partial-failure path so reviewers can see the stderr inform line live.
  • Examples and expected output — copy-pastable command lines + the jq-extracted expected output, plus one example showing the partial-failure shape.
  • Fixture notes — capture date in UTC; explicit "synthetic 429" / "synthetic 5xx" disclosure for any hand-edited fixtures; whether a proxy was needed.
  • Verification — tick the checklist above; for each item write done / partial / deferred with a one-line reason.

Out of scope

  • --no-merge / inspect-raw-fan-out toggle (not required; the merged output keeps per-upstream rich records under the sources / records map, so a caller who needs the raw upstream rows reads them from there).
  • Filter by --studio, --genre, --format beyond what the upstream endpoints accept natively.
  • Concurrent fan-out across more than 2 sources (today only AniList + Jikan).
  • The aggregate search / show / crossref commands — those live in P5-search-show and P5-crossref.
  • Phase 7 AniDB integration — completely separate track.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions