feat: add Browse page with agenda view#23
Open
fsargent wants to merge 4 commits into
Open
Conversation
Adds a new `civico` provider class that paginates Civico's archived-meetings JSON API (www.civico.net/api/{authority}/{offset}/archived) and ingests meeting metadata. UIDs are prefixed (civico-{authority}-{id}) to stay globally unique across authorities, since Civico's integer id is only per-tenant. `get_meetings` is currently a pass-through: Westminster's Civico tenant publishes no captions (live_transcription is 0; no subtitle track in DASH/HLS manifests), so transcript search will return no Westminster results until a future ASR pipeline is added. To register Westminster after deploy: POST /meetings/add_provider?provider=civico POST /meetings/add_authority?authority=westminster&provider=civico&nice_name=Westminster Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per-authority INFO logs (build index, fetch transcripts, store) plus top-level start/complete, so 'tail -f' shows clear ingest progress. Per-authority try/except + logger.exception so one provider's failure doesn't terminate the whole batch silently. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds GET /meetings/browse for listing meetings without a search query, with
filters for authority, date range, transcript availability, and sort. Useful
for spotting meetings that have been ingested but not yet transcribed.
Adds GET /meetings/{uid}/agenda that reads the existing `agenda` table
(populated by add_meetings_to_db but never exposed). Returns ordered agenda
items.
Extends the Civico provider to scrape #agenda_data li.agenda from each
meeting's HTML page, since Civico publishes no machine-readable agenda
endpoint. Agenda fetching runs in a ThreadPoolExecutor like PublicI's
transcript fetch, and a single failed page degrades to an empty agenda
rather than aborting the whole ingest.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds /browse: a paginated list view of every meeting in the DB, with the
same authority/date filters as Search plus a "Transcript availability"
selector so users can see which meetings have not been transcribed yet.
Each card has a "Show agenda" toggle that lazily fetches /meetings/{uid}/agenda
and renders it inline.
Wraps the app in a v-app-bar with Search/Browse tabs so users can switch
between the two views.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
GET /meetings/browse— paginated list of meetings without a search query, with filters for authority, date range, transcript availability, and sort. Useful for spotting meetings that have been ingested but lack transcripts.GET /meetings/{uid}/agenda— reads the existingagendatable (populated on ingest but previously unreadable from the API).#agenda_data li.agendafrom each meeting's HTML page. Civico has no agenda JSON endpoint, but the agenda is rendered server-side. Scraping runs in aThreadPoolExecutor; per-page failures degrade to an empty agenda rather than aborting ingest./browseVue page with an expandable "Show agenda" section per meeting card, plus a Search/Browse tab bar inApp.vue.Why
The existing Search view requires a query and only ranks transcripts, so meetings without a transcript (which is currently 100% of Westminster's Civico meetings) are invisible. This adds a way to browse the corpus, see ingest coverage gaps, and read agendas even when no transcript exists.
Test plan
GET /meetings/browsewith eachhas_transcriptvalue (true/false/omitted) returns the expected counts.GET /meetings/{uid}/agendareturns 6 items forcivico-westminster-18872, matching the Civico page.agendatable, no errors.🤖 Generated with Claude Code