Ibiltari: Basque Country Transport Alerts

A Python + FastAPI backend to query transport arrivals and receive Telegram notifications for the Basque Country (Euskadi).

Features

• FastAPI for Stops, Arrivals, and Subscriptions API.
• Telegram bot for subscription management.
• Worker to ingest GTFS Static data and poll GTFS-RT (Real-Time) feeds.
• Integrates with Moveuskadi data index for comprehensive operator coverage in Euskadi.

Requirements

• Docker + Docker Compose, or Python 3.11+
• A valid GTFS-RT source (Moveuskadi recommended)

Configuration

1. Copy .env.example to .env and fill in the required values.
   • TELEGRAM_BOT_TOKEN (Required for Telegram bot)
   • Data Sources (choose one set):
     • GTFS_STATIC_URL / GTFS_RT_URL: Direct URLs for GTFS Static and GTFS-RT feeds.
     • OR MOVEUSKADI_INDEX_GTFS_URL / MOVEUSKADI_INDEX_GTFS_RT_URL: URLs for the Moveuskadi GTFS and GTFS-RT index JSON files.
   • MOVEUSKADI_INDEX_GTFS_CACHE and MOVEUSKADI_INDEX_GTFS_RT_CACHE: Local paths for caching Moveuskadi index files.
   • MOVEUSKADI_INDEX_REFRESH_HOURS: Automatic refresh interval for cached Moveuskadi index files.
   • OPERATOR_FILTER: If set, resolves URLs only for this operator (comma-separated list).
   • OPERATOR_ALLOWLIST: Comma-separated list of operators to explicitly allow (if resolving all operators from Moveuskadi).
   • OPERATOR_BLOCKLIST: Comma-separated list of operators to explicitly block.
   • INCIDENTS_URL: Comma-separated JSON feed URLs for incident ingestion.
   • INCIDENTS_REFRESH_SECONDS and REDIS_INCIDENTS_TTL_SECONDS: Incident refresh and cache TTL.
2. (Optional) MOVEUSKADI_INDEX_GTFS_URL and MOVEUSKADI_INDEX_GTFS_RT_URL for operator discovery via index.
3. Worker tunables are centralized in config/app.toml (not in env), including notifier polling cadence and proximity update window.

Getting Started

Moveuskadi Index Utilities

The scripts/extract_moveuskadi_index.py utility helps resolve feed URLs from the Moveuskadi index:

uv run python -m scripts.extract_moveuskadi_index \
  --index "https://s3.itbatera.euskadi.eus/02-pro-e3525cfb1b3d99109c5220a2b24bcb30-inet/transport/moveuskadi/data-index-gtfs.json" \
  --operator "<operator>"

To export to a file:

uv run python -m scripts.export_operator_feeds \
  --index "https://s3.itbatera.euskadi.eus/02-pro-e3525cfb1b3d99109c5220a2b24bcb30-inet/transport/moveuskadi/data-index-gtfs.json" \
  --operator "<operator>" \
  --out "data/operator_gtfs.json"

Run GTFS data quality verification and generate per-operator reports:

uv run python scripts/verify_data_quality.py \
  --insecure \
  --out-md reports/data_quality_report.md \
  --out-json reports/data_quality_issues.json

Run synthetic DB/Redis scale benchmark report:

uv run python scripts/benchmark_scale.py \
  --operators 10 \
  --stops-per-operator 2000 \
  --out reports/scale_benchmark_report.md

Run synthetic routing benchmark report:

uv run python scripts/benchmark_routing.py \
  --stops 120 \
  --departures-per-link 6 \
  --queries 400 \
  --max-p95-ms 25 \
  --out reports/routing_benchmark_report.md

CI smoke benchmark:

• Workflow: .github/workflows/routing-benchmark-smoke.yml
• Runs automatically on PRs touching routing core/benchmark code.
• Writes report to a temporary path in CI and uploads it as artifact.

Running the Application

docker-compose up --build

Running the Web Frontend (MVP)

npm --prefix apps/web install && npm --prefix apps/web run dev

Frontend docs: apps/web/README.md Web CI smoke: .github/workflows/web-smoke.yml

API Endpoints (FastAPI)

• GET /operators?source=gtfs|gtfs-rt (returns name + slug)
• GET /stops?query=
• GET /stops/{stop_id}/arrivals?enrich=true
• GET /route?from_stop_id=&to_stop_id=&departure_ts=&max_transfers=
• POST /subscriptions
• GET /subscriptions/{id}
• GET /subscriptions?chat_id=...
• DELETE /subscriptions/{id}
• GET /incidents?operator=&route_id=&stop_id=&active=true

Swagger UI: http://localhost:8000/docs OpenAPI JSON: http://localhost:8000/openapi.json OpenAPI YAML (generated): openapi/openapi.yaml (run scripts/generate_clients.sh)

Telegram Bot Commands

• /start: Initializes the bot conversation.
• /menu: Returns to main menu.
• /cancel: Cancels current flow and returns to main menu.
• /add <stop_id> [route_id] [threshold] [delta] [op=OPERATOR]: Adds a new arrival alert.
• /list: Lists your active alerts.
• /route <from_stop_id> <to_stop_id> [departure_ts] [max_transfers] [op=OPERATOR]: Plans an itinerary.
• /remove <id>: Removes an alert by its ID.
• /setthreshold <id> <min>: Sets the notification threshold for an alert.
• /setdelta <id> <min|NNs>: Sets the notification delta for an alert (minutes or seconds, e.g. 90s).
• /language [eu|es|en|fr]: Shows/sets preferred language for bot messages.

Implementation Details

• GTFS Static data is downloaded and loaded into PostgreSQL every GTFS_STATIC_REFRESH_HOURS.
• GTFS-RT data is polled every GTFS_RT_POLL_SECONDS seconds.
• Incident feeds (INCIDENTS_URL) are normalized, deduplicated, persisted in DB, and cached in Redis key incidents:active.
• IDs are namespaced as OPERATOR:stop_id to prevent collisions across operators.
• GET /stops and /arrivals endpoints accept an operator parameter for filtering.
• The Moveuskadi index can be used to resolve operator feeds dynamically.
• Alert delivery diagnostics runbook: reports/alerts_delivery_runbook.md.

Contributing

See CONTRIBUTING.md for development workflow and guidelines.

OCI Operations

• Auto-restart for OCI Always Free instances is available at infra/oci/auto-restart.
• It implements Alarm -> Notifications -> Function -> controlled restart (SOFTRESET with RESET fallback).