feat: add meteosix wrf observation

README.md

@@ -1,73 +1,61 @@
 # Chronodash
 
-## Project Flow & Structure Info
+## Project Overview
 
-Chronodash is an Elixir/Phoenix application designed to provide web-based functionality and business logic handling.  
-The project follows a clear separation of concerns:
+Chronodash is a multi-datasource monitoring and monitoring application built with **Elixir**, **Phoenix**, and the **Ash Framework**. It is designed to collect time-series metrics from various providers (starting with MeteoSIX) and visualize them in **Grafana** using **TimescaleDB**.
 
-1. **Configuration** – All environment-specific and runtime settings are under `config/`.
-2. **Codebase** – Core Elixir modules live in `lib/`, split between backend logic (`chronodash`) and web interface (`chronodash_web`).
-3. **Privileged Resources** – `priv/` holds static assets, database migrations, seeds, and translation files.
-4. **Dockerization** – All container setup is located in `docker/` for easy deployment.
-5. **Tests** – Test files are isolated under `test/` to validate modules and controllers (not detailed here).
+---
+
+## Key Architecture Concepts
+
+1. **Generic Polling Engine** – Orchestrates data collection from multiple sources using a standardized pipeline.
+2. **Standardized Data Contract** – All DataSources return a unified `ObservationData` struct, decoupling fetching logic from persistence.
+3. **Ash Framework** – Manages the domain logic and persistence layer with high-performance bulk operations.
+4. **TimescaleDB** – Optimized storage for time-series observations, enabling efficient long-term data retention and fast queries.
+5. **Observability** – Integrated with **PromEx** for system metrics and **Grafana** for business/weather metrics.
 
 ---
 
 ## Project Structure
 
 ```
 .
-├── AGENTS.md                 # Documentation or definition of system agents
-├── Makefile                  # Make script to automate project tasks
-├── README.md                 # Main project documentation
-├── config                    # Application configuration files
-│   ├── config.exs            # General project configuration
-│   ├── dev.exs               # Development-specific configuration
-│   ├── prod.exs              # Production-specific configuration
-│   ├── runtime.exs           # Runtime-loaded configuration
-│   └── test.exs              # Test environment configuration
-├── docker                    # Docker containerization files
-│   ├── Dockerfile
-│   ├── docker-compose.tel.yml
-│   └── docker-compose.yml
-├── lib                       # Main Elixir source code
-│   ├── chronodash            # Core business logic modules
-│   │   ├── application.ex
-│   │   ├── mailer.ex
-│   │   └── repo.ex
-│   ├── chronodash.ex         # Main project entry point
-│   ├── chronodash_web        # Web (Phoenix) layer of the project
-│   │   ├── controllers       # Web controllers handling routes and requests
-│   │   │   ├── error_json.ex
-│   │   │   └── health_controller.ex
-│   │   ├── endpoint.ex
-│   │   ├── gettext.ex
-│   │   ├── router.ex
-│   │   └── telemetry.ex
-│   └── chronodash_web.ex     # Web module entry point
-├── mix.exs                   # Mix configuration file (Elixir build tool)
-├── mix.lock                  # Dependency lockfile
-├── priv                      # Private application resources
-│   ├── gettext               # Translation files
-│   │   ├── en
-│   │   │   └── LC_MESSAGES
-│   │   │       └── errors.po
-│   │   └── errors.pot           # Translation template file
-│   ├── repo                   # Database-related files
-│   │   ├── migrations
-│   │   └── seeds.exs
-│   ├── specs                  # Specification files
-│   │   └── cronodash
-│   │       └── openapi.json
-│   └── static                 # Static assets served by Phoenix
-│       ├── favicon.ico
-│       └── robots.txt
-└── test                       # Project test files
-    ├── chronodash_web
-    │   └── controllers
-    │       └── error_json_test.exs
-    ├── support
-    │   ├── conn_case.ex
-    │   └── data_case.ex
-    └── test_helper.exs
+├── .env                        # Environment variables (API keys, DB credentials, etc.)
+├── .gitignore                  # Git ignore rules
+├── Makefile                    # Automation scripts (deploy, attach, db-up, etc.)
+├── config/                     # Configuration files
+├── docker/                     # Deployment infrastructure
+├── etc/                        # Provisioning and configuration
+│   ├── grafana/                # Dashboards and Datasource provisioning
+│   └── prometheus/             # Scraping configuration
+├── lib                         # Main Elixir source code
+│   ├── chronodash/             # Core Application
+│   │   ├── accounts/           # Ash Domain: User management
+│   │   ├── datasource/         # High-level data orchestration
+│   │   ├── metrics/            # Ash Domain: Locations and Observations
+│   │   ├── models/             # Standardized DTOs and internal contracts
+│   │   ├── polling/            # Generic Polling Engine (Supervisors/Workers)
+│   │   └── prom_ex/            # Custom metrics plugins
+│   ├── chronodash_web/         # Phoenix Web Layer
+│   ├── http_client/            # Centralized HTTP client (Finch + Req)
+│   └── meteosix/               # MeteoSIX API v5 Client
+├── priv                        # Database and static resources
+│   ├── repo/migrations/        # TimescaleDB hypertable migrations
+│   └── specs/schema.dbml       # Database design documentation
+└── test                        # Unit and integration tests
+    ├── chronodash/             # Core logic tests
+    └── support/                # Mock clients and test cases
 ```
+
+
+## Getting Started
+
+### Prerequisites
+- Docker & Docker Compose
+- Elixir 1.15+ (for local development)
+
+### Quick Start
+1. Get your MeteoSIX API Key and add it to `.env`.
+2. Configure your DB credentials in `.env`.
+3. Deploy the app with telemetry: `make deploy_with_tel`
+4. Access Grafana: `http://localhost:3000` (admin/admin)

config/config.exs

@@ -54,6 +54,63 @@ config :logger, :default_formatter,
 # Use Jason for JSON parsing in Phoenix
 config :phoenix, :json_library, Jason
 
+# Polling configuration
+config :chronodash, :polling_jobs, [
+  %{
+    id: :meteosix_wrf_sky_state,
+    mfa: {Chronodash.DataSource.MeteoSIX.WRF, :get_forecast, [{43.37, -8.42}, :sky_state]},
+    rate: :timer.hours(6),
+    metadata: %{source: "meteosix"}
+  },
+  %{
+    id: :meteosix_wrf_temperature,
+    mfa: {Chronodash.DataSource.MeteoSIX.WRF, :get_forecast, [{43.37, -8.42}, :temperature]},
+    rate: :timer.hours(6),
+    metadata: %{source: "meteosix"}
+  },
+  %{
+    id: :meteosix_wrf_precipitation,
+    mfa:
+      {Chronodash.DataSource.MeteoSIX.WRF, :get_forecast, [{43.37, -8.42}, :precipitation_amount]},
+    rate: :timer.hours(6),
+    metadata: %{source: "meteosix"}
+  },
+  %{
+    id: :meteosix_wrf_wind,
+    mfa: {Chronodash.DataSource.MeteoSIX.WRF, :get_forecast, [{43.37, -8.42}, :wind]},
+    rate: :timer.hours(6),
+    metadata: %{source: "meteosix"}
+  },
+  %{
+    id: :meteosix_wrf_humidity,
+    mfa:
+      {Chronodash.DataSource.MeteoSIX.WRF, :get_forecast, [{43.37, -8.42}, :relative_humidity]},
+    rate: :timer.hours(6),
+    metadata: %{source: "meteosix"}
+  },
+  %{
+    id: :meteosix_wrf_clouds,
+    mfa:
+      {Chronodash.DataSource.MeteoSIX.WRF, :get_forecast, [{43.37, -8.42}, :cloud_area_fraction]},
+    rate: :timer.hours(6),
+    metadata: %{source: "meteosix"}
+  },
+  %{
+    id: :meteosix_wrf_pressure,
+    mfa:
+      {Chronodash.DataSource.MeteoSIX.WRF, :get_forecast,
+       [{43.37, -8.42}, :air_pressure_at_sea_level]},
+    rate: :timer.hours(6),
+    metadata: %{source: "meteosix"}
+  },
+  %{
+    id: :meteosix_wrf_snow,
+    mfa: {Chronodash.DataSource.MeteoSIX.WRF, :get_forecast, [{43.37, -8.42}, :snow_level]},
+    rate: :timer.hours(6),
+    metadata: %{source: "meteosix"}
+  }
+]
+
 # Finch configuration
 config :chronodash, :http_client, Chronodash.HttpClient.Finch
 

docker/docker-compose.tel.yml

@@ -7,7 +7,7 @@ services:
     ports:
       - "9090:9090"
     networks:
-      - cronodash
+      - chronodash
 
   grafana:
     image: grafana/grafana:latest
@@ -23,9 +23,9 @@ services:
     depends_on:
       - prometheus
     networks:
-      - cronodash
+      - chronodash
 
 networks:
-  cronodash:
+  chronodash:
     external: true
-    name: ash-postgres-network
+    name: chronodash

docker/docker-compose.yml

@@ -53,5 +53,5 @@ services:
 
 networks:
   chronodash:
-    name: cronodash
+    name: chronodash
     driver: bridge