Solar Energy Intelligence Platform

Time-series forecasting + anomaly detection + ROI expectation, using a real 1-year solar dataset, with experiment tracking and a dashboard experience.

Project Objectives

• Production-minded PoC for solar energy intelligence with Pytest, Ruff and Version control
• Implement forecasting and anomaly detection pipelines.
• Compare model performance against a naive persistence baseline.
• Track and compare experiments with MLflow.
• Calculate ROI values based on defined building and solar infra parameters.
• Expose insights through a Streamlit dashboard.

Scope

• Real 1-year solar dataset.
• Forecasting horizons: 3-day, 14-day, and 30-day.
• Model optimization workflows included.
• Reproducible local execution with ML-Flow and Streamlit app.

Target Folder Structure

solar-intelligence-platform/
├── data/
│   ├── raw/
│   └── processed/
├── src/
│   ├── ingestion.py
│   ├── features.py
│   ├── models/
│   │   ├── forecaster.py
│   │   └── anomaly.py
│   ├── evaluation.py
│   └── schemas.py
├── app/
│   └── dashboard.py
├── tests/
├── notebooks/
│   └── 01_eda.ipynb
├── Dockerfile
├── pyproject.toml
└── RESULTS.md

Core Libraries

• Polars
• Prophet
• PyTorch
• scikit-learn
• MLflow
• SHAP
• Optuna
• FastAPI
• Streamlit
• Plotly
• PostgreSQL

Standout Features

• Multi-horizon overlay chart (3d/14d/30d) with confidence bands and anomaly markers.
• Calculate and optimize ROI on solar structure infrastructure.
• Real metrics in RESULTS.md compared against a naive persistence baseline.

Features Used/Enginered

The pipeline first builds the modeling frame in src/pipeline.py, then runs feature engineering in src/pipeline.py. The actual engineered columns come from src/features.py:

• hour
• day_of_week
• month
• day_of_year
• is_weekend
• lag features like solar_production_kwh_lag_1, solar_production_kwh_lag_96, solar_production_kwh_lag_672 when samples_per_day=96
• rolling features like solar_production_kwh_roll_mean_* and solar_production_kwh_roll_std_*

Solar-specific Features & MLflow + Streamlit Integration

• pvlib-derived features: The pipeline can augment the modeling frame with solar-specific synthetic features using pvlib (clear-sky baselines and geometry). When enabled (set pvlib.enabled = true in configs/forecast_config.json) the feature engineering step adds:

  • azimuth, elevation, zenith_angle (solar geometry)
  • airmass_relative (optical air mass)
  • ghi, dni, dhi (clear-sky irradiance baselines) These features are generated before lag/rolling feature creation so models and anomaly detectors can use them as exogenous inputs.

• Enablement & requirements: Install pvlib into your environment to activate these features:

pip install pvlib

• MLflow experiment tracking: The pipeline logs experiment runs (params, metrics, and artifacts) to the local mlruns/ directory. Each run includes model predictions, evaluation metrics (RMSE/MAE by horizon), anomaly metrics, and saved forecast artifacts (data/processed/*.parquet). To inspect runs locally:

mlflow ui --backend-store-uri mlruns
# then open http://127.0.0.1:5000 in your browser

• Streamlit + MLflow workflow: The Streamlit dashboard (app/dashboard.py) is designed to work alongside MLflow: run the pipeline to produce forecasts and MLflow logs, start the mlflow ui to inspect runs, and open the Streamlit app to visualize time-series, anomalies, and model comparisons. Typical workflow:

python -c "from src.pipeline import run_pipeline; run_pipeline('data/raw/solar_data.xlsx')"
mlflow ui --backend-store-uri mlruns
streamlit run app/dashboard.py

• Notes: The dashboard limits large dataframes for browser stability and uses explicit keys for Plotly charts to avoid duplicate element IDs. If pvlib is not installed or the pvlib config is disabled, the pipeline will continue without the solar-derived features.

ROI Calculation & Model Comparison

• Annualized ROI: The pipeline computes realistic Return on Investment by extrapolating the test-period forecast window to a full 365-day year. Instead of treating test-split sums as annual cash flows (which underestimates by 10–20×), the calculation:

  1. Measures the test period duration from featured_test timestamps
  2. Computes annualization factor: $ \text{factor} = \frac{365}{\text{test_period_days}} $
  3. Scales production and consumption to annual projections before discounting
  • Example: 30-day test window with $100 kWh production → annualized to $1,200 kWh/year
  • Result: Payback years and NPV are now realistic (e.g., 8–12 years instead of never)

• Per-Model ROI: The pipeline now computes separate ROI, NPV, and payback metrics for each forecasting model (Chronos, N-HiTS). This enables direct comparison of model value:

  • Each model's forecast is used to project energy production and self-consumption
  • Output files: data/processed/roi_chronos.parquet, data/processed/roi_nhits.parquet
  • MLflow logs per-model metrics: chronos_roi_upac_mean, nhits_npv_iu_mean, etc.
  • Asset-type breakdown also tracked per model (upac vs iu segments)

• ROI Scenarios: For each model, the pipeline computes three financial scenarios:

  • Low (×0.9): Conservative production estimate using yhat_lower confidence bounds
  • Mid (×1.0): Base case using median forecast yhat
  • High (×1.1): Optimistic scenario using yhat_upper confidence bounds
  • Anomaly penalty: If anomaly_rate > threshold, payback extends by up to 30% effective degradation

• Inputs & Parameters: ROI calculation uses per-building assumptions (defaults: $9k CAPEX, $180/year OPEX, $0.24 grid tariff, $0.08 export tariff). Override via roi_defaults in pipeline config or edit src/evaluation.py.

Quality Objectives

• RMSE and MAE reported against persistence baseline.
• Anomaly precision and recall measured on labelled events.
• MLflow experiment comparison logged for all major runs.
• Pydantic schemas enforced on all I/O boundaries.
• Unit tests for feature engineering, models, and schemas.
• Docker workflow runs with one command.
• No raw data committed to git via .gitignore policy.

Initial Execution Plan

1. Data ingestion and validation
2. Feature engineering for temporal signals
3. Forecast models integrated
4. Anomaly detection pipeline
5. Evaluation and metric reporting into RESULTS.md
6. MLflow tracking and model comparison
7. Dashboard implementation in Streamlit

Definition of Done

• End-to-end run from raw input to dashboard output.
• Metrics and baseline comparison documented in RESULTS.md.
• MLflow records available for model selection review.
• Tests pass for critical modules.

Current Implementation Status

• Implemented: schema contracts, ingestion (XLSX), feature engineering, baseline forecaster wrapper, rule-based anomaly labeling + isolation forest wrapper, evaluation utilities, ROI utilities, and pipeline orchestrator.
• Implemented: Streamlit dashboard scaffold with 3/14/30 overlay and confidence bands.
• Implemented: unit test suite for schemas, features, wrappers, and evaluation/ROI utilities.

Results

Quickstart (Local)

Prerequisites

• Python 3.12

• conda or any virtual environment manager

• uv for dependency installation:

• Pytorch with CUDA if using a compatible GPU -> pip install torch --index-url https://download.pytorch.org/whl/cu121

  pip install uv

1. Prepare a Python 3.12 environment:

   conda create -n solar_platform python=3.12 conda activate solar_poc

2. Install core dependencies:

   uv pip install -e ".[dev,notebook]"

3. Install advanced forecasting models (N-HiTS + Chronos-Forecasting):

   uv pip install neuralforecast chronos-forecasting

   If skipped, both backends fall back to persistence automatically.

4. Place data in data/raw/.

5. Run the pipeline:

   python -c "from src.pipeline import run_pipeline; print(run_pipeline('data/raw/solar_data.xlsx'))"

6. Launch dashboard:

   streamlit run app/dashboard.py

Development & Quality

Run tests, linters and pre-commit hooks locally to validate changes before pushing:

# install pre-commit hooks (once)
pre-commit install

# run all hooks (linting, formatting checks)
pre-commit run --all-files

# run unit tests
pytest -q

# run ruff directly for quick checks
ruff check .

Add a CI workflow to run these commands on pull requests to enforce quality checks.

Forecasting Backends

• Pipeline forecasting now runs a side-by-side comparison of nhits and chronos-forecasting in one execution.
• Output files are written as forecast_nhits.parquet, forecast_chronos.parquet, and combined forecast.parquet in data/processed/.
• If optional packages are not installed, both advanced backends gracefully fall back to persistence so pipeline execution still succeeds.
• Horizons remain 3/14/30 days; internally these map to hourly steps for advanced backends.
• Chronos is kept as zero-shot target-context inference. Engineered lag/rolling features are still produced and persisted for analysis and other model paths.

Acknowledgements

Special thanks to the authors and teams behind the forecasting models used in this project:

• Chronos-forecasting (Amazon): https://github.com/amazon-science/chronos-forecasting
• NeuralForecast (Nixtla): Nixtla team. Project: https://github.com/Nixtla/neuralforecast/
• Pv-lib (Anderson, K., Hansen, C., Holmgren, W., Jensen, A., Mikofski, M., and Driesse, A. "pvlib python: 2023 project update." Journal of Open Source Software, 8(92), 5994, (2023)): https://github.com/pvlib/pvlib-python

Data Governance

See data/README.md for local data policy and naming conventions.

Contributing & License

Contributions are welcome. Please add a CONTRIBUTING.md with contribution guidelines and a LICENSE file (MIT/Apache2 recommended) for open-source projects. Ensure tests and pre-commit hooks pass before opening a PR.