Battery impact analysis

Small pipeline to build household/vehicle/heat/hot-water demand profiles from raw DAT files, run a Pyomo-based storage/price optimization across many storage volumes, and derive supply/demand curves consumed by Streamlit visualizations.

Why this repo

• Reproducible pipeline from raw time series (DAT) to cleaned CSVs, composited profiles, optimization results and curve CSVs used by interactive Streamlit dashboards.

Quickstart (run from repository root)

1. Install Python runtime and basic deps:

python -m pip install -r requirements.txt
# additional runtime deps used by scripts (not listed in requirements.txt):
python -m pip install pyomo tqdm

2. Prepare cleaned time series (reads data/raw_data/*/*.dat -> writes data/clean_data/*/*.csv):

python data/preparation.py

3. Build profile definitions + timeseries (writes profiles/profiles.csv and profiles/profile_timeseries.csv):

python profiles/build_profiles.py

4. Run optimization (requires a Pyomo solver; writes optimization/optimization_results.csv):

python optimization/run_optimization.py

Notes: run_optimization.py calls PricingFramework.optimize(solver="gurobi"). You can pass another Pyomo-compatible solver (CBC, IPOPT, etc.) but make sure it is installed and available on PATH.

5. Build curves and start visualizations:

python curves/build_curves.py
streamlit run curves/visualize_curves.py
streamlit run profiles/visualize_timeseries.py

Important files & data flow

• data/preparation.py — parses data/raw_data/*/*.dat and writes cleaned CSVs with columns load_w and datetime under data/clean_data/*/.
• profiles/build_profiles.py — constructs profiles/profiles.csv and profiles/profile_timeseries.csv by combining cleaned time series according to profiles (adds profile_id).
• optimization/pricing_framework.py — core Pyomo model (Storage + PricingFramework). See it when changing model variables, bounds or objective.
• optimization/run_optimization.py — orchestration: slices profiles, converts to hourly kW, builds PricingFramework instances for many storage volumes and saves optimization/optimization_results.csv.
• curves/build_curves.py — converts optimization results into curves/demand_curves.csv and curves/supply_curves.csv used by Streamlit apps.
• curves/visualize_curves.py, profiles/visualize_timeseries.py — Streamlit dashboards expecting the CSVs above.

Project-specific conventions & gotchas

• Relative paths: scripts expect to be run from the repo root and often use index_col=0 when reading CSVs. Changing working directories will break file resolution.
• Units & columns: cleaned series use load_w (watts) and datetime. run_optimization.py converts to load_kw (kW) before optimization.
• Indexing / time handling:
  • PricingFramework contains __check_timeseries_indices__() that enforces timezone-aware DatetimeIndex (UTC) and equal indices across inputs. That helper is not invoked automatically in the current pipeline.
  • run_optimization.py intentionally passes prices=prices_ts.reset_index(drop=True) (a RangeIndex) and accesses timeseries by position (.iloc) when building model timesteps. If you switch the optimizer to use datetime indices, update both run_optimization.py and PricingFramework consistently.
• Markets disabled by bounds: several PV/community variables are created with bounds=(0,0) in pricing_framework.py to effectively disable those markets. To enable community or wholesale flows, edit the variable bounds in set_model_variables().

Debugging tips

• Optimization errors: first check that a solver is installed and callable (e.g., gurobi, cbc). Pyomo will raise solver-specific errors — run python -c "import pyomo; print(pyomo.__version__)" to confirm installation.
• Empty visualizations: confirm optimization/optimization_results.csv, profiles/profile_timeseries.csv, and curves/*.csv exist and have the expected columns.
• Index/time errors: when errors mention DatetimeIndex or timezone, check how you constructed prices, solar_generation, and demand (see pricing_framework.__check_timeseries_indices__).

Extending the repo

• Add missing runtime deps to requirements.txt (e.g., pyomo, tqdm) if you want easier dev setup.
• To change optimization horizons or price inputs, edit optimization/run_optimization.py start/end datetimes and ensure input price/solar CSVs align with that window.
• To enable additional markets or storage use-cases, modify storage_use_cases when constructing PricingFramework and adjust set_model_variables() / set_model_constraints() accordingly.

Contact / maintenance

If you add new entrypoints, change CSV shapes, or flip the optimizer to use datetime indices, please update .github/copilot-instructions.md and this README.md so future contributors (and AI agents) get the correct guidance.

License

See SPDX notices in source files; some modules carry an AGPL header. Follow file headers for licensing guidance.