Predict the maximum amount of energy that can be safely exported from a home battery during the evening peak, without leaving the home short before solar recovers the next morning.
Status: Phase 3 (Home Assistant integration) underway. Phases 1 (data extraction) and 2 (modelling) are complete. A Node-RED flow has run the model live at P50 confidence since 2026-05-22, as a low-cost stand-in for the eventual HACS integration (targeted September 2026).
Each evening at 6pm, residential battery owners with solar face a decision:
- Export aggressively to capture the day's highest grid-feed-in tariffs, but risk running the battery flat overnight and importing expensive grid power at the worst possible time.
- Hold back to guarantee comfort overnight, leaving export revenue on the table.
ha-safe-export is a Home Assistant integration (eventually — see phases) that takes the guesswork out of this decision. At 6pm each day it answers a single question:
Given the current battery charge, the weather forecast, and what we've learned from past nights, how much can be safely exported between now and 9pm such that the battery still has enough at 11am tomorrow?
The answer is exposed as an HA sensor that can drive automations or simply inform manual decisions.
The naive approach — "export anything above a fixed reserve threshold" — works on average but fails on the days that matter most. A cold cloudy night with high heating load can drain a battery that seemed safe at 6pm. A clear sunny morning following a moderate evening can leave the battery wastefully full. The decision needs to anticipate:
- Overnight consumption, which scales with outdoor temperature and household occupancy
- Morning solar recovery, which depends on the day-ahead weather forecast
- Provider context, which changes the value of every kWh in or out of the battery
- Curtailment effects, where a battery already at 100% during the day couldn't absorb all available solar
The model uses ~2.5 years of historical operational data to learn these relationships and produce a calibrated, uncertainty-aware export limit.
This project is currently scoped to one specific home installation:
| Component | Detail |
|---|---|
| Battery | BYD Battery-Box Premium HV, 13.8 kWh |
| Solar inverter | Fronius (via SolarNet integration in HA) |
| Smart meter | Fronius 63A single-phase |
| Climate sensors | Netatmo (indoor + outdoor) |
| Location | Melbourne, Australia |
| Energy providers (over time) | Energy Australia → Amber Energy → GloBird |
The data extraction is hardcoded to these specific sensors and providers. Generalising this is a Phase 3 concern; the integration version will discover available sensors at config time.
┌────────────────────┐ ┌──────────────────┐ ┌─────────────────────┐
│ HA recorder DB │───▶│ extract.py │───▶│ ha-safe-export.db │
│ (read-only) │ │ daily extraction │ │ (one row per night) │
└────────────────────┘ └──────────────────┘ └──────────┬──────────┘
│
▼
┌────────────────────┐ ┌──────────────────┐
│ Solcast forecast │───▶│ model.py │
│ Weather forecast │ │ predict() at 6pm │
│ Live HA state │ └────────┬─────────┘
└────────────────────┘ │
▼
┌──────────────────┐
│ HA sensor │
│ safe_export_wh │
└──────────────────┘
Phase 1 builds the extraction half. Phase 2 builds the prediction half. Phase 3 wraps both in an HA integration.
ha-safe-export/
├── CLAUDE.md ← Standing instructions for AI agents working on the code
├── README.md ← This file
├── CHANGELOG.md ← Version history (Keep a Changelog format)
├── TODO.md ← Working notes and live-testing follow-ups
├── config/
│ ├── config.example.yaml ← Template config
│ └── config.yaml ← gitignored; your sensor names, coefficients, and history
├── docs/
│ ├── SPEC.md ← Project specification: prediction objective, success criteria
│ ├── DATASET.md ← Data contract: schema, sensors, formulas, validation samples
│ ├── DECISIONS.md ← Rationale log for design choices (read before changing them)
│ └── analysis/ ← Background analysis docs (model selection, schema evolution)
├── src/
│ ├── config.py ← Config dataclass + load_config()
│ ├── extract.py ← Builds and refreshes the dataset (Phase 1)
│ ├── schema.sql ← Canonical DDL for the dataset DB
│ ├── windows.py ← Timezone-aware window math
│ ├── model.py ← Four-zone predictor + predict() function (Phase 2)
│ └── migrations/ ← Historical schema migrations (not auto-applied; schema.sql is canonical)
├── tests/
│ ├── fixtures.py ← Known-good values for three validation days
│ ├── test_extract.py ← Extraction fixture tests
│ ├── test_model.py ← Model unit and regression tests
│ └── test_sync.py ← Enforces coefficient parity across config.yaml/conftest.py/nodered-flow.json/model.py
├── tools/
│ ├── backtest.py ← Economic backtest; outputs backtest_report.html/.json
│ ├── retrain.py ← Refits the four-zone model from the dataset
│ └── nodered-flow.json ← Live Node-RED flow: runs predict() at 6pm, writes to HA helpers
├── data/ ← gitignored; holds the dataset DB
└── pyproject.toml
| Document | What it covers |
|---|---|
docs/SPEC.md |
What the model predicts, success criteria, inference-time inputs, what's out of scope |
docs/DATASET.md |
The data contract — every column, every sensor, every formula, three validation fixtures |
docs/DECISIONS.md |
Why each significant design choice was made; rejected alternatives; evidence |
CLAUDE.md |
Standing context for AI agents (Claude Code, etc.) — gotchas and conventions |
The DECISIONS.md log is the most important one to consult before changing how anything is computed. Several non-obvious choices (timezone handling, sensor selection, balance-derived consumption) have specific evidence behind them and should not be undone without strong justification.
| Phase | Deliverable | Status |
|---|---|---|
| 1. Data extraction | src/extract.py builds an incrementally-updateable SQLite dataset (see src/schema.sql for the current version); passes three validation fixtures |
Complete |
| 2. Modelling | src/model.py — four-zone linear consumption model with calibrated P90/P95 uncertainty bounds; predict() callable for Phase 3 |
Complete |
| 3. HA integration | Node-RED flow live at P50 confidence since 2026-05-22. A HACS component is a possible future step; whether it's worth building is an open trade-off, not a committed date | Underway |
# Create and activate a virtual environment
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
# Install (no external dependencies beyond tzdata on Windows)
pip install -e .
# First-time extraction (point at your HA recorder DB)
python -m src.extract /path/to/home-assistant_v2.db
# Rebuild from scratch (e.g. after a methodology change)
python -m src.extract /path/to/home-assistant_v2.db --rebuild
# Run tests
python -m pytestThe work-issue skill (and CLAUDE.md's GitHub conventions generally) work with plain gh, but are smoother with the gh-axi skill installed, which wraps gh for issues/PRs/CI/releases. It requires gh itself to be installed and authenticated (gh auth status) first:
npx skills add kunchenguid/gh-axi --skill gh-axi -gNot required — agents fall back to plain gh (or native GitHub MCP tools, in sessions that expose them) if gh-axi isn't present.
The model needs overnight mean temperature and humidity derived from the hourly weather forecast. Add these trigger-based template sensors to your HA configuration.yaml (or a package YAML file). They call weather.get_forecasts on a schedule and average the 6pm–11am window.
template:
- trigger:
- trigger: time
at: "17:59:00"
- trigger: homeassistant
event: start
- trigger: event
event_type: event_template_reloaded
action:
- action: weather.get_forecasts
data:
type: hourly
target:
entity_id: weather.truganina_hourly # Replace with your weather entity
response_variable: hourly
sensor:
- name: "Overnight Forecast Temp Mean"
unique_id: overnight_forecast_temp_mean
unit_of_measurement: "°C"
state_class: measurement
state: >
{% set forecasts = hourly['weather.truganina_hourly']['forecast'] %} {# Replace entity name #}
{% set tomorrow = (now().date() + timedelta(days=1)).strftime('%Y-%m-%d') %}
{% set tonight = now().date().strftime('%Y-%m-%d') %}
{% set ns = namespace(total=0, count=0) %}
{% for f in forecasts %}
{% set dt = f.datetime %}
{% set is_tonight = dt >= tonight ~ 'T18:00:00' and dt < tonight ~ 'T24:00:00' %}
{% set is_tomorrow_morning = dt >= tomorrow ~ 'T00:00:00' and dt <= tomorrow ~ 'T11:00:00' %}
{% if is_tonight or is_tomorrow_morning %}
{% set ns.total = ns.total + f.temperature %}
{% set ns.count = ns.count + 1 %}
{% endif %}
{% endfor %}
{{ (ns.total / ns.count) | round(1) if ns.count > 0 else 'unknown' }}
- name: "Overnight Forecast Humidity Mean"
unique_id: overnight_forecast_humidity_mean
unit_of_measurement: "%"
state_class: measurement
state: >
{% set forecasts = hourly['weather.truganina_hourly']['forecast'] %} {# Replace entity name #}
{% set tomorrow = (now().date() + timedelta(days=1)).strftime('%Y-%m-%d') %}
{% set tonight = now().date().strftime('%Y-%m-%d') %}
{% set ns = namespace(total=0, count=0) %}
{% for f in forecasts %}
{% set dt = f.datetime %}
{% set is_tonight = dt >= tonight ~ 'T18:00:00' and dt < tonight ~ 'T24:00:00' %}
{% set is_tomorrow_morning = dt >= tomorrow ~ 'T00:00:00' and dt <= tomorrow ~ 'T11:00:00' %}
{% if is_tonight or is_tomorrow_morning %}
{% set ns.total = ns.total + f.humidity %}
{% set ns.count = ns.count + 1 %}
{% endif %}
{% endfor %}
{{ (ns.total / ns.count) | round(1) if ns.count > 0 else 'unknown' }}The event_template_reloaded trigger fires immediately when you reload templates via the UI — no HA restart needed after a config change.
tools/nodered-flow.json is a ready-to-import Node-RED flow that runs the predictor automatically at 6pm each day and writes the results back to Home Assistant helpers.
- Triggers at 6pm (plus a manual trigger button for testing)
- Reads five HA sensors in sequence: overnight forecast temp, humidity, Solcast tomorrow, battery SOC, and min SOC cutoff
- Runs the four-zone linear model in a function node (no Python needed — coefficients are embedded as JS constants)
- Writes results to two HA helpers:
input_number.safe_export_wh— P50 safe export in Wh (integer) — the flow's current default output, found to be the viable operating level (see TODO.md). Use this directly as a W export limit for a 1-hour window, or divide by 3 to spread over 3 hours. All four confidence levels are available in the detail JSON below if you want a more conservative figure.input_text.safe_export_detail— compact JSON with all four confidence levels and context. Allp50/p75/p90/p95values in the JSON are Wh;avail_kwhis kWh. Internal model fields (consumption,buffer,total_needed,grid_needed) are kWh.
1. Create the HA helpers (Settings → Devices & Services → Helpers):
| Type | Entity ID | Min | Max | Step | Unit |
|---|---|---|---|---|---|
| Number | safe_export_wh |
0 | 13800 | 1 | Wh |
| Text | safe_export_detail |
— | — | — | max length 255 |
2. Import the flow in Node-RED: hamburger menu → Import → paste the contents of tools/nodered-flow.json.
3. Set the HA server on each node (they'll show as unconfigured until you select your Home Assistant connection).
4. Update the sensor entity IDs. The flow's state-reader nodes are pre-populated with the author's sensor names. Open each of the five "Get …" nodes and replace the entity ID with your own:
| Node | Entity ID to replace | What it reads |
|---|---|---|
| Get overnight temp | sensor.overnight_forecast_temp_mean |
Your overnight temp template sensor (see above) |
| Get overnight humidity | sensor.overnight_forecast_humidity_mean |
Your overnight humidity template sensor (see above) |
| Get Solcast tomorrow | sensor.solcast_pv_forecast_forecast_tomorrow |
Your Solcast forecast entity |
| Get battery SOC | sensor.byd_battery_box_premium_hv_state_of_charge |
Your battery's state-of-charge sensor |
| Get min SOC cutoff | sensor.byd_battery_box_premium_hv_soc_minimum |
Your battery's minimum SOC sensor |
The flow fails closed (writes safe_export = 0, zone: "error") both when a required sensor's state is unknown/unavailable and when a "Get …" node's entity ID doesn't exist at all (mistyped, renamed, or deleted). The latter case is handled by the "Catch sensor read errors" node, scoped to the five "Get …" nodes plus the model function node — any thrown error is caught and routed through the same fail-closed writer path, so the HA helpers always get a fresh zone: "error" result instead of holding a stale value from the previous run. Still double-check the five entity IDs above against Developer Tools → States after import.
5. Update the battery capacity in the "Four-zone model" function node. Near the top, change BATTERY_KWH to match your battery's usable capacity in kWh:
const BATTERY_KWH = 13.8; // ← replace with your battery's usable capacity6. Deploy and test using the Manual trigger button. Check the debug sidebar for the full result object.
When you retrain (every few months), only the constants at the top of the "Four-zone model" function node need updating.
This is more than a handful of values now: the heating and cooling zone coefficients (b0/b1/b2 each, plus their temp-only fallback b0/b1 pairs), the two P95 buffer values, the WARM and MILD empirical percentile tables (four values each), and the four-value confidence buffer-scale ladder (CONF).
tests/test_sync.py enforces that these stay identical to src/model.py, config/config.yaml, and tests/conftest.py — run pytest after editing either side to confirm they still agree.
The Node-RED flow writes the result to input_text.safe_export_detail. You can expose this as a proper HA sensor with full attribute support using a template sensor in your configuration.yaml:
template:
- trigger:
- trigger: state
entity_id: input_text.safe_export_detail
sensor:
- name: "Overnight Forecast Safe Power Export"
unique_id: overnight_forecast_safe_power_export
unit_of_measurement: "Wh"
device_class: energy
state_class: measurement
variables:
j: >
{% set raw = states('input_text.safe_export_detail') %}
{% set parsed = raw | from_json(default=None) %}
{{ parsed }}
state: "{{ j.p75 if j else 'unknown' }}"
attributes:
zone: "{{ j.zone if j else 'unknown' }}"
temp: "{{ j.temp if j else 'unknown' }}"
soc: "{{ j.soc if j else 'unknown' }}"
avail_kwh: "{{ j.avail_kwh if j else 'unknown' }}"
p50: "{{ j.p50 if j else 'unknown' }}"
p75: "{{ j.p75 if j else 'unknown' }}"
p90: "{{ j.p90 if j else 'unknown' }}"
p95: "{{ j.p95 if j else 'unknown' }}"
at: "{{ j.at if j else 'unknown' }}"The sensor's state is P75 (a reasonable default for most nights). All four confidence levels and the full prediction context are available as attributes. Change j.p75 in the state: line to j.p90 if you prefer a more conservative default.
A tile card that surfaces all four confidence levels at once:
type: tile
grid_options:
columns: full
entity: sensor.overnight_forecast_safe_power_export
name: Safe Export Wh
icon: mdi:chart-bell-curve
show_entity_picture: false
hide_state: false
state_content:
- p50
- p75
- p90
- p95
vertical: false
features_position: bottomRead the values from HA and call predict() from the command line:
.venv\Scripts\python -c "
from pathlib import Path
from src.config import load_config
from src.model import PredictInputs, predict
cfg = load_config(Path('config/config.yaml'))
result = predict(PredictInputs(
soc_at_6pm=85.0, # live battery SoC at 6pm (%)
bom_temp_mean=10.5, # sensor.overnight_forecast_temp_mean
bom_humidity_mean=87.0, # sensor.overnight_forecast_humidity_mean
solcast_forecast_tomorrow_wh=18000, # sensor.solcast_pv_forecast_forecast_tomorrow
min_soc=0.10, # battery min SoC setting (0.20 in storm mode)
confidence=0.90,
), cfg)
print(f'Safe export: {result.safe_export_wh:.0f} Wh ({result.safe_export_wh/1000:.2f} kWh)')
print(result.reasoning)
"Or from a Python script/REPL:
from pathlib import Path
from src.config import load_config
from src.model import PredictInputs, predict
cfg = load_config(Path("config/config.yaml"))
result = predict(PredictInputs(
soc_at_6pm=85.0, # live battery SoC at 6pm (%)
bom_temp_mean=10.5,
bom_humidity_mean=87.0,
solcast_forecast_tomorrow_wh=18000,
min_soc=0.10,
confidence=0.90,
), cfg)
print(f"Safe export: {result.safe_export_wh:.0f} Wh")
print(f"Zone: {result.zone}, model: {result.model_variant}")
print(f"Predicted consumption: {result.predicted_consumption_kwh:.1f} kWh "
f"+ {result.error_buffer_kwh:.1f} kWh buffer")
print(result.reasoning)- Python 3.11+
- A Home Assistant installation with at least 12 months of recorded statistics for the sensors listed in
docs/DATASET.md - SQLite (bundled with Python; no separate install needed)
- For Phase 2 onwards: Solcast PV forecasting integration in HA, plus a weather forecast integration
This is a personal infrastructure project built heavily with Claude AI. The design docs (DECISIONS.md, DATASET.md, SPEC.md) and the CLAUDE.md standing instructions are written so that an AI agent can pick up the codebase cold — if you fork this and want to adapt it to your own hardware, that's the intended path.
Issues are unlikely to get personal attention. If something is broken or unclear, your best bet is to fork, use Claude Code (or similar) to work through the adaptation, and iterate from there. Pull requests that fix bugs or improve the documentation are welcome, but support requests for getting it running on different hardware won't be addressed.
MIT — see LICENSE.
Kept last so cat README.md leaves it on screen.
# Incremental extraction — the normal path after refreshing the HA DB
.venv/Scripts/python -m src.extract data/home-assistant_v2.db
# Full rebuild — only when extraction methodology changed
.venv/Scripts/python -m src.extract data/home-assistant_v2.db --rebuild
# Economic backtest (writes tools/backtest_report.{html,json})
.venv/Scripts/python -m tools.backtest
# Commit gate: lint + tests, both must pass
.venv/Scripts/python -m ruff check .
.venv/Scripts/python -m pytest