Skip to content

Add script for aggregating annual load curve for _sb release #494

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
alexhyunminlee wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Add script for aggregating annual load curve for _sb release #494

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
11c4563
Add script for single bldg aggregation
alexhyunminlee Jul 29, 2026
0caecf1
Add context doc
alexhyunminlee Jul 29, 2026
acd614f
Add annual aggregation
alexhyunminlee Jul 30, 2026
7ce717f
Finish updating tariff mapper and re run for CT
alexhyunminlee Jul 31, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions context/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,8 @@ ResStock, EIA, NYISO, ISO-NE: where data comes from, how to read and prepare it.
| File | Purpose |
| -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| resstock_data_preparation_run_order.md | Run order for ResStock `_sb` release prep: recommended `main.py` invocation (single command covers fetch → metadata → approx → MF adj → utility assignment → monthly loads → upload), plus legacy individual Justfile step reference for debugging |
| resstock_sb_release_pipeline_main_py.md | Unified `data/resstock/main.py` pipeline: step-by-step reference for all implemented steps (2b through 2d), constants (`_SB_EXCLUDED_FILE_TYPES`, `sb_file_types`, column sets), config.yaml, state_configs.yaml (`SUPPORTED_UTILITY_STATES`, polygon filenames, per-state settings), sample mode, `path_raw` vs `path_sb` data flow, file types included/excluded in `_sb`, how to expand `_sb` scope |
| resstock_sb_release_pipeline_main_py.md | Unified `data/resstock/main.py` pipeline: step-by-step reference for all implemented steps (2b through 2d), constants (`SB_CLONE_EXCLUDED_FILE_TYPES`, `sb_file_types`, column sets), config.yaml, state_configs.yaml (`SUPPORTED_UTILITY_STATES`, polygon filenames, per-state settings), sample mode, `path_raw` vs `path_sb` data flow, file types included/excluded in `_sb`, how to expand `_sb` scope |
| resstock_sb_annual_load_curves.md | `_sb` `load_curve_annual` from modified hourly (combined with monthly in `aggregate_loads`): one hourly read → monthly write + annual row; keep/drop/aggregate decisions; upgrade 01–05 savings; `upgrade_name`; energy_delivered 4× bug (bsf before 1.6.6); pipeline step 2d |
| resstock_lmi_metadata_guide.md | ResStock 2024.2 parquet metadata: columns for LMI tier assignment, FPL/SMI, income |
| parquet_reads_local_vs_s3.md | Reading Parquet from local disk vs S3: per-GET overhead, file discovery, Hive filters vs path construction, best practices for whole-state and per-utility reads |
| polars_laziness_and_validation.md | Polars LazyFrame best practices: when to collect, runtime data-quality asserts vs laziness/streaming, strategies for small and large data |
Expand All @@ -145,7 +146,7 @@ ResStock, EIA, NYISO, ISO-NE: where data comes from, how to read and prepare it.
| ny_genability_charge_fetch_map.md | Exhaustive charge-level table for NY Genability tariffs: tariffRateId, fetch_type, variableRateKey, master_charge, decision; used to fetch 2025 monthly rates from Arcadia API for top-up implementation. |
| psegli_revenue_requirement_estimation.md | PSEG-LI revenue requirement estimation: why no rate case, TOU overcount problem, bill-proportional method from LIPA budget, charge classification, implementation via `estimate_psegli_rr.py`. |
| tariff_generation_pipeline.md | Tariff generation pipeline: Genability → monthly_rates YAML → flat and default-structure URDB v7 JSONs. Covers fetch_monthly_rates.py, charge classification, monthly_rates YAML structure, create_flat_tariffs.py (top-down from RR), create_default_structure_tariffs.py (bottom-up from filed rates), BASE_TARIFF_PATTERN, Justfile wiring (all-pre), CAIRO calibration, and why flat vs default rates differ. |
| md_tariff_fetch.md | MD electric and gas tariff fetching pipeline: how `fetch_electric_tariffs_genability.py` produces URDB JSONs for all 10 MD electric utilities directly from Arcadia (bypassing the NY/RI monthly_rates YAML pipeline), gas fetching via RateAcuity + Chesapeake county split + UGI PGC, three `tariff_fetch` bug patches (empty-band applicability, calculationFactor fold, duplicate inline+rider dedup), verified rate values per utility, and remaining work (UGI/Easton commodity merge). |
| md_tariff_fetch.md | MD electric and gas tariff pipeline: Arcadia electric URDBs, RateAcuity gas tariffs, Chesapeake county and RES-1/RES-2 mapping from `_sb` annual therms, UGI/Easton commodity rates, `tariff_fetch` bug patches, and verified rate values. |

### code/marginal_costs/

Expand Down
9 changes: 9 additions & 0 deletions context/code/data/md_tariff_fetch.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,15 @@ based on **annual consumption** reviewed at calendar year-end:
- **RES-1** (≤ 150 therms/year): lower volumetric rate, lower fixed charge (`$8/month`)
- **RES-2** (> 150 therms/year): lower volumetric rate but higher fixed charge (`$10/month`)

`utils/pre/gas_tariff_mapper.py` derives annual therms from
`out.natural_gas.total.energy_consumption.kwh` in the generated Switchbox
`res_2024_amy2018_2_sb/load_curve_annual` dataset (÷ 29.3001 kWh/therm). The shared
`rate_design/hp_rates/Justfile` passes the `_sb` annual directory for the requested
upgrade. This is intentional: `_sb` annual totals are rebuilt from modified `_sb`
hourly curves, so Chesapeake tariff classes reflect non-HP approximation and any
other Switchbox load adjustments. The mapper does not fall back to raw NREL annual
loads when the `_sb` annual directory is missing.

The "lower rate, higher fixed charge" pattern at higher usage is consistent with Maryland
rate design practice: high-use customers (predominantly space-heating customers) pay more
in fixed charges to reduce volumetric cross-subsidization.
Expand Down
Loading