Optional BA-level project aggregation via --aggregate_projects

[allisonmcampbell]

Summary

• Adds --aggregate_projects flag to 7 data toolkit scripts, allowing users to aggregate all generators to the BA-technology level instead of the default disaggregated approach
• Default behavior is unchanged: thermal/storage use plant-level names, var-gen/hydro are BA-aggregated
• When enabled, all projects use COALESCE(agg_project, gridpath_technology) || '_' || balancing_authority_code_eia with GROUP BY, producing names like Gas_CC_CISO instead of individual plant IDs
• Includes COALESCE fix to AGG_PROJECT_NAME_STR so technologies without agg_project in the key file fall back to gridpath_technology

Motivation

The default disaggregated approach produces individual plant-level projects for most of the fleet. For a full WECC system, this creates a very large optimization problem that takes too long to solve on a local machine. Aggregating to the BA-technology level makes a full-system test case tractable while preserving the technology mix and capacity at each BA.

Scripts modified

• project_data_filters_common.py — COALESCE in AGG_PROJECT_NAME_STR
• eia860_to_project_portfolio_input_csvs.py
• eia860_to_project_specified_capacity_input_csvs.py
• eia860_to_project_load_zone_input_csvs.py
• eia860_to_project_fixed_cost_input_csvs.py
• eia860_to_project_availability_input_csvs.py
• eia860_to_project_opchar_input_csvs.py
• manual_adjustments.py

Test plan

• All 3 data toolkit tests pass locally (default behavior, no aggregation)
• No test output CSV changes
• CI should pass on all OS/Python matrix combinations

🤖 Generated with Claude Code