Add new-format costs_connection templating (VRE new entrants)#107
Add new-format costs_connection templating (VRE new entrants)#107EllieKallmier wants to merge 5 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests.
🚀 New features to boost your workflow:
|
nick-gorman
left a comment
nick-gorman
left a comment
There was a problem hiding this comment.
Looks good Ellie. Some nice extensions to the fuzzy maping infrastructure. And very much apprecaite the chunking into a smallish PR! Just some tiny comments from me.
| "renewable_energy_zones": renewable_energy_zones, | ||
| "flow_path_transfer_capability": flow_path_transfer_capability, | ||
| "initial_transmission_limits": initial_transmission_limits, | ||
| "connection_cost_forecast_wind_and_solar": csv_str_to_df( |
There was a problem hiding this comment.
Why no check for corresponding output?
There was a problem hiding this comment.
whoops yep not intentional! It'll return an empty costs df so I just blanked on checking that it actually exists. Will fix up :)
| lambda v: match_dict[v] if match_dict[v] is not None else v | ||
| ) | ||
| _log_fuzzy_match(name_series, matched, task_desc) | ||
| _warn_unmatched_fuzzy_values(name_series, match_dict, task_desc) |
There was a problem hiding this comment.
Just a thought really. But if canonicalisation fails, this kind of implies that dataset produced will contain in consistencies and maybe shouldn't be used as is? Or maybe I'm missing something. But I guess validation would pick this up later.
There was a problem hiding this comment.
yeah I think that's right, and I was thinking validation would pick it up but prob better to just raise as it happens (or drop?). The case that was in my mind is in the connection costs table for non-vre, 'BOTN - Cethana' has it's own entry in what is otherwise only standard 'technology'-labelled entries - so thought I'd just leave unmatched/unconverted so we could handle as a special case later. But I don't know how common that kind of case might be (and assume probably not very) so yeah probably better to handle known special cases before canonicalisation and just error out if canonicalisation fails. Will update!
| ].reset_index(drop=True) | ||
|
|
||
|
|
||
| def _enforce_numeric_cols(df: pd.DataFrame, numeric_cols: list[str]) -> pd.DataFrame: |
There was a problem hiding this comment.
Good for me to remember this is here. Could be extracted to helpers down the track.
…cal fuzzy matching
EllieKallmier
force-pushed
the
new-format-connection-cost-table
branch
from
822d517 to
fe60215
Compare
2 participants
Summary
Scope note:
Connection cost templating for non-VRE technologies is split out to come in a follow-up PR. The reasons:
The follow-up PR should build directly on this one. There are a few other things like more assertive canonicalisation to come as the rest of generator/storage templater updates built out, but for the sake of smaller chunks I'm leaning on the feature flag approach to more incrementally implement stuff - hope that's ok, and very happy to change that workflow/interpretation if it doesn't work long term!