Conversation
Will handle summaries differently.
RA Toolkit refactoring, enhancements, and additional tests
Support for handling large timepoint queries and avoiding SQLite limits.
* Add indexes for common lookups (this changes the order of inputs, so some small changes to test example objective function values are observed) * CTE-based queries for profiles
…it (#1359) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Remove unnecessary setup_results_import function and all calls * Refactor db to conn for consistency, add commit_immediately flag to spin_on_db_lock * Commit on closing DB, not on spin_on_db_lock * Commit after each CSV structure row import
Avoids 'too many open files' error with large simulations.
Incomplete subproblems are now evenly distributed across worker processes instead of being discovered after parallelization.
* Minor SQL cost query fixes 1. in costs.py, `INSERT INTO results_project_costs_operations_agg` lists columns as `(... variable_om_cost, fuel_cost, ...)` but selects `SUM(fuel_cost) AS fuel_cost` first and `SUM(variable_om_cost) AS variable_om_cost` second. SQLite's INSERT is positional so the columns are stored swapped: the column named `variable_om_cost` actually holds period-aggregated fuel cost and vice versa. Reorder the SELECT to match the column list. 2. `results_project_costs_capacity_agg` writes `spinup_or_lookahead = 0` for non-spinup rows while `results_project_costs_operations_agg` (tx as well) write NULL, so the view's `a.spinup_or_lookahead = b.spinup_or_lookahead` join fails (NULL = 0 is UNKNOWN) and every operations / tx row is silently dropped. Wrap both sides in `COALESCE(..., 0)` so the condition is robust to either convention. 3. The 4th LEFT JOIN in `results_project_costs_capacity_agg` aliases `results_transmission_hurdle_costs_by_timepoint_agg` as `e` but its ON clause references `d.scenario_id` etc., so the join is effectively against the previous table again rather than the aliased one. Switch the references to `e.`. Issues found with help from Claude Opus 4.7. * Remove merge on period from timepoint hurdle rate table Views `results_costs_by_period_load_zone` and `results_costs_by_period` aggregate by period and included either joins or summation over `results_transmission_hurdle_costs_by_timepoint_agg`. However, this view does not include period. This was silently ignored before and is now removed for clarity.
…ojects (#1360) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.