There are several changes that are needed for how Metrics work:
- Optimisation consistency - all metrics should be normalised between 0 and 1, and defined in such a way that they should be minimised.
- Replacing aggregates with display metrics - currently, we don't use the aggregate part of metrics, and a more interpretable component would be more useful. This means that the Report dataclass should be changed to hold a
target field for optimisation, and a result field (feel free to suggest a more appropriate name) for tabulating/plotting.
- Refactoring BayesOpt to support displaying metrics. This means that the bayesopt loop needs to be explicit, so we can use intermediate results from the simulator to create the final report.
There are several changes that are needed for how Metrics work:
targetfield for optimisation, and aresultfield (feel free to suggest a more appropriate name) for tabulating/plotting.