A regression/history-check script would be helpful to keep in docs

[logan-nc]

This came up in #207.

Previously, I've been keeping compiled versions of all the releases on portal, and scripts like the ones in pypec could be used to compare a desired output (say DCON dW) across all the releases. Now, portal is deprecated. There are also reasons to change the DIII-D ideal example as outlined in #207. These kinds of things will continue to get in the way.... so maybe we should rely on a script that does the whole thing: checking out all the releases, compiling them, running some examples, and storing the desired outputs.

I think @matt-pharr has done something like this already - is it something you can annotate and add to the source for others to use?

[matt-pharr]

So let's make sure I understand this correctly. We want a script that:

• Lives in project_home/docs/
• Checks out every tagged release
• Runs specified examples
• Stores the outputs of these examples
• Maybe compares the outputs?

This is to be run by users of the code who want to check changes in the code outputs over time?

[logan-nc]

Correct. Is this something easy that you basically have in hand? It feels like you've been doing something similar with all your scripted tests of compilers.
I think storing / writing the results is sufficient → users can plot however they prefer.