Skip to content

[Combined] Improve testing framework #51

Description

@SFrijters
  • Allow to better specify combinations of parameters in json.
  • Allow to specify differentiate between coverage tests, Travis CI runnable tests (as short as possible) and physics tests (to be manually checked). Normally these would be the same simulations, with the physics tests running as long as necessary to obtain physically correct results, the automated tests running only long enough such that diffing makes sense, and the coverage tests running ideally for only one timestep.
  • Allow array token replacement in testing scripts (e.g. properly expand %lb.force.gcc[0][1]%).
  • Add text diff, ignoring comment lines.
  • Add option to restore from checkpoint in tests.
  • Rewrite or at least go through once and clean up.
  • Make use a class for each test so they can be easily passed around?
  • Or maybe even change to D? Create json spec directly from class / struct impl.

For Travis testing:

  • Tests should not last longer than a few seconds.
  • Change reference data to GDC when available, and change the lax/strict flags accordingly.
  • ~~Add warning if a test takes longer than ~30s.~~
  • Tests should be run serially and also parallel - or should some explicit parallelization tests be added?

Metadata

Metadata

Assignees

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions