In many cases, Smoke could potentially run multiple tests concurrently. Lots of the examples (e.g. the calculator) in the fixtures directory would work.
Because Smoke spawns programs that might do anything, I think this should be opt-in, behind a switch that sets the maximum number of concurrent tests (e.g. --concurrency=4).
Often, most of your tests can be run concurrently, but a few may not be able to. For example, tests that mutate files used by other tests are probably way out. This means that there needs to be a way to mark certain tests as serial, potentially with a YAML switch such as serial: true or exclusive: true.
I expect there's certain tests that should never be run concurrently. For example, if two tests refer to the same directory in a revert: declaration (or one refers to a descendant of another), those tests should never be run at the same time. However, they might be able to run in parallel with other tests. A simple implementation may not want to bother with this, though, and just force all tests with a revert: declaration to run serially.
In many cases, Smoke could potentially run multiple tests concurrently. Lots of the examples (e.g. the calculator) in the fixtures directory would work.
Because Smoke spawns programs that might do anything, I think this should be opt-in, behind a switch that sets the maximum number of concurrent tests (e.g.
--concurrency=4).Often, most of your tests can be run concurrently, but a few may not be able to. For example, tests that mutate files used by other tests are probably way out. This means that there needs to be a way to mark certain tests as serial, potentially with a YAML switch such as
serial: trueorexclusive: true.I expect there's certain tests that should never be run concurrently. For example, if two tests refer to the same directory in a
revert:declaration (or one refers to a descendant of another), those tests should never be run at the same time. However, they might be able to run in parallel with other tests. A simple implementation may not want to bother with this, though, and just force all tests with arevert:declaration to run serially.