ENH Fix running the test suite#44
Conversation
I.e., running `benchopt test .` now works.
…ations for Chronos-2 without CUDA
… linter, formatter, ...)
|
The one deprecation warning will be fixed in the next |
Their download hosts (timeseriesclassification.com returns 401; mitdb source rate-limits) block CI runners, but both load fine locally. Replace the misleading xfail with a CI-only pytest.skip applied to every test that downloads them (get_data, benchmark_objective, solver_run), and restore mitdb to the test datasets so it is still exercised locally. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
the issue with UCR is that we don't cache it yet because the data is not downloaded in a controlable directory. In the meantime, we could cache an extra directory: The failure for now is because we test that every solver can at least run with one dataset. As for classification, we rely on UCR and We could also skip tests for mantis for now as it is the only classif only model |
Skipping ucr in CI left Mantis (the only classification-only solver) with no
runnable dataset, which benchopt rejects ("ensure at least one configuration
runs"). Add a tiny network-free synthetic classification dataset and include it
in the Objective test_config so the classification path always has a dataset
that runs offline in CI.
The data is multivariate (3 channels: two targets + one covariate); the class
is encoded in the trend (rising/falling/flat) of one target channel. Verified
with a full `benchopt test .` (CI mode): 103 passed, 31 skipped, 0 failed, with
test_solver_run[mantis-dummy-classification] passing on CPU.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
@tomMoral Even better caching might not fully resolve this issue, e.g., if the cache is empty (initially or after changing environments). I guess a dummy/simulated dataset is the safer choice. I added one. |
2 participants
Running
benchopt test .andpython -m pytestnow fully work with zero failures. The latter now runs more tests than before.Also updates the CI environment to never versions.