Add instructions for running unit tests

[dr-joe-wirth]

These instructions should be included somewhere on the repo. I suggest including them in the contributing doc as well.

This is part of the review at openjournals/joss-reviews#8065

[ira-zibbu]

That would be useful. I will add instructions in the documentation and in contributing.md.

[ira-zibbu]

Hi @dr-joe-wirth ,

I have added instructions for testing to contributing.md

[dr-joe-wirth]

i ran into an issue with your test environment file as described in contributing.md

% conda env create -f seabreeze/test/unit_tests/environment.yml --name seabreeze_test
Channels:
 - conda-forge
 - bioconda
 - defaults
Platform: osx-arm64
Collecting package metadata (repodata.json): done
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

  - _libgcc_mutex==0.1=conda_forge

Current channels:

  - https://conda.anaconda.org/conda-forge
  - https://conda.anaconda.org/bioconda
  - https://repo.anaconda.com/pkgs/main
  - https://repo.anaconda.com/pkgs/r

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

Looking at this env file, it looks like you did conda env export to create it. This is not ideal, especially from a maintenance perspective. I would suggest minimizing the lines you use in your yaml and only include what is absolutely necessary for your tests and let conda solve the rest.

[ira-zibbu]

Hi @dr-joe-wirth,

Thanks for bringing this issue to my attention. I have updated the environment.yml to be minimally specified.

[dr-joe-wirth]

When I try creating the environment I get the following error:

PackagesNotFoundError: The following packages are not available from current channels:

  - isescan=1.7.3

After some digging, I believe that isescan is not compatible with macos arm64 (M1/M2/M3 chips).

[ira-zibbu]

Sorry to hear that @dr-joe-wirth . Do you have suggestions for how we can move forward?

[dr-joe-wirth]

Do you have suggestions for how we can move forward?

perhaps indicate that this will not run on macOS in your contributing doc?

[dr-joe-wirth]

TLDR

I ran the unit tests on a AWS linux server. your environment file is incomplete; matplotlib needs to be included as a dependency. see below for th

failed tests

I ran the following commands

conda env create --name seabreeze-test --file ./seabreeze/test/unit_tests/environment.yml
conda activate seabreeze-test
pytest ./seabreeze/test/unit_tests/*py

this failed because it couldn't find files. i assumed that your unit tests had hardcoded a requirement that cwd is ./seabreeze, so i did the following:

cd ./seabreeze
pytest ./test/unit_tests/*py

now this ran, but there were several failures:

===================================== test session starts =====================================
platform linux -- Python 3.9.23, pytest-8.3.3, pluggy-1.6.0
rootdir: /local/home/joewirth/seabreeze-review/seabreeze
configfile: pytest.ini
collected 13 items                                                                            

test/unit_tests/fast_stats_test.py .                                                    [  7%]
test/unit_tests/find_IS_elements_test.py F                                              [ 15%]
test/unit_tests/find_reindex_bases_test.py ..                                           [ 30%]
test/unit_tests/mask_IS_elements_test.py .                                              [ 38%]
test/unit_tests/nucmer_alignment_test.py .                                              [ 46%]
test/unit_tests/plotsr_test.py .F                                                       [ 61%]
test/unit_tests/reindex_assembl_test.py .                                               [ 69%]
test/unit_tests/rename_contigs_test.py .                                                [ 76%]
test/unit_tests/replichore_arms_analyse_test.py ..                                      [ 92%]
test/unit_tests/syri_call_variants_test.py .                                            [100%]

here are the relevant snippets of the failure messages

========================================== FAILURES ===========================================
_________________________________________ test_isecan _________________________________________

... a lot of text ...

OSError: libssw.so: cannot open shared object file: No such file or directory
______________________________________ test_plotsr_plots ______________________________________

... a lot of text ...
 
ModuleNotFoundError: No module named 'matplotlib'
=================================== short test summary info ===================================
FAILED test/unit_tests/find_IS_elements_test.py::test_isecan - subprocess.CalledProcessError: Command '['isescan.py', '--seqfile', 'test/data/Anc-_0gen_R...
FAILED test/unit_tests/plotsr_test.py::test_plotsr_plots - AssertionError: assert False
================================ 2 failed, 11 passed in 3.11s =================================

fixing the problem

I deleted the conda environment, and changed the environment file to the one below:

name: seabreeze_test
channels:
  - conda-forge
  - bioconda
dependencies:
  - biopython=1.84
  - blast=2.16.0
  - isescan=1.7.3
  - matplotlib         # this is the only line i added
  - matplotlib-base=3.3.4
  - mummer=3.23
  - numpy=1.26.4
  - pandas=2.0.3
  - pytest=8.3.3
  - syri=1.6.3 

then i rebuilt the conda environment and ran the tests. now all tests passing.

[dr-joe-wirth]

none of the tests in the test directory passed:

conda activate seabreeze-test    # this was built with the modified file in my previous comment
pytest ./test/*.py

every test failed:

=================================== short test summary info ===================================
FAILED test/seabreeze_batch_all_test.py::test_batch_run_all_masked - AssertionError: Command failed with error: None
FAILED test/seabreeze_batch_all_test.py::test_batch_run_all_unmasked - AssertionError: Command failed with error: None
FAILED test/seabreeze_batch_workflow_test.py::test_batch_analyse_genome_sizes - AssertionError: Command failed with error: None
FAILED test/seabreeze_batch_workflow_test.py::test_batch_predict_IS_elements - AssertionError: Command failed with error: None
FAILED test/seabreeze_batch_workflow_test.py::test_batch_predict_structural_variants_unmasked - AssertionError: Command failed with error: None
FAILED test/seabreeze_batch_workflow_test.py::test_batch_predict_structural_variants_masked - AssertionError: Command failed with error: None
FAILED test/seabreeze_batch_workflow_test.py::test_batch_predict_replichore_balance_unmasked - AssertionError: Command failed with error: None
FAILED test/seabreeze_batch_workflow_test.py::test_batch_predict_SV_mechanism_unmasked - AssertionError: Command failed with error: None
FAILED test/seabreeze_batch_workflow_test.py::test_batch_predict_SV_mechanism_masked - AssertionError: Command failed with error: None
FAILED test/seabreeze_batch_workflow_test.py::test_batch_annotate_SV_regions_unmasked - AssertionError: Command failed with error: None
FAILED test/seabreeze_batch_workflow_test.py::test_remove_test_data - FileNotFoundError: [Errno 2] No such file or directory: 'test/seabreeze_batch_test/03_rein...
FAILED test/seabreeze_run_all_test.py::test_run_run_all_masked - AssertionError: Command failed with error: None
FAILED test/seabreeze_run_all_test.py::test_run_run_all_unmasked - AssertionError: Command failed with error: None
FAILED test/seabreeze_run_workflow_test.py::test_run_analyse_genome_sizes - AssertionError: Command failed with error: None
FAILED test/seabreeze_run_workflow_test.py::test_run_predict_IS_elements - AssertionError: Command failed with error: None
FAILED test/seabreeze_run_workflow_test.py::test_run_predict_structural_variants_unmasked - AssertionError: Command failed with error: None
FAILED test/seabreeze_run_workflow_test.py::test_run_predict_structural_variants_masked - AssertionError: Command failed with error: None
FAILED test/seabreeze_run_workflow_test.py::test_run_predict_replichore_balance_unmasked - AssertionError: Command failed with error: None
FAILED test/seabreeze_run_workflow_test.py::test_run_predict_replichore_balance_masked - AssertionError: Command failed with error: None
FAILED test/seabreeze_run_workflow_test.py::test_run_predict_SV_mechanism_unmasked - AssertionError: Command failed with error: None
FAILED test/seabreeze_run_workflow_test.py::test_run_predict_SV_mechanism_masked - AssertionError: Command failed with error: None
FAILED test/seabreeze_run_workflow_test.py::test_run_annotate_SV_regions_unmasked - AssertionError: Command failed with error: None
FAILED test/seabreeze_run_workflow_test.py::test_remove_test_data - FileNotFoundError: [Errno 2] No such file or directory: 'test/seabreeze_run_test/03_reinde...
===================================== 23 failed in 0.47s ======================================

[ira-zibbu]

Hi @dr-joe-wirth,

Sorry to hear about the trouble you have been having.

• Regarding ISEScan not running on arm64 MacOS: I have noted this issue on the documentation, as well the contributing.md to give users a heads up.

• Regarding the missing matplotlib dependency: I currently had only specified matplotlib-base=3.3.4 as a dependency in the test environment. The unit tests ran fine on my system, as well as on GitHub Actions. I have updated the environment recipe to instead include matplotlib of matplotlib-base=3.3.4. This should resolve that missing dependency issue you encountered.

• Regarding failed tests: The step you ran was the workflow tests (as opposed to the unit tests). As specified in contributing.md, these tests need to be run from the seabreeze execution conda environment and not the seabreeze_test directory built for the unit tests. The seabreeze execution environment contains additional important dependencies. I have updated contributing.md to clarify this better. Try:

conda activate seabreeze #assuming this is what your seabreeze execution env is named
pytest test/*.py

I have also updated contributing.md to indicate that the tests should be run from the seabreeze root directory in order for the test input file locations to be correctly found.

Let me know if you face any other issues, have questions or suggestions.

[dr-joe-wirth]

i installed seabreeze using:

conda create --name seabreeze-from-conda
conda activate seabreeze-from-conda
conda install seabreeze-genomics -c conda-forge -c bioconda

then i tried running the tests:

conda activate seabreeze-from-conda
cd ./seabreeze
pytest ./test/*py

and everything passed.

[dr-joe-wirth]

one note:

The gitignore file currently attempts to specify every possible path to __pycache__, but it does not catch all of them. Instead just add __pycache__

ie replace this:

src/seabreeze/__pycache__/
src/seabreeze/plotsr/__pycache__
src/__pycache__/
src/seabreeze/plotsr/plotsr/scripts/__pycache__

with this:

__pycache__

Running the tests creates a bunch of __pycache__ directories and it would be nice if git ignored them.

[ira-zibbu]

Hi @dr-joe-wirth I have updated the .gitignore file.