cgat-0.7.4 pip install : FileNotFoundError: [Errno 2] No such file or directory: 'requires.txt'

[ns-rse]

Further to cgat-core #177 I'm still having trouble installing cgat on my system.

NB I've read the install instructions and have seen the preferred approach is to use Conda environments but would prefer to use plain virtual environments because of the recent furore and potential licensing issues associated with Conda.

In #177 I attempted to abstract out the problem from the pyproject.toml I mentioned and manually install packages and dependencies. I continue with that approach here.

❱ pip install cgat     
Collecting cgat
  Downloading cgat-0.7.4.tar.gz (408 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [21 lines of output]
      <string>:52: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
      WARNING: depency check for UCSC tools(wigToBigWig) failed, error 127
      WARNING: depency check for bedtools(bedtools) failed, error 127
      Traceback (most recent call last):
        File "/home/neil/.virtualenvs/isoslam/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/neil/.virtualenvs/isoslam/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/neil/.virtualenvs/isoslam/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-8nf_pluf/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 333, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-8nf_pluf/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 303, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-8nf_pluf/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 319, in run_setup
          exec(code, locals())
        File "<string>", line 143, in <module>
      FileNotFoundError: [Errno 2] No such file or directory: 'requires.txt'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

Downloading the source, extracting and attempting to install...

https://files.pythonhosted.org/packages/6c/b3/ab52100e736cad58ebebbf67b04322345f2139d36ee8d0637737eb4b8a6f/cgat-0.7.4.tar.gz
tar xzvf cgat-0.7.4.tar.gz
cd cgat
pip install .
Processing /home/neil/work/git/hub/cgat-developers/cgat-0.7.4
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [21 lines of output]
      <string>:52: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
      WARNING: depency check for UCSC tools(wigToBigWig) failed, error 127
      WARNING: depency check for bedtools(bedtools) failed, error 127
      Traceback (most recent call last):
        File "/home/neil/.virtualenvs/isoslam/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/neil/.virtualenvs/isoslam/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/neil/.virtualenvs/isoslam/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-s18pqlf2/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 333, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-s18pqlf2/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 303, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-s18pqlf2/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 319, in run_setup
          exec(code, locals())
        File "<string>", line 143, in <module>
      FileNotFoundError: [Errno 2] No such file or directory: 'requires.txt'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

requires.txt

Looking for requires.txt in the package files.

grep -R "requires\.txt" -B5 -A5
setup.py-138-    r'^-e (?P<link>.*).+#(?P<package>.+)-(?P<version>\d(?:\.\d)*)$')
setup.py-139-install_requires = []
setup.py-140-dependency_links = []
setup.py-141-
setup.py-142-for requirement in (
setup.py:143:        l.strip() for l in open('requires.txt') if not l.startswith("#")):
setup.py-144-    match = REPO_REQUIREMENT.match(requirement)
setup.py-145-    if match:
setup.py-146-        assert which(match.group('vcs')) is not None, \
setup.py-147-            ("VCS '%(vcs)s' must be installed in order to "
setup.py-148-             "install %(link)s" % match.groupdict())

Searching for requires.txt

find . -name "requires.txt"

The file doesn't exist.

Checking cgat-0.7.2

cd ..
wget https://files.pythonhosted.org/packages/ef/c4/51ff925472539041ad23edf0ebf3b5f3fa496c716f13f22f85f88254c7ee/cgat-0.7.2.tar.gz
tar xzvf cgat-0.7.2.tar.gz
find . -name "requires.txt"
./requires.txt

The file is present in the previous release, but its an empty file...

ls -l requires.txt 
.rw-r--r-- neil neil 0 B Fri Apr  5 11:17:00 2024  requires.txt
 11:43:26 am GMT  percentage:  neil  ~/work/git/hub/cgat-developers/cgat-0.7.2  
cat requires.txt

Workaround

Add an empty file to cgat-0.7.4 and its possible to pip install ., but it fails further along

cd ../cgat-0.7.4
touch requires.txt
pip install .
Processing /home/neil/work/git/hub/cgat-developers/cgat-0.7.4
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: cgat
  Building wheel for cgat (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for cgat (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [335 lines of output]
      <string>:52: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
      /tmp/pip-build-env-9uh7wu05/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py:261: UserWarning: Unknown distribution option: 'test_suite'
        warnings.warn(msg)
      WARNING: depency check for UCSC tools(wigToBigWig) failed, error 127
      WARNING: depency check for bedtools(bedtools) failed, error 127
      running bdist_wheel
      running build
      running build_py
      creating build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/Masker.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/__init__.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/AString.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/SetTools.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/Blat.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/Motifs.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/RLE.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/MEME.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/Histogram2D.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/Glam2Scan.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/Iterators.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/IndexedGenome.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/Genomics.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/makeGeneset.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/Tree.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/TreeTools.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/dictzip.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/SequenceProperties.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/Variants.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/Fastq.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/Stats.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/Bed.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/AGP.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/GTF.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/FastaIterator.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/Histogram.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/version.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/GFF3.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/MatrixTools.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/Intervals.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/VCF.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/IndexedFasta.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/cgat.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/RateEstimation.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/Glam2.py -> build/lib.linux-x86_64-cpython-312/cgat
      copying cgat/Sra.py -> build/lib.linux-x86_64-cpython-312/cgat
      creating build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bam2bam.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/fasta2table.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/fastq2tsv.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/__init__.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bam_compare_alignments.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/fasta2bed.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/gff2psl.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/diff_chains.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bed2table.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/gff2table.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/vcf_vs_vcf.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/fastq2table.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/split_file.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bam2bam_split_reads.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/data2histogram.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/gff32gtf.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/fasta2variants.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/vcf_stats.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/diff_gtf.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/genome_bed.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/cgat_script_template.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/fasta2stats.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/gtf2table.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/gff2bed.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/fastq2fasta.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/gtf2gff.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bed2bed.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/table2table.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bam2wiggle.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/beds2counts.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bed2graph.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/gff2histogram.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bams2bam.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bam2fastq.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/vcf2tsv.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/index2bed.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/fastq2tpm.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bed2fasta.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bam_vs_bam.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/diff_fasta.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/gff2gff.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/csvs2csv.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/vcf2vcf.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/table2stats.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/cgat_get_options.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/vcf_compare_phase.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/gtf2fasta.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bcl2fastq.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/gff2fasta.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/fastqs2fasta.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/diff_bed.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/fasta2vcf.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/split_fasta.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/fastas2fasta.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/tables2table.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/beds2beds.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/fastq2summary.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/gff2stats.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/diff_bam.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/gtf2gtf.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/split_gff.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bed2stats.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/vcfstats2db.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/extract_stats.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bam2bed.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bam_vs_gtf.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/fastq2fastq.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bam2window_stats.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bed_vs_bed.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bam2stats.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/fasta2fasta.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/fastqs2fastq.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bam2depth.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bed2gff.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bam_vs_bed.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/csv2csv.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/fastqs2fastqs.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/fasta2fastq.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/cgat_rebuild_extensions.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/fasta2kmercontent.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/gtfs2tsv.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/gtf2tsv.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/wig2bed.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bam2fasta.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/gff2coverage.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/index_fasta.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bam_pileup2tsv.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      copying cgat/tools/bam2geneprofile.py -> build/lib.linux-x86_64-cpython-312/cgat/tools
      creating build/lib.linux-x86_64-cpython-312/cgat/NCL
      copying cgat/NCL/__init__.py -> build/lib.linux-x86_64-cpython-312/cgat/NCL
      running egg_info
      writing cgat.egg-info/PKG-INFO
      writing dependency_links to cgat.egg-info/dependency_links.txt
      writing entry points to cgat.egg-info/entry_points.txt
      writing top-level names to cgat.egg-info/top_level.txt
      reading manifest file 'cgat.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      warning: no previously-included files matching '*' found under directory 'tests'
      adding license file 'LICENSE'
      writing manifest file 'cgat.egg-info/SOURCES.txt'
      /tmp/pip-build-env-9uh7wu05/overlay/lib/python3.12/site-packages/setuptools/command/build_py.py:220: _Warning: Package 'cgat.BamTools' is absent from the `packages` configuration.
      !!
      
              ********************************************************************************
              ############################
              # Package would be ignored #
              ############################
              Python recognizes 'cgat.BamTools' as an importable package[^1],
              but it is absent from setuptools' `packages` configuration.
      
              This leads to an ambiguous overall configuration. If you want to distribute this
              package, please make sure that 'cgat.BamTools' is explicitly added
              to the `packages` configuration field.
      
              Alternatively, you can also rely on setuptools' discovery methods
              (for example by using `find_namespace_packages(...)`/`find_namespace:`
              instead of `find_packages(...)`/`find:`).
      
              You can read more about "package discovery" on setuptools documentation page:
      
              - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
      
              If you don't want 'cgat.BamTools' to be distributed and are
              already explicitly excluding 'cgat.BamTools' via
              `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
              you can try to use `exclude_package_data`, or `include-package-data=False` in
              combination with a more fine grained `package-data` configuration.
      
              You can read more about "package data files" on setuptools documentation page:
      
              - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
      
      
              [^1]: For Python, any directory (with suitable naming) can be imported,
                    even if it does not contain any `.py` files.
                    On the other hand, currently there is no concept of package data
                    directory, all directories are treated like packages.
              ********************************************************************************
      
      !!
        check.warn(importable)
      /tmp/pip-build-env-9uh7wu05/overlay/lib/python3.12/site-packages/setuptools/command/build_py.py:220: _Warning: Package 'cgat.Components' is absent from the `packages` configuration.
      !!
      
              ********************************************************************************
              ############################
              # Package would be ignored #
              ############################
              Python recognizes 'cgat.Components' as an importable package[^1],
              but it is absent from setuptools' `packages` configuration.
      
              This leads to an ambiguous overall configuration. If you want to distribute this
              package, please make sure that 'cgat.Components' is explicitly added
              to the `packages` configuration field.
      
              Alternatively, you can also rely on setuptools' discovery methods
              (for example by using `find_namespace_packages(...)`/`find_namespace:`
              instead of `find_packages(...)`/`find:`).
      
              You can read more about "package discovery" on setuptools documentation page:
      
              - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
      
              If you don't want 'cgat.Components' to be distributed and are
              already explicitly excluding 'cgat.Components' via
              `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
              you can try to use `exclude_package_data`, or `include-package-data=False` in
              combination with a more fine grained `package-data` configuration.
      
              You can read more about "package data files" on setuptools documentation page:
      
              - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
      
      
              [^1]: For Python, any directory (with suitable naming) can be imported,
                    even if it does not contain any `.py` files.
                    On the other hand, currently there is no concept of package data
                    directory, all directories are treated like packages.
              ********************************************************************************
      
      !!
        check.warn(importable)
      /tmp/pip-build-env-9uh7wu05/overlay/lib/python3.12/site-packages/setuptools/command/build_py.py:220: _Warning: Package 'cgat.FastqTools' is absent from the `packages` configuration.
      !!
      
              ********************************************************************************
              ############################
              # Package would be ignored #
              ############################
              Python recognizes 'cgat.FastqTools' as an importable package[^1],
              but it is absent from setuptools' `packages` configuration.
      
              This leads to an ambiguous overall configuration. If you want to distribute this
              package, please make sure that 'cgat.FastqTools' is explicitly added
              to the `packages` configuration field.
      
              Alternatively, you can also rely on setuptools' discovery methods
              (for example by using `find_namespace_packages(...)`/`find_namespace:`
              instead of `find_packages(...)`/`find:`).
      
              You can read more about "package discovery" on setuptools documentation page:
      
              - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
      
              If you don't want 'cgat.FastqTools' to be distributed and are
              already explicitly excluding 'cgat.FastqTools' via
              `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
              you can try to use `exclude_package_data`, or `include-package-data=False` in
              combination with a more fine grained `package-data` configuration.
      
              You can read more about "package data files" on setuptools documentation page:
      
              - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
      
      
              [^1]: For Python, any directory (with suitable naming) can be imported,
                    even if it does not contain any `.py` files.
                    On the other hand, currently there is no concept of package data
                    directory, all directories are treated like packages.
              ********************************************************************************
      
      !!
        check.warn(importable)
      /tmp/pip-build-env-9uh7wu05/overlay/lib/python3.12/site-packages/setuptools/command/build_py.py:220: _Warning: Package 'cgat.VCFTools' is absent from the `packages` configuration.
      !!
      
              ********************************************************************************
              ############################
              # Package would be ignored #
              ############################
              Python recognizes 'cgat.VCFTools' as an importable package[^1],
              but it is absent from setuptools' `packages` configuration.
      
              This leads to an ambiguous overall configuration. If you want to distribute this
              package, please make sure that 'cgat.VCFTools' is explicitly added
              to the `packages` configuration field.
      
              Alternatively, you can also rely on setuptools' discovery methods
              (for example by using `find_namespace_packages(...)`/`find_namespace:`
              instead of `find_packages(...)`/`find:`).
      
              You can read more about "package discovery" on setuptools documentation page:
      
              - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
      
              If you don't want 'cgat.VCFTools' to be distributed and are
              already explicitly excluding 'cgat.VCFTools' via
              `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
              you can try to use `exclude_package_data`, or `include-package-data=False` in
              combination with a more fine grained `package-data` configuration.
      
              You can read more about "package data files" on setuptools documentation page:
      
              - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
      
      
              [^1]: For Python, any directory (with suitable naming) can be imported,
                    even if it does not contain any `.py` files.
                    On the other hand, currently there is no concept of package data
                    directory, all directories are treated like packages.
              ********************************************************************************
      
      !!
        check.warn(importable)
      copying cgat/GeneModelAnalysis.pyx -> build/lib.linux-x86_64-cpython-312/cgat
      creating build/lib.linux-x86_64-cpython-312/cgat/BamTools
      copying cgat/BamTools/bamtools.pyx -> build/lib.linux-x86_64-cpython-312/cgat/BamTools
      copying cgat/BamTools/geneprofile.pyx -> build/lib.linux-x86_64-cpython-312/cgat/BamTools
      copying cgat/BamTools/peakshape.pyx -> build/lib.linux-x86_64-cpython-312/cgat/BamTools
      creating build/lib.linux-x86_64-cpython-312/cgat/Components
      copying cgat/Components/Components.pyx -> build/lib.linux-x86_64-cpython-312/cgat/Components
      copying cgat/Components/connected_components.cpp -> build/lib.linux-x86_64-cpython-312/cgat/Components
      creating build/lib.linux-x86_64-cpython-312/cgat/FastqTools
      copying cgat/FastqTools/fastqtools.pyx -> build/lib.linux-x86_64-cpython-312/cgat/FastqTools
      creating build/lib.linux-x86_64-cpython-312/cgat/VCFTools
      copying cgat/VCFTools/vcftools.pyx -> build/lib.linux-x86_64-cpython-312/cgat/VCFTools
      copying cgat/NCL/cnestedlist.pyx -> build/lib.linux-x86_64-cpython-312/cgat/NCL
      copying cgat/NCL/intervaldb.c -> build/lib.linux-x86_64-cpython-312/cgat/NCL
      running build_ext
      Compiling cgat/Components/Components.pyx because it changed.
      [1/1] Cythonizing cgat/Components/Components.pyx
      building 'cgat.Components' extension
      creating build/temp.linux-x86_64-cpython-312/cgat/Components
      g++ -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -fPIC -I/home/neil/.virtualenvs/isoslam/include -I/usr/include/python3.12 -c cgat/Components/Components.cpp -o build/temp.linux-x86_64-cpython-312/cgat/Components/Components.o
      cgat/Components/Components.cpp:1249:10: fatal error: connected_components.h: No such file or directory
       1249 | #include "connected_components.h"
            |          ^~~~~~~~~~~~~~~~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/g++' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for cgat
Failed to build cgat
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (cgat)

There are some warnings about the cgat.VCFTools / cgat.BamTools / cgat.Components / cgat.FastqTools not being correctly configured for setuptools but the install proceeds.

Compilation of the C++ code fails because there is a missing header file connected_components.h.

Trying to install cgat-0.7.2

Dropping back a version I encounter again the Numpy issue described in #177 as the very first thing setup.py does is check that numpy can be imported. It should be importable because I have explicitly installed it...

pip show numpy | head -n8
Name: numpy
Version: 2.1.2
Summary: Fundamental package for array computing in Python
Home-page: https://numpy.org
Author: Travis E. Oliphant et al.
Author-email: 
License: Copyright (c) 2005-2024, NumPy Developers.
All rights reserved.

However this is not detected correctly...

pip install .
Processing /home/neil/work/git/hub/cgat-developers/cgat-0.7.2
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [26 lines of output]
      Traceback (most recent call last):
        File "<string>", line 15, in <module>
      ModuleNotFoundError: No module named 'numpy'
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/home/neil/.virtualenvs/isoslam/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/neil/.virtualenvs/isoslam/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/neil/.virtualenvs/isoslam/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-gygxww3j/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 333, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-gygxww3j/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 303, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-gygxww3j/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 521, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-gygxww3j/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 319, in run_setup
          exec(code, locals())
        File "<string>", line 17, in <module>
      ImportError: the CGAT code collection requires numpy to be installed before running setup.py (pip install numpy)
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

I decided to try using the Setuptools method of Declaring required dependency using setup.cfg (although the current ecosystem is standardising on the pyproject.toml format as per PEP621).

setup.cfg

[project]
install_requires =
    "Cython",
    "numpy",
    "pysam"


[egg_info]
tag_build =
tag_date = 0

setup.py

Disable the checks for these packages at the start¹

import sysconfig
import sys
import os
import subprocess
import re

########################################################################
#######################################################################
# Check for dependencies
#
# Is there a way to do this more elegantly?
#     1. Run "pip install numpy"
#     2. Wrap inside functions (works for numpy/pysam, but not cython)
# try:
#     import numpy
# except ImportError:
#     raise ImportError(
#         "the CGAT code collection requires numpy to be installed "
#         "before running setup.py (pip install numpy)")

# try:
#     import Cython
# except ImportError:
#     raise ImportError(
#         "the CGAT code collection requires cython to "
#         "be installed before running setup.py (pip install cython)")

# try:
#     import pysam
# except ImportError:
#     raise ImportError(
#         "the CGAT code collection requires pysam to "
#         "be installed before running setup.py (pip install pysam)")
...

Try installing but it now doesn't install because an attempt is made to import from Cython, which I have explicitly manually installed along with pysam, fails.

pip show Cython pysam      
Name: Cython
Version: 3.0.11
Summary: The Cython compiler for writing C extensions in the Python language.
Home-page: https://cython.org/
Author: Robert Bradshaw, Stefan Behnel, Dag Seljebotn, Greg Ewing, et al.
Author-email: cython-devel@python.org
License: Apache-2.0
Location: /home/neil/.virtualenvs/isoslam/lib/python3.12/site-packages
Requires: 
Required-by: 
---
Name: pysam
Version: 0.22.1
Summary: Package for reading, manipulating, and writing genomic data
Home-page: https://github.com/pysam-developers/pysam
Author: Andreas Heger
Author-email: Andreas Heger <andreas.heger@gmail.com>
License: MIT License
Location: /home/neil/.virtualenvs/isoslam/lib/python3.12/site-packages
Requires: 
Required-by: isoslam

❱ pip install .
Processing /home/neil/work/git/hub/cgat-developers/cgat-0.7.2
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [21 lines of output]
      <string>:52: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
      Traceback (most recent call last):
        File "/home/neil/.virtualenvs/isoslam/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/neil/.virtualenvs/isoslam/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/neil/.virtualenvs/isoslam/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-lmjixk24/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 333, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-lmjixk24/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 303, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-lmjixk24/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 521, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-lmjixk24/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 319, in run_setup
          exec(code, locals())
        File "<string>", line 57, in <module>
      ModuleNotFoundError: No module named 'Cython'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Possible solutions?

• Adding the missing header file connected_components.h seems like it would be needed to build and install the package (after adding back in an empty requires.txt.
• Longer term rehashing the package metadata and standardising on pyproject.toml would perhaps help with some of these problems as setup.py tries to import and use some of the packages it depends on before they have been pulled in and installed as dependencies.

Footnotes

1. I note the comments in setup.py about Is there a way to do this more elegantly?. The traditional method of specifying dependencies in setup.py is within the call to setup() see setup.py tab here but as mentioned the preferred configuration is now via pyproject.toml and I think the reason this inelegant solution exists is because the packages are imported and used before they have been installed as dependencies. ↩

[Acribbs]

oh I see what the potential problem is... im not sure why our testing didnt pick this up. In the latest release connected_components.h isnt in the setup.py. In the meantime can you use the 0.7.2 version?

[Acribbs]

I will fix this in next few days and make a new release

[Acribbs]

Iv pulled the release for the moment

[Acribbs]

you able to test this pull, it should fix the issues?
#135

[ns-rse]

Sorry for the delay, only work on the project that uses this part time and have been on annual leave.

In a clean environment this now installs...

(tmp-3c6c4b33689cb6a) ❱ git clone https://github.com/cgat-developers/cgat-apps.git
Cloning into 'cgat-apps'...
remote: Enumerating objects: 39732, done.
remote: Counting objects: 100% (1615/1615), done.
remote: Compressing objects: 100% (775/775), done.
pReceiving objects:  13% (5474/39732), 25.81 MiB | 5.33 MiB/s/s/s
remote: Total 39732 (delta 784), reused 1355 (delta 725), pack-reused 38117 (from 1)
Receiving objects: 100% (39732/39732), 390.89 MiB | 4.12 MiB/s, done.
Resolving deltas: 100% (24717/24717), done.
(tmp-3c6c4b33689cb6a) ❱ cd cgat-apps 
(tmp-3c6c4b33689cb6a) ❱ git switch AC-fixsetup 
branch 'AC-fixsetup' set up to track 'origin/AC-fixsetup'.
Switched to a new branch 'AC-fixsetup'
(tmp-3c6c4b33689cb6a) ❱ pip install .
Processing /home/neil/tmp/cgat/cgat-apps
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: cgat
  Building wheel for cgat (pyproject.toml) ... \
done
  Created wheel for cgat: filename=cgat-0.7.4-cp312-cp312-linux_x86_64.whl size=10638983 sha256=a0474c183080bdfadba0946cd20ce9c4539f6b88eb15f40f7586668822453751
  Stored in directory: /home/neil/.cache/pip/wheels/e2/2f/5d/1497f4f335bec91ea7fb6335d7da2df5e56a38796409b114e2
Successfully built cgat
Installing collected packages: cgat
Successfully installed cgat-0.7.4

[ns-rse]

I'm getting bitten by this in CI now as cgat-0.7.2 is looking for numpy and failing to find it, despite it explicitly being installed first.

Log file can be found here but since they disappear after a while the output from pip install is shown below.

I've highlighted where numpy is installed prior the attempt to install cgat.

Run pip install --upgrade virtualenv
  pip install --upgrade virtualenv
  pip install --upgrade pip setuptools
  virtualenv --upgrade-embed-wheels
  pip install numpy
  pip install -e .[tests]
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.1[2](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:2).7/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/[3](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:3).12.7/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.7/x6[4](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:4)
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.7/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.7/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.12.7/x64/lib
Collecting virtualenv
  Downloading virtualenv-20.27.1-py3-none-any.whl.metadata (4.[5](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:5) kB)
Collecting distlib<1,>=0.3.7 (from virtualenv)
  Downloading distlib-0.3.9-py2.py3-none-any.whl.metadata (5.2 kB)
Collecting filelock<4,>=3.12.2 (from virtualenv)
  Downloading filelock-3.1[6](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:6).1-py3-none-any.whl.metadata (2.9 kB)
Collecting platformdirs<5,>=3.9.1 (from virtualenv)
  Downloading platformdirs-4.3.6-py3-none-any.whl.metadata (11 kB)
Downloading virtualenv-20.2[7](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:7).1-py3-none-any.whl (3.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.1/3.1 MB 60.[8](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:8) MB/s eta 0:00:00
Downloading distlib-0.3.[9](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:9)-py2.py3-none-any.whl (468 kB)
Downloading filelock-3.16.1-py3-none-any.whl (16 kB)
Downloading platformdirs-4.3.6-py3-none-any.whl (18 kB)
Installing collected packages: distlib, platformdirs, filelock, virtualenv
Successfully installed distlib-0.3.9 filelock-3.16.1 platformdirs-4.3.6 virtualenv-20.27.1
Requirement already satisfied: pip in /opt/hostedtoolcache/Python/3.12.7/x64/lib/python3.12/site-packages (24.3.1)
Collecting setuptools
  Downloading setuptools-75.6.0-py3-none-any.whl.metadata (6.7 kB)
Downloading setuptools-75.6.0-py3-none-any.whl (1.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 35.9 MB/s eta 0:00:00
Installing collected packages: setuptools
Successfully installed setuptools-75.6.0
upgrade pip for python 3.8 with current pip-24.3.1-py3-none-any.whl
upgrade setuptools for python 3.8 with current setuptools-75.2.0-py3-none-any.whl
upgrade wheel for python 3.8 with current wheel-0.44.0-py3-none-any.whl
upgrade pip for python 3.9 with current pip-24.3.1-py3-none-any.whl
upgrade setuptools for python 3.9 with current setuptools-75.2.0-py3-none-any.whl
upgrade wheel for python 3.9 with current wheel-0.44.0-py3-none-any.whl
upgrade pip for python 3.[10](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:10) with current pip-24.3.1-py3-none-any.whl
upgrade setuptools for python 3.10 with current setuptools-75.2.0-py3-none-any.whl
upgrade wheel for python 3.10 with current wheel-0.44.0-py3-none-any.whl
upgrade pip for python 3.[11](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:11) with current pip-24.3.1-py3-none-any.whl
upgrade setuptools for python 3.11 with current setuptools-75.2.0-py3-none-any.whl
upgrade wheel for python 3.11 with current wheel-0.44.0-py3-none-any.whl
upgrade pip for python 3.[12](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:12) with current pip-24.3.1-py3-none-any.whl
upgrade setuptools for python 3.12 with current setuptools-75.2.0-py3-none-any.whl
upgrade wheel for python 3.12 with current wheel-0.44.0-py3-none-any.whl
upgrade pip for python 3.[13](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:13) with current pip-24.3.1-py3-none-any.whl
upgrade setuptools for python 3.13 with current setuptools-75.2.0-py3-none-any.whl
upgrade wheel for python 3.13 with current wheel-0.44.0-py3-none-any.whl
upgrade pip for python 3.[14](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:14) with current pip-24.3.1-py3-none-any.whl
upgrade setuptools for python 3.14 with current setuptools-75.2.0-py3-none-any.whl
upgrade wheel for python 3.14 with current wheel-0.44.0-py3-none-any.whl
upgraded wheel for python 3.8 in 0:00:13.481616 new entries found:
	NewVersion(filename=wheel-0.45.0-py3-none-any.whl), found_date=2024-11-21 14:12:37.776518+00:00, release_date=2024-11-08 21:44:59+00:00, source=manual)
upgraded pip for python 3.10 in 0:00:13.508307 new entries found:
	NewVersion(filename=pip-24.3.1-py3-none-any.whl), found_date=2024-11-21 14:12:37.763969+00:00, release_date=2024-10-27 18:35:53+00:00, source=manual)
upgraded wheel for python 3.13 in 0:00:13.441020 new entries found:
	NewVersion(filename=wheel-0.45.0-py3-none-any.whl), found_date=2024-11-21 14:12:37.895536+00:00, release_date=2024-11-08 21:44:59+00:00, source=manual)
upgraded wheel for python 3.11 in 0:00:13.675127 new entries found:
	NewVersion(filename=wheel-0.45.0-py3-none-any.whl), found_date=2024-11-21 14:12:37.819624+00:00, release_date=2024-11-08 21:44:59+00:00, source=manual)
upgraded pip for python 3.8 in 0:00:13.770411 new entries found:
	NewVersion(filename=pip-24.3.1-py3-none-any.whl), found_date=2024-11-21 14:12:37.771740+00:00, release_date=2024-10-27 18:35:53+00:00, source=manual)
upgraded pip for python 3.9 in 0:00:13.775614 new entries found:
	NewVersion(filename=pip-24.3.1-py3-none-any.whl), found_date=2024-11-21 14:12:37.764774+00:00, release_date=2024-10-27 18:35:53+00:00, source=manual)
upgraded wheel for python 3.10 in 0:00:13.841341 new entries found:
	NewVersion(filename=wheel-0.45.0-py3-none-any.whl), found_date=2024-11-21 14:12:37.775167+00:00, release_date=2024-11-08 21:44:59+00:00, source=manual)
upgraded pip for python 3.13 in 0:00:13.76[15](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:16)38 new entries found:
	NewVersion(filename=pip-24.3.1-py3-none-any.whl), found_date=2024-11-21 14:12:37.868022+00:00, release_date=2024-10-27 18:35:53+00:00, source=manual)
upgraded wheel for python 3.12 in 0:00:13.790123 new entries found:
	NewVersion(filename=wheel-0.45.0-py3-none-any.whl), found_date=2024-11-21 14:12:37.848349+00:00, release_date=2024-11-08 21:44:59+00:00, source=manual)
upgraded wheel for python 3.9 in 0:00:13.875404 new entries found:
	NewVersion(filename=wheel-0.45.0-py3-none-any.whl), found_date=2024-11-21 14:12:37.769526+00:00, release_date=2024-11-08 21:44:59+00:00, source=manual)
upgraded pip for python 3.12 in 0:00:13.868303 new entries found:
	NewVersion(filename=pip-24.3.1-py3-none-any.whl), found_date=2024-11-21 14:12:37.821266+00:00, release_date=2024-10-27 18:35:53+00:00, source=manual)
upgraded wheel for python 3.14 in 0:00:13.809338 new entries found:
	NewVersion(filename=wheel-0.45.0-py3-none-any.whl), found_date=2024-11-21 14:12:37.935188+00:00, release_date=2024-11-08 21:44:59+00:00, source=manual)
upgraded pip for python 3.11 in 0:00:13.980762 new entries found:
	NewVersion(filename=pip-24.3.1-py3-none-any.whl), found_date=2024-11-21 14:12:37.780362+00:00, release_date=2024-10-27 18:35:53+00:00, source=manual)
upgraded pip for python 3.14 in 0:00:13.901360 new entries found:
	NewVersion(filename=pip-24.3.1-py3-none-any.whl), found_date=2024-11-21 14:12:37.915952+00:00, release_date=2024-10-27 18:35:53+00:00, source=manual)
upgraded setuptools for python 3.8 in 0:00:14.19[16](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:17)76 new entries found:
	NewVersion(filename=setuptools-75.3.0-py3-none-any.whl), found_date=2024-11-21 14:12:37.769901+00:00, release_date=2024-10-29 10:23:24+00:00, source=manual)
upgraded setuptools for python 3.12 in 0:00:14.128076 new entries found:
	NewVersion(filename=setuptools-75.6.0-py3-none-any.whl), found_date=2024-11-21 14:12:37.831347+00:00, release_date=2024-11-20 18:16:10+00:00, source=manual)
upgraded setuptools for python 3.13 in 0:00:14.155189 new entries found:
	NewVersion(filename=setuptools-75.6.0-py3-none-any.whl), found_date=2024-11-21 14:12:37.887668+00:00, release_date=2024-11-20 18:16:10+00:00, source=manual)
upgraded setuptools for python 3.10 in 0:00:14.313731 new entries found:
	NewVersion(filename=setuptools-75.6.0-py3-none-any.whl), found_date=2024-11-21 14:12:37.772635+00:00, release_date=2024-11-20 18:16:10+00:00, source=manual)
upgraded setuptools for python 3.9 in 0:00:14.332700 new entries found:
	NewVersion(filename=setuptools-75.6.0-py3-none-any.whl), found_date=2024-11-21 14:12:37.7655[17](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:18)+00:00, release_date=2024-11-20 18:16:10+00:00, source=manual)
upgraded setuptools for python 3.11 in 0:00:14.324987 new entries found:
	NewVersion(filename=setuptools-75.6.0-py3-none-any.whl), found_date=2024-11-21 14:12:37.788419+00:00, release_date=2024-11-20 [18](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:19):16:10+00:00, source=manual)
upgraded setuptools for python 3.14 in 0:00:14.225363 new entries found:
	NewVersion(filename=setuptools-75.6.0-py3-none-any.whl), found_date=[20](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:21)24-11-21 14:12:37.926997+00:00, release_date=2024-11-20 18:16:10+00:00, source=manual)
SystemExit: None
Collecting numpy
  Downloading numpy-2.1.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (62 kB)
Downloading numpy-2.1.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.0 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.0/16.0 MB 161.4 MB/s eta 0:00:00
Installing collected packages: numpy
Successfully installed numpy-2.1.3                            ############# Numpy is installed
Obtaining file:///home/runner/work/IsoSLAM/IsoSLAM
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Checking if build backend supports build_editable: started
  Checking if build backend supports build_editable: finished with status 'done'
  Getting requirements to build editable: started
  Getting requirements to build editable: finished with status 'done'
  Preparing editable metadata (pyproject.toml): started
  Preparing editable metadata (pyproject.toml): finished with status 'done'
Collecting apsw (from isoslam==0.1.dev1+g57124b0)
  Downloading apsw-3.47.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.4 kB)
Collecting cgat (from isoslam==0.1.dev1+g57124b0)
  Downloading cgat-0.7.2.tar.gz (4[21](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:22) kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [26 lines of output]
      Traceback (most recent call last):
        File "<string>", line 15, in <module>
      ModuleNotFoundError: No module named 'numpy'
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/opt/hostedtoolcache/Python/3.12.7/x64/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/opt/hostedtoolcache/Python/3.12.7/x64/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/hostedtoolcache/Python/3.12.7/x64/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-p_l_nrzv/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 334, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-p_l_nrzv/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 304, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-p_l_nrzv/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 5[22](https://github.com/sudlab/IsoSLAM/actions/runs/11953428873/job/33321276774?pr=62#step:4:23), in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-p_l_nrzv/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 320, in run_setup
          exec(code, locals())
        File "<string>", line 17, in <module>
      ImportError: the CGAT code collection requires numpy to be installed before running setup.py (pip install numpy)
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
Error: Process completed with exit code 1.

[Acribbs]

Looking at your github actions file, have you tried adding --no-build-isolation to the pip install cgat, looks like there may be an isolation issue occuring where cgat builds some modules that are already there. Could be wrong though.

[ns-rse]

Thanks for the suggestion @Acribbs we have progress!

I added the --no-build-isolation to the installation of cgat and cgatcore (see pull request).

cgat installs fine but cgatcore now complains about a missing dependency on Cython, see log lines 1077-1118.

Collecting cgat
  Downloading cgat-0.7.2.tar.gz (421 kB)
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error
  
  Preparing metadata (pyproject.toml) did not run successfully.
  exit code: 1
  
  [21 lines of output]
  Traceback (most recent call last):
    File "<string>", line 22, in <module>
  ModuleNotFoundError: No module named 'Cython'
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
      main()
    File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 149, in prepare_metadata_for_build_wheel
      return hook(metadata_directory, config_settings)
    File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\setuptools\build_meta.py", line 377, in prepare_metadata_for_build_wheel
      self.run_setup()
    File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\setuptools\build_meta.py", line 522, in run_setup
      super().run_setup(setup_script=setup_script)
    File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\setuptools\build_meta.py", line 320, in run_setup
      exec(code, locals())
    File "<string>", line 24, in <module>
  ImportError: the CGAT code collection requires cython to be installed before running setup.py (pip install cython)
  [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

Encountered error while generating package metadata.

See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

I first looked in pyproject.toml (since this is the standard for configuring package dependencies defined in PEP518) but its quite sparse so then looked at setup.py and can see that it only checks for the presence of numpy, Cython and pysam (lines 13-20) rather than explicitly stating them as dependencies.

It would probably be possible to port much of setup.py to pyproject.toml. Reading through there are...

• version checks for minimum Python requires-python = >=3.6
• setuptools minimum version.
• cgat version.
• Python dependencies numpy / Cython / pysam
• external dependencies WigToBigWig and bedtools
• building C extensions

I could migrate some of this if it would be helpful, but the v0.7.4 release which was pulled has these build requirements in place already (although much of the metadata and setup is still via setup.py and hasn't been migrated to pyproject.toml).

I checked installation the other day of v0.7.4 and was able to do so. Would it be possible to re-release that (or a v0.7.4.1 / v0.7.4.0-post as in my experience its not possible to make a release to PyPI that matches something that already exists, even if its been deleted to prevent malicious packages being uploaded).