Nextflow

[katestan]

Hi there - excited to use the tool. Is the nextflow pipeline available yet?

[YuheCheng62]

Hello! Thank you for your interest in Using DupCaller.
We are in the process of creating a nextflow pipeline for DupCaller. In the latest push, we have just added a docker image that can be easily run with either Docker or Singularity.

[katestan]

Hi thanks for your response.

I see that you've made changes to the scripts and now include *_coverage.bed.gz and *_duplex_allele_counts.txt in the output files.

Using the old scripts the pipeline completed successfully - I got snv and indel vcfs per sample.
Now when I try to run the pipeline I get the error below. Have you encountered this before?

A few additional questions:

(1) For NanoSeq the barcode pattern is 'NNNXXXX' but for UD-seq data I am using 'NNNNNNNN' because the IDT kit has fixed UMI sequences. Is this correct?

(2) We have generated exome UD-seq data. Should I be using the exome capture regions as my --regions input or just all human contigs chr1-22,X,Y?

(3) What should reestimatebed be if I have exome data? Looking at the code it looks like the trinucleotide frequencies are already corrected using coverage in *_trinuc_by_duplex_group.txt. Does this not already normalise for my capture regions?

[c3/8ab6fc] TRIM_BARCODES (VAN8811A1)   | 1 of 1, cached: 1 ✔
[e2/c73cc1] BWA_ALIGN (VAN8811A1)       | 1 of 1, cached: 1 ✔
[71/b70147] MARK_DUPLICATES (VAN8811A1) | 1 of 1, cached: 1 ✔
[b2/9294b1] CALL_VARIANTS (VAN8811A1)   | 0 of 1 ✘
[-        ] ESTIMATE_BURDEN             -
Execution cancelled -- Finishing pending tasks before exit

================================================================
Pipeline completed at: 2026-04-28T12:33:07.001664715+01:00
Execution status: FAILED
Duration: 2h 5m 34s
Output directory: ./results
================================================================

ERROR ~ Error executing process > 'CALL_VARIANTS (VAN8811A1)'

Caused by:
  Process `CALL_VARIANTS (VAN8811A1)` terminated with an error exit status (1)


Command executed:

  singularity exec --bind /nemo --bind /flask /nemo/lab/rouhanif/home/users/stanlek/singularity/dupcaller-1.1.0.sif DupCaller.py call \
      -b VAN8811A1.mkdped.bam \
      -f genome.fa \
      -o VAN8811A1 \
      -p 8 \
      -r chr1 chr2 chr3 chr4 chr5 chr6 chr7 chr8 chr9 chr10 chr11 chr12 chr13 chr14 chr15 chr16 chr17 chr18 chr19 chr20 chr21 chr22 chrX chrY \
       \
      -g germline_with_af.vcf.gz \
      -m NOISE_GRCh38.wgns.bed.gz \
       \
      -maf 0.1 \
      -gaf 0.001 \
      -d 10 \
      -tt 7 \
      -tr 7 \
      -mq 40 \
      -w 100000
  mv VAN8811A1/* .

Command exit status:
  1

Command output:
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr2 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr3 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr4 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr5 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr6 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr7 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr8 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr9 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr10 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr11 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr12 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr13 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr14 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr15 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr16 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr17 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr18 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr19 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr20 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr21 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chr22 of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chrX of VAN8811A1.mkdped.bam......
  ......Splitting bam regions into roughly equal chunks. Counting reads in chrY of VAN8811A1.mkdped.bam......
  ....Genomic regions splitted in 119.73719330628712 minutes...
  ............Completed region splitting in 119.73719433546066 minutes............
  .........Starting estimating error rate.............
  Process 1: Initiated. Regions:chr2:108000000-chr5:131900000
  Process 1: screening for highly-damaged or misaligned reads
  Process 1: finished screening highly damaged reads in  5.53 minutes. Blacklisted 1018525 ( 2.54%) possible highly damaged read and started variant calling.
  Process 7: Initiated. Regions:chr19:45800000-chrY:57227415
  Process 7: screening for highly-damaged or misaligned reads
  Process 7: finished screening highly damaged reads in  5.02 minutes. Blacklisted 608488 ( 1.52%) possible highly damaged read and started variant calling.
  Process 0: Initiated. Regions:chr1:0-chr2:108000000
  Process 0: screening for highly-damaged or misaligned reads
  Process 0: finished screening highly damaged reads in  5.18 minutes. Blacklisted 307665 ( 0.77%) possible highly damaged read and started variant calling.
  Process 3: Initiated. Regions:chr8:138800000-chr11:63900000
  Process 3: screening for highly-damaged or misaligned reads
  Process 3: finished screening highly damaged reads in  5.23 minutes. Blacklisted 876990 ( 2.19%) possible highly damaged read and started variant calling.
  Process 5: Initiated. Regions:chr15:40400000-chr17:36500000
  Process 5: screening for highly-damaged or misaligned reads
  Process 5: finished screening highly damaged reads in  5.26 minutes. Blacklisted 332240 ( 0.83%) possible highly damaged read and started variant calling.
  Process 6: Initiated. Regions:chr17:36500000-chr19:45800000
  Process 6: screening for highly-damaged or misaligned reads
  Process 6: finished screening highly damaged reads in  5.26 minutes. Blacklisted 329185 ( 0.82%) possible highly damaged read and started variant calling.
  Process 2: Initiated. Regions:chr5:131900000-chr8:138800000
  Process 2: screening for highly-damaged or misaligned reads
  Process 2: finished screening highly damaged reads in  5.33 minutes. Blacklisted 388370 ( 0.97%) possible highly damaged read and started variant calling.
  Process 4: Initiated. Regions:chr11:63900000-chr15:40400000
  Process 4: screening for highly-damaged or misaligned reads
  Process 4: finished screening highly damaged reads in  5.41 minutes. Blacklisted 294988 ( 0.74%) possible highly damaged read and started variant calling.

Command error:
  /usr/local/lib/python3.12/site-packages/DupCaller_sub/funcs/call.py:488: RuntimeWarning: invalid value encountered in divide
    dmgmat = dmgmat / dmgmat.sum(axis=1, keepdims=True)
  /usr/local/lib/python3.12/site-packages/DupCaller_sub/funcs/call.py:414: RuntimeWarning: invalid value encountered in divide
    ampmat = ampmat / ampmat.sum(axis=1, keepdims=True)
  /usr/local/lib/python3.12/site-packages/DupCaller_sub/funcs/call.py:488: RuntimeWarning: invalid value encountered in divide
    dmgmat = dmgmat / dmgmat.sum(axis=1, keepdims=True)
  /usr/local/lib/python3.12/site-packages/DupCaller_sub/funcs/call.py:414: RuntimeWarning: invalid value encountered in divide
    ampmat = ampmat / ampmat.sum(axis=1, keepdims=True)
  /usr/local/lib/python3.12/site-packages/DupCaller_sub/funcs/call.py:488: RuntimeWarning: invalid value encountered in divide
    dmgmat = dmgmat / dmgmat.sum(axis=1, keepdims=True)
  /usr/local/lib/python3.12/site-packages/DupCaller_sub/funcs/call.py:414: RuntimeWarning: invalid value encountered in divide
    ampmat = ampmat / ampmat.sum(axis=1, keepdims=True)
  /usr/local/lib/python3.12/site-packages/DupCaller_sub/funcs/call.py:488: RuntimeWarning: invalid value encountered in divide
    dmgmat = dmgmat / dmgmat.sum(axis=1, keepdims=True)
  /usr/local/lib/python3.12/site-packages/DupCaller_sub/funcs/call.py:414: RuntimeWarning: invalid value encountered in divide
    ampmat = ampmat / ampmat.sum(axis=1, keepdims=True)
  /usr/local/lib/python3.12/site-packages/DupCaller_sub/funcs/call.py:488: RuntimeWarning: invalid value encountered in divide
    dmgmat = dmgmat / dmgmat.sum(axis=1, keepdims=True)
  multiprocessing.pool.RemoteTraceback: 
  """
  Traceback (most recent call last):
    File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 125, in worker
      result = (True, func(*args, **kwds))
                      ^^^^^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 51, in starmapstar
      return list(itertools.starmap(args[0], args[1]))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.12/site-packages/DupCaller_sub/funcs/call.py", line 1227, in callBam
      ) = profileTriNucMismatches(
          ^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.12/site-packages/DupCaller_sub/funcs/learn.py", line 332, in profileTriNucMismatches
      str_dmg = hp_int[2, dmg_antimask][hp_int[3, dmg_antimask] == 1]
                ~~~~~~^^^^^^^^^^^^^^^^^
  IndexError: index 2 is out of bounds for axis 0 with size 2
  """
  
  The above exception was the direct cause of the following exception:
  
  Traceback (most recent call last):
    File "/usr/local/bin/DupCaller.py", line 516, in <module>
      do_call(args)
    File "/usr/local/lib/python3.12/site-packages/DupCaller_sub/Caller.py", line 377, in do_call
      results = pool.starmap(
                ^^^^^^^^^^^^^
    File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 375, in starmap
      return self._map_async(func, iterable, starmapstar, chunksize).get()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 774, in get
      raise self._value
  IndexError: index 2 is out of bounds for axis 0 with size 2

Work dir:
  /nemo/lab/rouhanif/home/users/stanlek/DupCaller/nextflow/work/b2/9294b10a74bbf74853e362fdc3d0ac

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

 -- Check '.nextflow.log' file for details```

[katestan]

This error was related to the index files. They need to be generated using the updated code and STR bed file (or the files made available in Resources).