dataloader default to 100 workers even when passing -t 20.

[Mikes92]

Hello!

Thank you for making such a powerful, yet simple to use program. I had absolutely no issues installing and running on a small test dataset. I did have a somewhat minor issue. although I passed -t 20, I got a pytorch warning that dataloader was using 100 workers (which was the default).

parser.add_argument("-t", "--threads", type=int, default=100, help="number of threads (default 100)")

it looks like some run_pangaea calls for pangaea.py aren't passing -t $threads, but i'm not sure if this is the culprit. This ended up not causing an issue though as i was running on a smaller test dataset.

I also have a somewhat conceptual question: I have a short & long metagenomic dataset that has relatively deep long-read coverage, generating over 30 contigs >1mb with depth ranging from 20-200x following a metaFlye assembly. Could these contigs be used to better bin the short reads (virtual barcode)? I'm thinking not, because low abundant organisms that aren't represented in the long-read-only contigs wont get a virtual barcode, but they might get barcoded when directly using the long-read data? Grateful for any thoughts you have!

Mike

[ZhangZhenmiao]

Hi Mike

You are very correct about the threads -- thank you so much. I fixed it in the new commit.

You proposed a good approach to do the hybrid assembly. For the question

Could these contigs be used to better bin the short reads (virtual barcode)? I'm thinking not, because low abundant organisms that aren't represented in the long-read-only contigs wont get a virtual barcode

These contigs will sigificantly improve binning for the short reads that can be mapped to the contigs, so the remaining short reads will not have virtual barcodes. Therefore, you are correct about the low abundant organisims.

One solution is to combine the metaFlye contigs, with the long reads that cannot be mapped to the metaFlye contigs, as a single dataset, and input it through "--longreads" of run_pangaea. This will make the virtual barcodes accurate and complete.

After getting the short reads with virtual barcodes (line 256 of run_pangaea):

1. You can continue running the hybrid mode of run_pangaea. The hybrid mode will generate results but the parameters of hybridspades (lines 320-325) might not be optimized for your task. But I think the parameters make sense to some extent. If your long read assembly by metaflye is error-prone, input it using --nanopore/--pacbio to hybridspades is ok.

2. Another way is to run the normal linked-reads pipeline of run_pangaea inputting only the short reads with virtual barcodes (not the hybrid mode, "-s" can be 10x). After that, you may want to combine the linked read assembly and the metaFlye assembly.

I am happy to discuss if you have any questions.

Best
Zhenmiao

[Mikes92]

Hi Zhenmiao, thanks for your quick reply, i'll pull the new commit and retry. I also have a few other questions but they're not issue related so i've sent you an email directly. Thanks!

Mike