UPPMAX

Tutorials

• by UPPMAX
  • Introduction to Bioinformatics
    • ... but "Getting a node of your own" section does not work in a free study period.
  • Linux introduction (screencast)

How to run the pipeline on UPPMAX?

Installation

git clone -b v3dev https://github.com/shka/STRTprep.git STRTprep3.test
cd STRTprep3.test
bin/install_uppmax.sh

Protocol

• Step 1: (Copy pre-build indexes, or) build indexes of reference genome and transcriptome, like as below. See also a help document.

sbatch -A snic2017-7-317 -t 12:00:00 -p core -n 1 <<EOF
#!/usr/bin/env bash
. bin/setup_uppmax.sh
bin/index_hg19_ercc92_ynbA_u13369.sh
bin/index_refGene.sh hg19 src/ebwt/hg19_ercc92_ynbA_u13369/ref
EOF

• Step 2: Edit conf.yaml and src/samples.csv
• Step 3: Run preprocessing and quality-check as follows, then check out/byGene/samples.xls
  • N at (i) should be equal to triple of total lanes of sequences but the maximum is 20.
  • Please put job number of (i) into Xi at (ii), and of (ii) into Xii at (iii); then the latter jobs will run if the former jobs succeeded.
  • ... Please contact when you process multiple libraries urgently - there is a method to run the pipeline more faster.

sbatch -A snic2017-7-317 -t 24:00:00 -p core -n N <<EOF ## (i)
#!/usr/bin/env bash
. bin/setup_uppmax.sh
PROCS=20 rake -m -j \$SLURM_NTASKS qc1 qc2
EOF
sbatch -A snic2017-7-317 -t 24:00:00 -p core -n 4 -d afterok:Xi <<EOF ## (ii)
#!/usr/bin/env bash
. bin/setup_uppmax.sh
PROCS=20 rake qc3
EOF
sbatch -A snic2017-7-317 -t 24:00:00 -p node -n 8 -d afterok:Xii <<EOF ## (iii)
#!/usr/bin/env bash
. bin/setup_uppmax.sh
PROCS=20 rake -m -j \$SLURM_NTASKS qc
EOF

• NOTE:
  • qc3 is frequently terminated by a memory overflow or time out; you can check it by tmp/step2c/logs/tophat.log. If TopHat was terminated, you can recover it as follow; you should give more nodes in case of memory overflow, and you should give more longer time for this job in case of time out. After this recovery step, you can restart from the step 3 above.

sbatch -A snic2017-7-317 -t 24:00:00 -p core -n 2 <<EOF
#!/usr/bin/env bash
. bin/setup_uppmax.sh
tophat -R tmp/step2c
EOF

TopHat2 requires bigger memory only in a few minutes at the end of run. And we cannot estimate size of the required memory; it depends on the library. However, we can easily waste the assigned calculation time, if we occupied many cores for all runtime of TopHat2 (task qc3).

• Step 4: Run differential expression tests and the down-stream analysis by plugins (for gene-based quantification) as follows.
  • Please extend time of job allocation (one hour), and change name of queue (devcore to core), when you met timeout
  • Please extend number of CPU allocation (one) when you met memory overflow

sbatch -A snic2017-7-317 -t 1:00:00 -p devcore -n 1 <<EOF
#!/usr/bin/env bash
. bin/setup_uppmax.sh
PROCS=20 rake plugins_gene
EOF

• [v3dev] Step 5: Add "CLASS.TFE" column for classification of samples into src/samples.csv table, run differential expression tests, based on transcription-start-regions, then check out/byTFE/diffexp.xls and analyze more further!
  • Yi at (ii) is a job ID of (i).

sbatch -A snic2017-7-317 -t 24:00:00 -p core -n 5 <<EOF ## (i)
#!/usr/bin/env bash
. bin/setup_uppmax.sh
PROCS=20 rake tfe1
EOF
sbatch -A snic2017-7-317 -t 24:00:00 -p node -n 10 -d afterok:Yi <<EOF ## (ii)
#!/usr/bin/env bash
. bin/setup_uppmax.sh
PROCS=20 rake -m -j \$SLURM_NTASKS tfe
EOF

• Step 6: Run differential expression tests and the down-stream analysis by plugins for TFE-based quantification as follows.
  • Please extend time of job allocation (one hour), and change name of queue (devcore to core), when you met timeout
  • Please extend number of CPU allocation (one) when you met memory overflow

sbatch -A snic2017-7-317 -t 1:00:00 -p devcore -n 1 <<EOF
#!/usr/bin/env bash
. bin/setup_uppmax.sh
PROCS=20 rake plugins_tfe
EOF