For a sample with multiple input file pairs, is the order preserved?

[bentyeh]

Consider a samples.json file as follows:

{
   "sample1": {
     "R1": [
       "<path_to_data>/sample1_run1_R1.fastq.gz",
       "<path_to_data>/sample1_run2_R1.fastq.gz",
     ],
     "R2": [
       "<path_to_data>/sample1_run1_R2.fastq.gz",
       "<path_to_data>/sample1_run2_R2.fastq.gz",
     ]
   }
}

To keep R1 and R2 reads paired, the order of files under R1 and R2 must be preserved by Snakemake. The relevant rule is split_fastq:

rule split_fastq:
    input:
        lambda wildcards: FILES[wildcards.sample][wildcards.read],
    ...
    shell:
        '''
        bash "{split_fastq}" ... {input:q} &> "{log}"
        '''

where

FILES[wildcards.sample][wildcards.read]

will evaluate to the list of files specified in samples.json. The question is whether that order is preserved by the {input:q} placeholder in the shell command.

Snakemake's documentation is unclear. For Snakemake v7, the Snakefiles and Rules page seems to indicate that order is not preserved only for named input files

Note that, when adding keywords or names for input or output files, their order won’t be preserved when accessing them as a whole via e.g. {output} in a shell command.

This is echoed on the Basics: An example workflow tutorial page:

If you name input or output files like above, their order won’t be preserved when referring to them as {input}.

This leaves the user to presume that order is preserved for positional input files.

Empirical testing

I've tested this empirically in minimal working examples, but I do not know if that is definitive.

Example test Snakefile:

FILES = {'sample1': ['a.bed', 'b.bed', 'c.bed']}

rule all:
    input:
        "sample1.out"

rule rule1:
    input:
        lambda wildcards: FILES[wildcards.sample]
    output:
        "{sample}.out"
    shell:
        '''
        echo {input:q}
        touch "{output}"
        '''

[bentyeh]

See question posed to the Snakemake developers here: snakemake/snakemake#3540