Skip to content

Record what actually ran: commit SHA, script id, real tool versions - #8

Merged
rec3141 merged 6 commits into
mainfrom
run-manifest
Jul 31, 2026
Merged

Record what actually ran: commit SHA, script id, real tool versions#8
rec3141 merged 6 commits into
mainfrom
run-manifest

Conversation

@rec3141

@rec3141 rec3141 commented Jul 31, 2026

Copy link
Copy Markdown
Owner

Follow-on to #6, which added run_manifest.json. That manifest had three holes, all of
which made a real dataset untraceable.

What went wrong

A 363-sample dataset analysed 2026-07-23 could not say what produced it. Its report gave
microscape 1.0.0 — a manifest string unchanged across this repo's entire history — and
a null commit. The .sif that ran it had been overwritten two days later. Neither that
container's main.nf nor the current one matches any commit here. Three versions in
play, no way to tell them apart.

Three causes, three fixes

workflow.commitId is null for every run launched from the .sif, because there is
no git repo inside it — and that is how this pipeline actually runs. The Dockerfile now
takes GIT_SHA/GIT_REF/BUILD_DATE build args into env vars and OCI labels; CI already
knew the SHA (it tags the image with it) and now passes it in. The manifest falls back to
that and records which source the commit came from, so "unknown" is visible rather
than silent.

script_id (the MD5 of main.nf) is now recorded unconditionally. It pins the source
even when both of the above are missing, and it is the only reason this was diagnosable.

containers recorded image names, whose tags give versions only if tags are pinned.
It now also runs bin/tool_versions.sh, which asks each binary — cutadapt, mafft,
FastTree, DADA2, DECIPHER, ape, papa2, microscape, python, R, Nextflow. The env spec says
what was requested; only the binary knows what was resolved, and a claim resting on
DADA2's error model is not reproducible against "bioconductor-dada2".

Two things found while testing it in the real container

papa2 and microscape are pip install git+… and both report a hardcoded 0.1.0 that
has never changed — exactly the meaningless version this PR exists to remove. PEP 610
writes the real commit into direct_url.json, so the probe now records that too
(microscape 0.1.0d9b4645d3cd4). Companion PRs make the version strings themselves
meaningful: rec3141/microscape#1 (merged) and rec3141/papa2#1.

The container build only triggered on Dockerfile and envs/**, while
COPY . /pipeline/ puts main.nf, bin/ and nextflow.config inside the image. Editing
the pipeline shipped nothing — which is the mechanism behind the drift above. Build
triggers now cover what the image actually contains. Baking the SHA in is only meaningful
if the image rebuilds when the code changes.

Verified

bin/tool_versions.sh was run inside the actual .sif on fir, not just a dev machine.
Everything answers — cutadapt 5.2, mafft 7.526, FastTree 2.2.0, R 4.5.3, dada2 1.38.0,
DECIPHER 3.6.0, ape 5.8.1, python 3.13.14 — and the only two tools whose version string
means nothing are pinned by commit instead. That probe run also caught two bugs in it: a
shell "command not found" was being reported as "present but not answering", and
nextflow -v is not a flag. Both fixed; vsearch dropped, since it is a danaSeq tool and
appears nowhere in this pipeline.

rec3141 and others added 6 commits July 23, 2026 23:49
Write a run manifest at workflow completion into ${outdir}/viz alongside
provenance.json: pipeline version/revision, Nextflow version, command line,
per-process container images (tool versions via tags), reference databases, and
the resolved analysis parameters (primer error rate, maxEE/truncQ/truncLen,
min_overlap, min_reads/samples/seqs, denoise engine). Guarded by try/catch so it
never affects pipeline success.

Lets omc-platform's AI Methods drafting + agents state the real workflow, versions,
and parameters instead of 'not specified in the pipeline outputs'.

Closes #5

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A 363-sample dataset analysed on 2026-07-23 could not say what produced it. Its report
gave "microscape 1.0.0" — a manifest string unchanged across the repo's history — and a
null commit. The .sif that ran it was overwritten two days later, and neither that
container's main.nf nor the current one matches ANY commit in this repository. Three
versions in play, no way to tell them apart.

Three causes, three fixes:

  workflow.commitId is null for every run launched from the .sif, because there is no
  git repo inside it. The Dockerfile now takes GIT_SHA/GIT_REF/BUILD_DATE build args and
  bakes them into env vars and OCI labels; CI already knew the SHA (it tags the image
  with it) and now passes it in. The manifest falls back to that and records which
  source the commit came from, so "unknown" is visible rather than silent.

  script_id (the MD5 of main.nf) is recorded even when both of the above are missing. It
  is what let this be diagnosed at all.

  containers/ recorded image names, whose tags give versions only if tags are pinned. It
  now also runs bin/tool_versions.sh, which ASKS each binary — cutadapt, mafft, FastTree,
  vsearch, DADA2, DECIPHER, ape, papa2, microscape, python, R — instead of reading the
  conda spec. The spec says what was requested; only the binary knows what was resolved,
  and a claim resting on DADA2's error model is not reproducible against
  "bioconductor-dada2". The probe never fails: an absent tool reports null, because a
  manifest with a hole is better than no manifest.
COPY . /pipeline/ puts main.nf, bin/ and nextflow.config inside the image, but the build
only triggered on Dockerfile and envs/. So editing the pipeline shipped nothing, and the
.sif drifted from git until neither the container's main.nf nor the running one matched
any commit.

This is the mechanism behind the provenance gap fixed in the previous commit: baking the
SHA in is only meaningful if the image is rebuilt when the code changes.
papa2 and microscape are installed from git URLs and report a version that never changes
— microscape has said 0.1.0 across every commit — so the previous probe would have
written exactly the meaningless version this whole change was meant to remove.

PEP 610 puts the real answer in direct_url.json: microscape 0.1.0 resolves to commit
765d9f90baa5. These two are also the tools most likely to change the numbers a claim
rests on, so they are the ones whose provenance matters most.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Probed inside the real .sif rather than a dev machine, which is the only way to learn
this. Merging stderr made a shell 'command not found' look like a version banner, so
vsearch — which is in neither env spec and is genuinely not installed — was reported as
'present but not answering'. Now the shell is asked first, and absent is null.

Also: nextflow -v is not a flag (it is -version), and vsearch is dropped from the list
rather than left to report a permanent null.

Everything else in the container does answer: cutadapt 5.2, mafft 7.526, FastTree 2.2.0,
R 4.5.3, dada2 1.38.0, DECIPHER 3.6.0, ape 5.8.1, python 3.13.14. The only tools whose
version string means nothing are papa2 and microscape, both pinned by commit instead.
main.nf conflicted in the manifest block: this branch rewrote the same lines to fall back
to the SHA baked into the container, so the resolution keeps this side, which is a
superset of main's for those fields.

Also records the two parameters main added while this branch was open —
primer_detect_reads and min_group_samples. A manifest that silently omits a parameter is
the same failure as one that reports a stale version, and merging without noticing is
exactly how it would happen.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant