Weebill on frameshift repair - #313
Conversation
…kage. Squashed from the claude/weebill-pipe-mode-integration-28ixct branch, which was forked from a stale local main and so carried a duplicated copy of upstream history. Rebuilt as a single commit on top of frameshift_repair. - condense: --joint NNLS deconvolution of SingleM + weebill coverages, with sylph-only species injection (Regime 3), species-level pinning and a novel-lineage budget. - pipe/renew: run weebill automatically when the metapackage bundles a weebill database; --no-weebill, --output-weebill-sketch, --weebill-injection. - metapackage: v7 bundles a two-stage weebill DB and stores the read-name taxonomy DB as DuckDB. - lyrebird: leave weebill out, as viral profiling does not support it.
… conda. weebill is now packaged on bioconda, so depend on it rather than asking for the binary to be put on the PATH by hand. - update_metapackage: require gtdb_genome_reps in the config and fail at parse time if it is missing, instead of silently building a metapackage with no weebill database. Drops the build_weebill_db conditionals. - update_metapackage: run weebill via the update-metapackage pixi env, as the other rules do for their tools. - pixi: require weebill >=0.3.0 for singlem itself and for update_metapackage. Also for the regime3 test fixture. - Set gtdb_genome_reps and weebill_c in the older configs. R220 stores each of its genomes as both .fna and .fna.gz, so restrict its glob to the compressed copy to avoid sketching everything twice. - config-R232: fix gtdb_genome_reps, which pointed at a path that does not exist; the r232 genome reps are under gtdb_release232. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2ab6a1b3dd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if sample in weebill_sample_to_hits: | ||
| return weebill_sample_to_hits[sample] |
There was a problem hiding this comment.
When pipe profiles more than one read file, the annotated weebill TSV preserves Sample_file values such as sample.fq/paths, while SingleM sample IDs are derived with FastaNameToSampleName and have basenames/extensions stripped. This exact lookup therefore misses every sample whenever the weebill profile has more than one sample, falls through to the warning path, and silently produces marker-only profiles instead of the new joint SingleM+weebill result. Normalize the weebill Sample_file the same way before storing or matching it.
Useful? React with 👍 / 👎.
| running_weebill = (outputting_taxonomic_profile and not no_weebill and | ||
| not viral_profile_output and | ||
| len(metapackage.weebill_databases()) > 0) |
There was a problem hiding this comment.
If a user runs pipe on reads with --output-weebill-sketch and only requests an OTU/archive table for later renew, this predicate keeps running_weebill false because no taxonomic profile is being written. As a result WeebillProfiler.run_from_reads(..., sketch_output=...) is never called and the requested sketch directory is silently not produced, despite the CLI exposing the option for reuse by renew --input-weebill-sketch.
Useful? React with 👍 / 👎.
…eM sample IDs Weebill's Sample_file column carries the raw read file (e.g. "reads/mock.r1.fq"), while SingleM sample IDs are derived from the same file with FastaNameToSampleName (e.g. "mock.r1"). _weebill_hits_for_sample matched on exact equality, so any weebill profile with more than one sample missed every lookup and silently fell back to marker-only profiles. Normalizing the Sample_file value at parse time keeps both sides on the same key. Addresses a Codex review comment on PR #313.
running_weebill previously gated both the taxonomic-profile integration and the sketch-saving path, so a user requesting only an OTU/archive table with --output-weebill-sketch (for later 'renew --input-weebill-sketch') got no sketch written at all, with no indication anything was skipped. Split out weebill_available and run a sketch-only WeebillProfiler pass when a sketch is wanted but no taxonomic profile is being produced. Addresses a Codex review comment on PR #313.
…pr option Confirmed the sketch is already written as compressed .sylspc (via --compressed-database), not the uncompressed .sylsp weebill also produces. Note this in the help text, and point users chasing further compression at weebill's reference-based .sylspr sketches, which SingleM does not build automatically since 'renew --input-weebill-sketch' does not yet read them.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ad13f894a0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if len(weebill_sample_to_hits) == 1: | ||
| return list(weebill_sample_to_hits.values())[0] |
There was a problem hiding this comment.
When a multi-sample run has weebill detections for exactly one sample, this len(...) == 1 fallback is also taken for every other SingleM sample whose weebill output has no rows. Those samples then receive the detected sample’s weebill_hits; with the default joint profile settings, species from the wrong sample can be pinned/injected into their taxonomic profiles. Only use the single-entry fallback when the TSV truly has no sample column (e.g. the key is None) or after confirming the OTU table contains a single sample.
Useful? React with 👍 / 👎.
| weebill_db=[db for db, c in old_metapackage.weebill_databases()], | ||
| weebill_c=[c for db, c in old_metapackage.weebill_databases()]) |
There was a problem hiding this comment.
When supplementing an existing v7 metapackage, this copies the old weebill database into the new metapackage even though the new genomes added by supplement are not in that database. Downstream pipe uses the bundled DB by default and the joint condense pins species not reported by weebill to zero, so reads from newly added species can be suppressed or disappear from default taxonomic profiles. Rebuild the weebill DB with the supplemental genomes included, or omit/disable it so users profile the supplemented metapackage with marker-only condense.
Useful? React with 👍 / 👎.
The previous Unreleased section only listed the last few commits; backfill entries for joint weebill profiling, frameshift repair, and the various pipe/condense/prefilter fixes made since the last release.
…eebill DB on supplement - condense: A weebill TSV with hits for exactly one named sample is no longer treated as a fallback for other samples in a multi-sample run, which previously silently reused one sample's weebill hits for every other sample. - supplement: Raise an error rather than silently carrying forward a stale weebill database, since weebill has no way to add genomes to an existing database and the new genomes would be invisible to the joint profile. - update_metapackage: Use find -exec instead of find|grep|parallel to concatenate per-package seqs/taxonomies, avoiding accidental matches from grep -F on the package name.
|
@copilot resolve the merge conflicts in this pull request |
…G conflict Co-authored-by: wwood <15348+wwood@users.noreply.github.com>
Resolved. The only conflict was in |
Stacked on the frameshift repair branch - that needs merging first