feat(parallelsearch): per-ID PEP for transient databases#11
Conversation
…odel-on-base, score-all-transients) Extracts the PEP component from PR #10 into its own branch/PR, so it can be reviewed and merged independently of the family-aware organism detection and other stacked features. Adds: - TransientPepAnalysisEngine: sealed EngineLayer class that trains ONE FastTree model on the (large, decoy-rich) base search and reuses it to assign PEP + PEP_QValue to every transient database, avoiding per-database retraining. - PepAnalysisEngine._randomSeed changed private→protected. - ParallelSearchTask integration: TrainBasePepModel() called after the base search on ManySearchTask; PEP assignments in PerformPostSearchAnalysis before metric collectors and stats. - BasicMetricCollector: 4 PEP-based confident count columns (PSM/peptide at 1% and 5% PEP_QValue). - TransientDatabaseMetrics: 4 PEP [Optional] properties with full Results-dictionary round-trip. - IsConfidentMatch static helper + OutputPepQValueThreshold const for testing. - 8 test files (4 new unit, 1 modified, 3 dedicated PEP tests, 1 E2E). Build: 0 errors. Tests: 19 PEP-specific pass, 14 existing pass.
| /// is the smallest value >= <paramref name="pep"/> (lower-bound). Because PEP_QValue is monotone in PEP, | ||
| /// this is the q-value at that confidence level. Returns 2 (the FdrInfo sentinel) when no curve exists. | ||
| /// </summary> | ||
| internal static double LookupBackgroundPepQValue(double pep, double[] pepAsc, double[] qByPep) |
There was a problem hiding this comment.
Is there a way to reuse the FDRAlignment that we have in EngineLayer/ParallelSearch/FdrAlignment instead of reinventing this from scratch?
There was a problem hiding this comment.
You are right that score->QValue and PEP->PEP_QValue are the same operation conceptually — both sort, compute target/decoy q-values via CalculateQValue, then look up. The implementation here calls FdrAnalysisEngine.CalculateQValue(..., pepCalculation: true), which IS reusing the existing FDR engine rather than reinventing it from scratch. A shared abstraction that parameterizes sort direction and the FdrInfo field would be a good follow-up refactor (the alignment services have extra state like notch tracking that the PEP curve does not need).
There was a problem hiding this comment.
The PEP baseline processing should be performed in the initialize method instead of the main run specific.
There was a problem hiding this comment.
Moved. TrainBasePepModel() now runs inside Initialize() after the baseline FDR and parsimony, before the base PSM/peptide/protein files are written, so the base output files include PEP and PEP_QValue in their FdrInfo.
| .ToList(); | ||
| } | ||
|
|
||
| // PEP: assign each transient PSM/peptide a posterior error probability + PEP_QValue (mapped onto the |
There was a problem hiding this comment.
This should happen in PerformPostSearchAnalysis instead of the Search method
There was a problem hiding this comment.
The PEP assignment block was already inside PerformPostSearchAnalysis (not in PerformSearch). The unified diff context was compressed and showed PerformSearch as the nearest enclosing function in the diff output, but the code at that location is inside PerformPostSearchAnalysis which starts at line 667. No change was needed.
| } | ||
| } | ||
|
|
||
| private const double OutputPepQValueThreshold = 0.05; |
There was a problem hiding this comment.
No new const. PEP and Q Value will operate upon the same threshold.
There was a problem hiding this comment.
Removed OutputPepQValueThreshold and the IsConfidentMatch helper that depended on it. PEP and QValue now operate on the same threshold.
There was a problem hiding this comment.
Removed OutputPepQValueThreshold and the IsConfidentMatch helper that depended on it. PEP and QValue now operate on the same threshold.
| /// is below pepQThreshold OR its score-based QValue is at or below qThreshold. When PEP is inactive, only the | ||
| /// score-based QValue is used. A null FdrInfo is never confident. | ||
| /// </summary> | ||
| internal static bool IsConfidentMatch(FdrInfo info, bool pepActive, double pepQThreshold, double qThreshold) |
There was a problem hiding this comment.
What is the purpose of this method? Do we need it?
There was a problem hiding this comment.
Removed. This was test-only scaffolding; testing can use explicit filtering instead. ConfidenceFilterTests.cs has also been removed.
…Match/OuputPepQValueThreshold, remove RT predictor field
2 participants
Extracts the PEP component from PR #10 (ms-analysis) into its own branch/PR so it can be reviewed and merged independently.
Summary
Trains ONE FastTree PEP model on the large, decoy-rich base (human) search and reuses it to assign a PEP + PEP_QValue to every transient database's hits. The transient databases are far too small to train their own model and are out-of-sample relative to the base search, so no cross-validation is needed for them.
New engine
Integration
Metrics
Tests (19 pass)
Build
Note
This is extracted from the larger ms-analysis PR (#10) that also includes family-aware organism detection, MSL library search, and visualization changes. Those will ship in follow-up PRs.