Re-run a single script and confirm, case by case, that a six-gate triage ensemble keeps every critical patient at a critical tier with zero false discharges and zero broken safety invariants.
When several classifiers vote on how urgently a patient should be seen, a plain average can quietly bury a single alarm raised by one module. In triage that arithmetic is dangerous: one model shouting "critical" should not be talked down by four models that happen to favour a calmer label. SAFE-Gate is built around that asymmetry. Six independent gates look at the same case from different angles -- vital-sign red flags, weighted cardiovascular scoring, data completeness, syndrome matching, Monte Carlo dropout uncertainty, and a finite-state temporal check -- and their verdicts are combined by a lattice rule that always keeps the most cautious tier rather than the most popular one.
The merge step, which we call Adaptive Confidence-Weighted Conservative Merging (ACWCM), starts from that conservative floor and is allowed to step down by at most one tier, and only when the other gates agree with high confidence. Three invariants are stated as theorems, checked on every case, and accompanied by a per-case certificate that counts how many feature changes it would take to move the assigned tier upward. This code exists so a reviewer does not have to take any of that on faith. One command regenerates the cohort, runs SAFE-Gate next to four fusion and tree baselines, and writes out the safety audit.
The honest counterpart to the safety story is the cost of paying for it. Because the merge concentrates decisions on the critical and low-risk ends of the scale, the high-risk, moderate, and minimal tiers are rarely assigned, so headline accuracy and discharge specificity are low. We report those numbers as the price of the guarantee, not as competitive triage performance, and the script reproduces them exactly so the trade-off is visible rather than hidden.
All figures below come from run_all.py on the held-out 805-case test split (seed 42); the safety audit runs over the full 6,400-case cohort.
- Critical-tier sensitivity (R1/R2) is 100.0% -- 175 of 175 critical cases retained, exact 95% CI 97.9-100.0% -- with zero false discharges and a false-negative rate of 0.0%.
- All four safety properties (conservative preservation, abstention correctness, critical non-dilution, no false discharge) hold with 0 violations across 805 test cases and across all 6,400 generated cases.
- The guarantee is bought with conservative over-triage: ACWCM overall accuracy is 29.9% and R5 discharge specificity is 0.0%; over-triage falls from 66.2% under plain minimum selection to 42.2% under ACWCM.
- Against alternatives on the same split, arithmetic averaging misses 126 of 175 critical cases (28.0% sensitivity), while Dempster-Shafer keeps all critical cases (100.0% sensitivity, 34.8% accuracy) and a single gradient-boosted tree separates the deliberately sharp synthetic tiers almost perfectly (100.0% accuracy). The lattice contribution is the proven zero-violation floor, not aggregate accuracy.
- Gate ablation leaves critical sensitivity at 100.0% under every single-gate removal; only removing the syndrome gate (G4) lets the merge reach the discharge tier (R5 specificity 90.7%), identifying G4 as the binding constraint on de-escalation.
Caveat: the cohort is synthetic and the tier boundaries are by design separable, which is why a plain tree scores so high on accuracy. The defensible claim here is the safety floor, not the leaderboard number.
src/ ACWCM merge, six gates, risk lattice, safety certificates, baselines, theorem checks
data/ Synthetic generator + train/val/test JSON splits (4797 / 798 / 805)
results/ Metrics regenerated by run_all.py (summary.json, safety_check.json, CSVs)
evaluation/ Figure scripts + exported manuscript/supplementary figures (PNG/PDF)
scripts/ Evaluation driver and manifest generation
tests/ Full-system behaviour checks
run_all.py One-command reproduction entry point
git clone https://github.com/ChatchaiTritham/SAFE-Gate.git
cd SAFE-Gate
python -m venv .venv && source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install -r requirements.txtpython run_all.py # fixed seed 42; a few minutes on a standard workstationThe driver regenerates the synthetic cohort, runs SAFE-Gate plus the baselines, evaluates the safety properties, and writes results/summary.json, results/safety_check.json, results/theorem_verification.json, and the per-tier, baseline, ablation, and confusion-matrix CSVs. Every metric is computed from a fixed seed, so a second run reproduces the same values; the determinism of the pipeline has been confirmed by repeated execution.
The curated figure set is listed in FIGURE_MANIFEST.csv (four manuscript figures, two supplementary). Each is exported as both PNG and PDF at 300 DPI under evaluation/manuscript_figures/.
evaluation/manuscript_figures/safety_performance.png(SAFE-F1) -- the safety dashboard: critical-case capture, abstention behaviour, risk distribution, and sensitivity by tier. Read it as the headline evidence that the critical tier is caught in full.evaluation/manuscript_figures/baseline_comparison.png(SAFE-F2) -- SAFE-Gate set against the fusion and tree baselines. The point to notice is the gap in critical sensitivity, where arithmetic averaging collapses to 28.0% while the conservative methods stay at the top.evaluation/manuscript_figures/confusion_matrix.png(SAFE-F3) -- the R1-R5 confusion matrix for ACWCM. Look along the R1/R2 rows: nothing lands in the R4 or R5 columns, the visual confirmation that no critical case is discharged, and note the empty R3 and R5 prediction columns that explain the low accuracy.evaluation/manuscript_figures/per_class_metrics.png(SAFE-F4) -- precision, recall, and F1 per tier. It makes the trade-off concrete: full recall at R1 and strong recall at R4, with R2, R3, and R5 at zero recall under the released calibration.evaluation/manuscript_figures/risk_distribution.png(SAFE-F5, supplementary) -- the observed risk-tier distribution, useful for placing the evaluation in context.evaluation/manuscript_figures/support_distribution.png(SAFE-F6, supplementary) -- per-tier support counts for the test cohort.
If only the numbers are needed, results/baseline_comparison.csv carries the same comparison as SAFE-F2, and results/confusion_matrix.csv and results/tier_metrics.csv back SAFE-F3 and SAFE-F4 directly.
No human subjects are involved and no ethics approval is required. The cohort is fully synthetic: 6,400 vertigo-triage cases with 52 clinical features, generated under seed 42 and split into 4,797 training, 798 validation, and 805 test cases (data/synthetic/). The generator lives in data/generation/, so the data can be rebuilt rather than trusted as a static dump.
@unpublished{tritham_safegate,
author = {Tritham, Chatchai and Snae Namahoot, Chakkrit},
title = {Safety-First Gated Triage with Conservative Risk Merging},
note = {Manuscript under review},
year = {2026}
}Released under the MIT License (see LICENSE).
Chatchai Tritham — Department of Computer Science and Information Technology, Faculty of Science, Naresuan University, Phitsanulok 65000, Thailand. Email: chatchait66@nu.ac.th · ORCID: 0000-0001-7899-228X Chakkrit Snae Namahoot — same affiliation. Email: chakkrits@nu.ac.th · ORCID: 0000-0003-4660-4590
| Repository | Role |
|---|---|
| BASICS-CDSS | Beyond-accuracy evaluation methodology |
| TRI-X | Framework-level package |
| ORASR | Routing and safety-action component |
| DRAS-5 | Dynamic risk-state component |
| SAFE-Gate | Safety-gated ensemble framework |
| SynDX | Synthetic validation and explainability evidence |
| SURgul | SRGL/governance reproducibility component |
| TRI-X-CDSS | Integration and implementation package |
| Selective-CDSS | Risk-controlled selective-prediction (abstention) component |
| Causal-CDSS | Causal-inference evaluation component |
| Beyond-Accuracy | Simulation-based safety/calibration evaluation framework |