Skip to content

Add conclave TS1 submission#42

Merged
eacharles merged 2 commits into
LSSTDESC:mainfrom
rhw:submit/conclave
Jul 7, 2026
Merged

Add conclave TS1 submission#42
eacharles merged 2 commits into
LSSTDESC:mainfrom
rhw:submit/conclave

Conversation

@rhw

@rhw rhw commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

conclave — Task Set 1 submission

conclave is a committee ensemble of three complementary photo-z estimators — PZFlow, GPz,
and FlexZBoost — combined with convex-QP optimal weights (chosen to minimize the ensemble
CDE loss on held-out data) and a global-PIT recalibration, on LSST 6-band + Roman (Y/J/H)
photometry. The QP drives weak members to zero, so the effective committee selects its own members.

Performance (our held-out split, identical metrics, mean over 5 random seeds)

σ_MAD outlier CDE loss PIT-KS
Cardinal — best single (PZFlow) 0.0101 0.40% −18.8 0.033
Cardinal — conclave 0.0092 0.18% −20.1 ± 0.15 0.027
Flagship — best single (PZFlow) 0.0105 0.47% −17.1 0.037
Flagship — conclave 0.0097 0.31% −18.6 ± 0.11 0.021

It beats the best single estimator on every metric in both simulations, sits ~10× inside the top
scoring tier on every graded metric (incl. at 1-year depth), and beats every accepted method we
could reproduce through the same rig (KNN −12.2/−13.6, DNF −8.4/−9.6, BPZ −1.1/−2.2, trainZ floor).

Deliverables

  • tests/test_conclave.py — the two run_taskset_1_* entry points (+ taskset_2 delegators),
    delegating to the pip-installable conclave package.
  • requirements_conclave.txt — installs conclave @ git+https://github.com/rhw/conclave@ts1-v1
    (BSD-3-Clause), which pulls the RAIL estimator stack + qp-prob + tables_io transitively.
  • .github/workflows/submit_conclave.yaml — CI workflow for this submission.
  • Pre-made estimates + pretrained models for cardinal/flagship × 1yr/10yr, hosted as the
    ts1-v1 release tarball (SUBMISSION_URL).

CI note

The full method trains three estimators on the ~100k training set × 4 sim/scenario, which exceeds
the GitHub-runner budget. The CI workflow sets PZDC_CI_MAX_TRAIN=4000 so the train+estimate path
trains on a subsample — CI proves the pipeline runs and emits valid p(z) for every object;
estimation runs on the full test set, and the shipped full-scale-trained models/estimates (release
tarball → estimation-only path) carry the real performance.

Validation

The submission passes the upstream harness end-to-end — run_taskset_1 over both simulations and
both depths, all pre-made / estimation-only / train+estimate checks green.

🤖 Generated with Claude Code

rhw and others added 2 commits July 2, 2026 08:19
…t, QP weights, global-PIT)

tests/test_conclave.py delegates to the pip-installable conclave package
(github.com/rhw/conclave, pinned ts1-v1); requirements_conclave.txt installs it + RAIL deps;
submit_conclave.yaml runs the CI with PZDC_CI_MAX_TRAIN so the train path subsamples (a full
3-estimator retrain on 100k x4 exceeds the CI budget). Pre-made full-scale estimates + models
ship via the release tgz for the estimation-only path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… only)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@eacharles eacharles self-requested a review July 7, 2026 16:20
@eacharles eacharles merged commit 0303f85 into LSSTDESC:main Jul 7, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants