CryoParticleSegment

A particle-picking pipeline for cryo-EM micrographs using semantic segmentation, trained on ground-truth labels generated by binarizing 2D projections of 3D masks derived from reconstructed volumes.

This repository is the initial implementation of CRISP — my master's thesis work at NSYSU Applied Math (2024 May). The extended version was later published in Journal of Structural Biology (2025).

TL;DR

• Problem: cryo-EM particle picking needs pixel-level annotations, but the low SNR makes manual labeling unreliable.
• GT pipeline: reconstruct a 3D volume from labeled particles, project the 3D mask back to 2D, binarize with Minimum Cross Entropy Thresholding, and place per-particle masks onto a blank canvas at the known coordinates — no human pixel-level labeling needed.
• Segmentation: FCN-ResNet101 and DeepLabv3-ResNet50 trained on the generated labels.
• Post-processing: Dense CRF (post-processing only — gradients vanish, so end-to-end finetuning is not possible), ConvCRF (post-processing and end-to-end finetuned variants), plus a no-CRF baseline — 4 conditions per backbone.
• Finding: Dense CRF moves metrics by less than 0.3 percentage points. ConvCRF raises Recall but lowers Precision / IoU / F1; finetuning recovers part of the drop but does not exceed the model-only baseline. The model picks up additional candidates not in the CryoPPP labels; whether these are real particles or background artifacts was not separately verified in this work.

Pipeline

flowchart TD
    EMP[(EMPIAR-10017<br/>84× 4096² micrographs)]
    CP[(CryoPPP<br/>particles_selected.star<br/>+ cropped particle stack)]

    EMP --> Topaz["§3.2 Topaz preprocessing<br/><i>!topaz preprocess</i> CLI<br/>(2-GMM normalization)"]
    Topaz --> EdgeCrop["Edge crop 4096→3840<br/>(remove fringing)"]
    EdgeCrop --> Norm[/Normalized micrograph .npy/]

    CP -->|particle stack + metadata| EXT

    subgraph EXT["External tools (called from Mask Generator.ipynb)"]
        direction TB
        AB[cryoSPARC<br/>ab initio reconstruction]
        REF[cryoSPARC<br/>non-uniform refinement]
        PYEM[pyem csparc2star<br/>.cs → .star]
        REL[RELION<br/>2D projection of 3D mask]
        AB --> REF --> PYEM --> REL
    end

    REL --> ProjStack[/Per-particle mask projection<br/>.mrcs stack/]
    ProjStack --> Seg["§3.3.3 Binarization<br/>threshold_li (Li's MCET)<br/>+ clear_border<br/>+ remove_small_objects(64)"]
    Seg --> BinPart[/Per-particle binary mask/]

    CP -->|coords X, Y=4096-Y| Place["Place 256×256 mask patches<br/>OR-merged into 4096² canvas"]
    BinPart --> Place
    Place --> GT[/Pixel-level GT<br/>binary micrograph .png/]

    Norm --> DS["§3.4.1 Modeling/dataset.py<br/>CenterCrop 3840<br/>→ Random/Grid Crop"]
    GT --> DS
    DS --> Split{Train 58<br/>Val 9<br/>Test 17}

    Split --> SEG["§3.4 Semantic Segmentation<br/>FCN-ResNet-101 (crop 512, batch 7)<br/>or DeepLabv3-ResNet-50 (crop 1024, batch 2)<br/>cross-entropy + label_smoothing=0.1<br/>Adam, ReduceLROnPlateau"]

    SEG --> Prob[/Predicted probability/]

    Prob --> C1[model only]
    Prob --> C2["+ Dense CRF<br/>post-process only<br/>(no finetune: gradient=0)"]
    Prob --> C3["+ ConvCRF<br/>no finetune"]
    SEG --> C4["+ ConvCRF layer<br/>end-to-end finetuned<br/>(lr=1e-5)"]

    C1 --> M["Metrics:<br/>Acc / Recall / Prec / IoU / F1"]
    C2 --> M
    C3 --> M
    C4 --> M

Loading

Modules

Folder	Purpose
simulation/	Synthetic micrograph + GT generator (ASPIRE-based). Used for the Appendix 6.2 sanity check — not part of the main training pipeline.
MaskGeneration/	CryoPPP dataset loader (download + extract). The mask-generation logic itself lives in notebook/Mask Generator.ipynb.
Modeling/	Segmentation model, ConvCRF layer, dataset, trainer, metrics, learning-rate scheduler, plotting utilities.
notebook/	Jupyter notebooks covering GT generation, dataset preparation, and the ablation experiments (see below).

Notebook execution order

1. notebook/Mask Generator.ipynb — runs Topaz preprocessing, drives cryoSPARC + pyem + RELION for the GT pipeline, applies MCET binarization, and writes per-micrograph GT masks.
2. notebook/Dataset preprocessing.ipynb — prepares the train / val / test split for the model notebooks.
3. notebook/FCN_ResNet101/ and notebook/DeepLabv3_ResNet50/ — each folder contains 4 notebooks numbered 1 to 4, corresponding to the four conditions: 1 model only, 2 pydensecrf, 3 convcrf without finetune, 4 convcrf with finetune. Run in numerical order within each backbone folder.

External dependencies

This repo contains the analysis code and notebooks, but a full end-to-end run is not pip install away. The GT generation pipeline drives several external cryo-EM tools from inside Mask Generator.ipynb (some via CLI, some via GUI), so you will need to install them separately:

• cryoSPARC — ab initio reconstruction and non-uniform refinement (GUI-driven step).
• RELION — 2D projection of the refined 3D mask.
• pyem — converts cryoSPARC .cs results to RELION .star format.
• Topaz — !topaz preprocess is called for 2-GMM micrograph normalization.
• ChimeraX — used in the thesis for visualizing 3D density maps / masks (not required to reproduce numbers).

Python package install for the modeling side:

pip install -e .

See simulation/README.md for the standalone synthetic-data CLI (used in the sanity-check experiments).

Reproduce

1. Acquire data — download EMPIAR-10017 raw micrographs and the CryoPPP dataset (the latter via MaskGeneration/CryoPPP.py or manually). CryoPPP supplies ground_truth/empiar-10017_particles_selected.star and the cropped particle stack.
2. Generate GT — run notebook/Mask Generator.ipynb, which orchestrates Topaz preprocessing, cryoSPARC reconstruction (manual GUI step), pyem format conversion, RELION mask projection, MCET binarization, and canvas placement. Output: normalized .npy micrographs + binary .png GT.
3. Prepare dataset — run notebook/Dataset preprocessing.ipynb to produce the 58 / 9 / 17 train / val / test split.
4. Train and evaluate — pick a backbone folder (notebook/FCN_ResNet101/ or notebook/DeepLabv3_ResNet50/) and run notebooks 1 through 4 in order.
5. (Optional) Sanity check — use simulation/ to reproduce the Appendix 6.2 experiment on fully-labeled synthetic data.

Publication & extended version

The work in this repo was extended and published as:

Chung, S.-C., Chou, P.-C. (2025). CRISP: A modular platform for cryo-EM image segmentation and processing with Conditional Random Field. Journal of Structural Biology. DOI: 10.1016/j.jsb.2025.108239

The maintained / paper-grade implementation lives at phonchi/CryoParticleSegment (forked from this repo). It generalizes the pipeline into a modular platform with additional backbones, a refined CRF layer, and downstream 3D-reconstruction evaluation.

Citation

If you reference the thesis work:

Chou, P.-C. (2024). A particle picking pipeline for cryo-EM using semantic segmentation
and conditional random field. Master's thesis, Department of Applied Mathematics,
National Sun Yat-sen University.

For the journal version, please cite the JSB 2025 paper above.

Acknowledgements

This work was supervised by Prof. Szu-Chi Chung at NSYSU Applied Math. The cryo-EM domain knowledge, problem framing, and follow-up extensions into CRISP were developed under his guidance.

License

MIT