COLMAP Split & Merge Pipeline

Tools for splitting a video-like image sequence (tested on the 7-Scenes dataset) into two overlapping sub-sequences, reconstructing each with COLMAP, merging the two sparse models back together, and comparing the result against ground-truth camera poses. It also includes a "full sequence" baseline path (no splitting) for comparison.

Every script in this repository is interactive and accepts one or more paths at each prompt (comma-separated) — there are no paths to hand-edit in the source before running anything.

---

1. Repository Layout

colmap-split-merge/
├── dataset/                         # Example 7-Scenes-style frames (a few sample frames only)
│   ├── frame-000000.color.png
│   ├── frame-000000.depth.png
│   ├── frame-000000.pose.txt
│   └── ...
├── install_torch.py                 # Detects your GPU/CUDA and installs a matching PyTorch build
├── requirements.txt
├── README.md
└── src/
    ├── dataset/                     # Step 1: build split/full workspaces from raw frames
    │   ├── full.py                  #   - one workspace with the whole sequence (no split)
    │   ├── overlap.py               #   - fixed center, varying overlap size (% or frame count)
    │   ├── center.py                #   - fixed overlap, varying center frame
    │   └── mean.py                  #   - overlap sampled around a target 3D-point density
    ├── descriptor.py                # Step 2 (optional): VGG16 global-descriptor image-pair retrieval
    ├── reconstruction.py            # Step 3: unified COLMAP reconstruction + merge (+ built-in recovery)
    ├── analyze.py                   # Step 4 (optional): 3D-point density profile, needed by dataset/mean.py
    ├── full_reconstruction_visualize.py  # Step 5: per-frame point-count bar chart (matplotlib)
    └── visualize.py                 # Step 5: Open3D viewer + ATE accuracy metrics vs. ground truth

The dataset/ folder in this repo only ships a handful of sample frames so you can see the expected file naming. For a real run, point the scripts at a full 7-Scenes (or 7-Scenes-formatted) scene directory.

---

2. Core Requirements

• Python 3.11 or 3.12
• COLMAP installed as a CLI binary, with colmap available on your system PATH. Verify with:

  colmap -h

• (Optional, recommended) NVIDIA GPU + CUDA driver — several steps (feature extraction, matching, the VGG16 descriptor network) run much faster on GPU. CPU-only works but is slow for anything but the sample dataset.

---

3. Installation

# 1. Clone the repository
git clone https://github.com/NeelayK/colmap-split-merge.git
cd colmap-split-merge

# 2. Create and activate a virtual environment
python -m venv venv

# Windows (PowerShell)
.\venv\Scripts\Activate.ps1
# Linux / macOS
source venv/bin/activate

# 3. Install the hardware-independent dependencies
pip install -r requirements.txt

# 4. Install PyTorch — this detects your GPU/CUDA version and installs a matching
#    build, offers a CPU-only build, or lets you install it yourself
python install_torch.py

Confirm COLMAP is reachable from the same shell:

colmap -h

Why isn't torch in requirements.txt? torch/torchvision wheels are hardware-specific — the correct one depends on whether you have an NVIDIA GPU and which CUDA version your driver supports. A single pinned line in requirements.txt would either miss GPU acceleration or fail to install on some systems, so it's handled separately by install_torch.py, which calls nvidia-smi to detect your CUDA version and installs the closest matching official PyTorch build (or lets you opt out and install manually from pytorch.org/get-started/locally).

---

4. Dataset Format

Scripts expect a 7-Scenes-style folder of frames, one triplet per frame:

frame-000000.color.png     # RGB image (also accepts plain .png/.jpg/.jpeg if no .color.png files exist)
frame-000000.depth.png     # depth image (not used by this pipeline, safe to omit)
frame-000000.pose.txt      # 4x4 camera-to-world matrix, whitespace-separated, 4 rows x 4 cols

Ground truth pose file example (frame-000000.pose.txt):

 9.9935108e-001  -1.5576084e-002   3.1508941e-002  -1.2323361e-001
 9.2375092e-003   9.8130137e-001   1.9211653e-001  -1.1206967e+000
-3.3912845e-002  -1.9170459e-001   9.8083067e-001  -9.8870575e-001
 0.0000000e+000   0.0000000e+000   0.0000000e+000   1.0000000e+000

visualize.py matches predicted cameras to ground truth by filename stem (e.g. frame-000000), so keep the frame-XXXXXX naming convention when using your own data, or adjust load_ground_truth_poses() in visualize.py accordingly.

---

5. Pipeline Walkthrough

Run python src/<script>.py and answer the prompts. Every script that accepts a directory path will also accept multiple, comma-separated directories so you can batch-process several scenes/configurations in one run.

Step 1 — Build workspaces from the raw dataset (src/dataset/*.py)

Each of these copies a selected frame range into <workspace>/<config_name>/images/ plus bookkeeping files, and prompts for: Dataset Source Directory, Base Workspace Directory, and a start/end frame range.

Script	What it produces	Extra prompts
full.py	One workspace full_sequence_<start>_to_<end>/ — the whole range, no split. Writes reconstruction_frame_list.txt.	—
overlap.py	One workspace per requested overlap value, e.g. overlap_20pct/, split around a fixed center frame with varying overlap size (percentage or frame count). Writes dataset1_list.txt / dataset2_list.txt.	overlap mode (% or frames), fixed center frame, comma-separated overlap values
center.py	One workspace per requested center frame, e.g. center_120_overlap_20pct/, with a fixed overlap size but varying center frame.	overlap mode + value, comma-separated center frame indices
mean.py	Overlap frames sampled probabilistically around target 3D-point-density values, using a precomputed scene_density_profile.json (see analyze.py below). Produces mean_<value>_common_<n>/.	path to scene_density_profile.json, number of common frames, sampling window radius, comma-separated target means

Every generated workspace contains an images/ folder plus, for split configurations, dataset1_list.txt and dataset2_list.txt (image filenames belonging to each half — these lists are what reconstruction.py uses to run two separate COLMAP mapping passes).

mean.py requires a density profile JSON produced by analyze.py (Step 4) from a prior full-sequence reconstruction — run a full.py reconstruction first if you want to use this splitting mode.

Step 2 (optional) — Global descriptor image-pair retrieval (src/descriptor.py)

Extracts a VGG16 global descriptor per image and writes the top-K most similar image pairs to image_pairs_dataset1.txt / image_pairs_dataset2.txt (or image_pairs_global.txt for a full-sequence workspace), for use by reconstruction.py in descriptor-matching mode instead of COLMAP's exhaustive/sequential matchers.

Prompts for one or more root workspace directories (comma-separated) and a top-K value. Automatically uses your GPU if torch.cuda.is_available().

python src/descriptor.py

Step 3 — Reconstruct + merge (src/reconstruction.py)

A single unified script that replaces the previous separate exhaustive / sequential / descriptor-matching / bulk / merge-recovery scripts. It:

1. Prompts for one or more workspace root directories (comma-separated).
2. Prompts for a matching strategy:
   • Exhaustive — colmap exhaustive_matcher (ALIKED + LightGlue). Most robust, slowest for long sequences.
   • Sequential — colmap sequential_matcher (overlap window = 10). Fast for temporally ordered video-like sequences.
   • Descriptor-pairs — colmap matches_importer using the pair files from descriptor.py (must be run first for each workspace).
3. Prompts whether to use GPU for COLMAP feature extraction/matching.
4. For each discovered workspace configuration (overlap_*, center_*, mean_*, full_*):
   • Split configurations: maps each half into sparse1/ and sparse2/, then merges with colmap model_merger into merged/, then bundle-adjusts.
   • Full-sequence configurations: maps directly into sparse/, then bundle-adjusts.
5. Writes a reconstruction_timing.json per workspace with per-step timings.

Built-in merge recovery. COLMAP's mapper can split a sequence into several disconnected sub-models (sparse1/0, sparse1/1, ...) instead of one. Before every merge, reconstruction.py scans all numbered sub-models on each side and picks the one with the most registered 3D points — this is the same strategy the old, separate merger_recovery.py script used, now built directly into the merge step. This also means the script is safe to simply re-run: if a workspace's sparse1//sparse2/ already contain valid sub-models from a previous run but the merge step failed or picked a bad candidate, re-running reconstruction.py on that workspace skips straight to re-evaluating and re-merging the best sub-models, instead of redoing feature extraction and mapping.

python src/reconstruction.py

Step 4 (optional) — 3D-point density analysis (src/analyze.py)

Converts a full-sequence sparse model to text, counts visible 3D points per frame, writes scene_density_profile.json (consumed by src/dataset/mean.py) and a histogram PNG.

Prompts for how many scenes to analyze, then for each: the full-sequence workspace directory (its sparse model is auto-detected the same way visualize.py does), the raw/original source image directory, and an output directory (defaults to <workspace>/analysis).

python src/analyze.py

Step 5 — Visualize results

• full_reconstruction_visualize.py — matplotlib bar chart of registered 3D points per frame. Prompts for one or more scene workspace directories (comma-separated).
• visualize.py — the main results viewer. Prompts for how many scenes to visualize, then for each: a ground-truth dataset directory and a workspace root (either a single configuration folder, or a parent directory containing several). For each discovered configuration it:
  • Auto-detects the model directory (merged/, sparse/0, or sparse/).
  • Aligns predicted camera centers to ground truth via Umeyama (similarity) alignment and reports Absolute Trajectory Error (ATE) — RMSE, mean, and max, in millimeters.
  • Prints a full metric summary (pipeline type, point/camera counts, ATE, and any saved reconstruction_timing.json step timings).
  • Opens an interactive Open3D window with the aligned point cloud (colored by which sub-dataset saw each point) and camera frustums (colored by dataset membership: dataset 1 / dataset 2 / overlap / full-sequence / unassigned) plus ground-truth vs. predicted trajectory lines (green vs. red).

python src/full_reconstruction_visualize.py
python src/visualize.py

---

6. Typical End-to-End Run

# 1. Split the dataset (example: varying overlap by percentage)
python src/dataset/overlap.py
#   -> ./colmap_workspace/overlap_10pct/, overlap_20pct/, ...

# 2. (optional) Generate retrieval pairs instead of exhaustive/sequential matching
python src/descriptor.py
#   -> prompted for: ./colmap_workspace

# 3. Reconstruct + merge every workspace under ./colmap_workspace
python src/reconstruction.py
#   -> prompted for: ./colmap_workspace, matcher = exhaustive
#   -> writes sparse1/, sparse2/, merged/, reconstruction_timing.json per workspace

# 4. Inspect results
python src/visualize.py
#   -> prompted for: dataset dir = ./dataset, workspace root = ./colmap_workspace

If step 3 fails partway (e.g. a bad merge), just re-run python src/reconstruction.py with the same workspace path — the built-in recovery logic will pick up from the existing sub-models and retry the merge.

---

7. Output Directory Reference

Inside each generated workspace (e.g. colmap_workspace/overlap_20pct/):

images/                      # union of images used by this configuration
dataset1_list.txt            # image filenames in sub-sequence 1 (split configs only)
dataset2_list.txt            # image filenames in sub-sequence 2 (split configs only)
reconstruction_frame_list.txt# image filenames in the sequence (full configs only)
database.db                  # COLMAP feature/match database
sparse1/                     # COLMAP mapper output for sub-sequence 1 (split configs; may contain 0/, 1/, ...)
sparse2/                     # COLMAP mapper output for sub-sequence 2 (split configs; may contain 0/, 1/, ...)
merged/                      # colmap model_merger + bundle_adjuster output (split configs)
sparse/                      # COLMAP mapper (+ bundle_adjuster) output (full configs)
reconstruction_timing.json   # per-step wall-clock timings
image_pairs_dataset1.txt     # descriptor.py output (if used)
image_pairs_dataset2.txt     # descriptor.py output (if used)
image_pairs_global.txt       # descriptor.py output, full configs (if used)

---

8. Troubleshooting

• colmap: command not found — COLMAP isn't on your PATH; reinstall or add its install directory to PATH, then re-open your shell.
• Merge step reports it couldn't find valid sub-models — both sparse1/ and sparse2/ failed to register anything; try a larger overlap when generating the split, or check the COLMAP mapping logs for that workspace.
• A previous merge looks wrong / used the wrong sub-model — just re-run python src/reconstruction.py on the same workspace path; it will re-evaluate all sub-models by point count and re-merge automatically.
• visualize.py reports "Insufficient overlapping frames matched with ground truth" — fewer than 3 registered image names matched a *.pose.txt stem; check that your dataset uses the frame-XXXXXX naming convention under the dataset directory you provided.
• install_torch.py can't detect a GPU but you have one — make sure nvidia-smi runs successfully in the same shell; if it's not on PATH, install PyTorch manually per the printed link, or add the NVIDIA driver's install directory to PATH.