VidMap

Exploiting Temporal Structure for Video-Based Structure-from-Motion

Zador Pataki · Paul-Edouard Sarlin · Marc Pollefeys

ECCV 2026

VidMap is an offline Structure-from-Motion system for video. It combines temporal tracks, loop closures, metric depth, and global optimization to estimate camera poses, camera intrinsics, and a sparse 3D map.

Quick Start

🔧 Setup — Install dependencies and prepare the environment.
🚀 Run VidMap — Reconstruct a video or image sequence.
📷 Camera rigs, panoramas, and fisheye — Configure your cameras.
👀 Visualization — Explore your reconstruction in the browser or Rerun.
🛠️ Pipeline Configurations — Customize the frontend and mapping settings.
⚡ Speed up VidMap — Find settings to make extraction and mapping faster.
📈 Benchmarks — Prepare datasets and evaluate reconstruction accuracy.

Setup

We provide the Python package vidmap. From a clean Python environment, clone VidMap and its pinned model dependencies:

git clone --recursive https://github.com/cvg/vidmap.git && cd vidmap

Build and install COLMAP and its PyCOLMAP bindings from source. Last tested with: COLMAP b1972bc2. For GPU mapper acceleration, build them against Ceres 2.3 or newer with CUDA and cuDSS instead (see how to enable).

Then install PyTorch, xFormers, and VidMap:

pip install -e .

The frontend was last tested on Linux x86-64 with Python 3.10, PyTorch 2.14.0, TorchVision 0.29.0 and xFormers 0.0.35 on NVIDIA GPUs. Fast loading of cached compiled models requires PyTorch 2.10+.

The first frontend run automatically downloads approximately 9 GB of model checkpoints and caches approximately 1.2 GB of compiled models.

Optional visualization setup. Install Rerun:

pip install rerun-sdk

Execution

The repository includes a silent example video. Run the frontend to build its mapper inputs, then run mapping:

INPUT_DATA=./assets/example_video.mp4
OUTPUT_DIR=./example_output/
MAPPER_INPUTS="$OUTPUT_DIR/mapper_inputs"

python -m vidmap.frontend "$INPUT_DATA" \
  --output "$OUTPUT_DIR"

python -m vidmap.map \
  --mapper-inputs "$MAPPER_INPUTS" \
  --output "$OUTPUT_DIR"

Alternatively, run frontend and mapping together:

python -m vidmap.run \
  --input_data "$INPUT_DATA" \
  --output "$OUTPUT_DIR"

Alternatively, --input_data may point to a directory of extracted frames.

Both mapping commands write the final COLMAP model under $OUTPUT_DIR/rec. Resolved frontend_config.yaml and mapping_config.yaml are written directly under $OUTPUT_DIR.

See files and folders for image layouts, frame associations, configuration files and run artifacts.

Camera rigs, panoramas, and fisheye

Input	Uncalibrated mono	Calibrated mono	Uncalibrated rig	Calibrated rig
Pinhole	✓	✓	✗	✓
Distorted perspective	✓	✓²	✗	✓²
Fisheye	✓¹	✓²	✗	✓²
Stitched 360° panorama	N/A	✓	N/A	✗

Distorted perspective models: SIMPLE_RADIAL, RADIAL, OPENCV, FULL_OPENCV, SIMPLE_DIVISION, DIVISION. See COLMAP camera models for model definitions and parameter order.

Native OPENCV_FISHEYE, forward-facing rays only.
Undistorted to pinhole before mapping. Calibrated rigs may mix camera models.

Visualization

Browser viewer

Open the vidmap-viewer in a Chromium-based browser and choose a run folder containing rec/ and mapper_inputs/. The viewer requires internet access.

[Embed a reconstruction - click to expand]

python -m vidmap.visualization.html --run-dir "$OUTPUT_DIR"

This embeds the reconstruction in $OUTPUT_DIR/vidmap-viewer-embedded.html, so it opens without selecting a run folder.

Rerun playback

To render optimization playback, record solver states during mapping:

python -m vidmap.map \
  --mapper-inputs "$MAPPER_INPUTS" \
  --output "$OUTPUT_DIR" \
  --save-playback-trace

This writes the trace to $OUTPUT_DIR/playback_trace/.

Create solver-playback and flythrough recordings. Local images, frame timing, and missing ground truth are detected automatically:

VISUALS=./example_visuals/

python -m vidmap.visualization.rerun.playback "$OUTPUT_DIR" \
  --view isometric \
  --output "$VISUALS/solver-playback.rrd"

python -m vidmap.visualization.rerun.flythrough "$OUTPUT_DIR" \
  --reconstruction final \
  --view follow \
  --output "$VISUALS/flythrough.rrd"

[Flythroughs with lifted depth maps - click to expand]

Depth lift replaces the current sparse highlight with the RGB-colored full depth prediction. Retain those predictions during reconstruction with --cache-depth-maps, then create an RRD:

python -m vidmap.visualization.rerun.flythrough "$OUTPUT_DIR" \
  --depth-lift \
  --output "$VISUALS/depth-flythrough.rrd"

If the first frontend run did not cache full depth maps, backfill them without repeating tracking or verification:

python -m vidmap.frontend "$INPUT_DATA" \
  --output "$OUTPUT_DIR" \
  --mapper-inputs "$OUTPUT_DIR/mapper_inputs" \
  --cache-depth-maps

Open the recordings with Rerun:

rerun "$VISUALS/solver-playback.rrd"
rerun "$VISUALS/flythrough.rrd"

Don't forget to try VidMap on your favorite videos from the web.

Configuration

VidMap extends COLMAP's global mapping pipeline, for which we provide easily adjustable hyperparameters via configs.

VidMap uses Hydra and OmegaConf for fine-grained configuration with typed defaults. Frontend configs control tracks, keyframes, depth, and calibration, while mapping configs control global positioning and bundle adjustment. Frontend settings follow the pipeline itself: keyframe matching, selection, and features are grouped together; track matching and propagation form a separate stage.

Print all available frontend and mapping defaults:

from vidmap.configuration import FrontendConfig, MappingConfig, summarize_cfg

print(summarize_cfg(FrontendConfig()))
print(summarize_cfg(MappingConfig()))

[Custom configuration - click to expand]

Each YAML file overrides only the relevant defaults and can inherit another configuration with defaults:.

For example, define a compatible frontend and mapping pair:

# vidmap/configs/frontend/uncalib/custom.yaml
defaults:
  - base

keyframes:
  selection:
    max_normalized_keypoint_drift: 0.09

# vidmap/configs/mapping/uncalib/custom.yaml
defaults:
  - base

mapper:
  gp:
    first_pass:
      loss_lc_geometry:
        weight: 0.3

Select custom configs and adjust either stage inline with Hydra overrides:

python -m vidmap.run \
  --input_data "$INPUT_DATA" \
  --output "$OUTPUT_DIR" \
  --frontend-conf uncalib/custom \
  --mapping-conf uncalib/custom \
  frontend.keyframes.selection.max_normalized_keypoint_drift=0.08 \
  mapping.mapper.gp.first_pass.loss_lc_geometry.weight=0.4

FlexGlue is the default matcher. Append frontend.matcher.name=romav2 to the command above to use RoMaV2 instead.

[GPU acceleration - click to expand]

Enable GPU acceleration for global positioning and bundle adjustment (requires Ceres 2.3 or newer with GPU support; see Setup):

python -m vidmap.run \
  --input_data "$INPUT_DATA" \
  --output "$OUTPUT_DIR" \
  mapping.mapper.gp.solver_backend.use_cuda=true \
  mapping.mapper.ba.solver_backend.use_cuda=true

Recommended configurations

default: The default reconstruction pipeline, based on the method described in the paper.
smooth_trajectory: Experimental pipeline. Adds keyframe-coordinate trajectory smoothing to the default pipeline for improved robustness on long sequences, as explored on external datasets.

Speed up VidMap

Reducing the number of tracked points, using coarse-only keyframing, and using fewer propagation hops can speed up VidMap. Running global positioning on CUDA reduced total mapping time by about 13% on a tested LaMAR HGE sequence. The benefit of GPU acceleration for bundle adjustment is less clear.

python -m vidmap.run \
  --input_data "$INPUT_DATA" \
  --output "$OUTPUT_DIR" \
  frontend.tracks.propagation.max_kps=1000 \
  frontend.keyframes.matching.coarse_only=true \
  'frontend.tracks.propagation.multiflow_hops=[1,2,4,8]' \
  mapping.mapper.gp.solver_backend.use_cuda=true \
  mapping.mapper.ba.solver_backend.use_cuda=true

The table below provides insight into the accuracy–runtime tradeoffs. Extraction savings approximately exclude startup by subtracting the recorded first calls.

Setting	Stage measured	Time saved	Mean accuracy drop
Hops `[1,2,4,8]`	Extraction	~9.9%	~0%
Hops `[1,2,4]`	Extraction	~15.3%	0.385%
Hops `[1,2]`	Extraction	~21.3%	1.563%
Coarse keyframing	Extraction	~9.0%	0.053%
1,000 points, balanced	Mapping	~40%	0.632%
500 points, balanced	Mapping	~66%	2.067%

Benchmarks

Prepare data

By default, prepared data, caches, and experiments live under ./local/{datasets,cache,experiments}/<dataset>/, relative to the directory where the command is run. To use other paths, create ./vidmap-paths.toml or set VIDMAP_<DATASET>_{DATA,CACHE,EXP,TESTSETS}_DIR.

Spires and Hilti require the ROS bag reader (pip install -e '.[dataset-preparation]').

Then select and prepare a benchmark dataset:

DATASET=lamar  # or: euroc, eth3d_slam, crocodl, spires, hilti
python -m vidmap.datasets.prepare --dataset "$DATASET"

This downloads and validates the supported evaluation inventory, then converts it into the dataset layouts below. LaMAR intentionally prepares only the images referenced by its benchmark testset: sessions and images absent from that testset are not retained. Downloads resume after interruption and completed sequences are reused. Validated source archives are retained by default; pass --delete-downloads to remove them after successful preparation. EuRoC and ETH3D-SLAM prepare only cam0 by default; --mode stereo or --mode all prepares both cameras, reusing existing cam0 images in the same folder. For Spires and Hilti, add --workers N to prepare N recordings concurrently.

[Expected prepared dataset layouts - click to expand]

Storage for the supported evaluation inventories:

Dataset	Download	Prepared
LaMAR (sample-300)	45.2 GB	24.0 GB
LaMAR (complete)	144.5 GB	76.7 GB
EuRoC (monocular)	24.7 GB	4.7 GB
EuRoC (rig)	24.7 GB	9.6 GB
ETH3D-SLAM (monocular)	18.6 GB	18.6 GB
ETH3D-SLAM (rig)	37.3 GB	37.3 GB
CroCoDL	15.0 GB	14.0 GB
Oxford Spires (rectified monocular, default)	89.7 GB	29.3 GB
Oxford Spires (fisheye monocular)	89.7 GB	15.8 GB
Oxford Spires (rectified rig)	89.7 GB	89.8 GB
Hilti 360°	132.1 GB	120 GB

Download sizes are the checksum-pinned source archives; prepared sizes cover the canonical dataset after conversion. LaMAR archives are session-level, so the download includes unused frames inside each required session ZIP; only the selected testset images are retained. Retaining downloads requires roughly the sum of both columns, plus conversion staging. Hilti's prepared size is estimated from 120 panoramas sampled across all 30 recordings.

After preparation, the selected dataset is available under ./local/datasets/<dataset>/, with its benchmark definitions under ./local/testsets/<dataset>/:

local/
├── datasets/<dataset>/<scene>/<recording>/
│   ├── preparation.json
│   └── <variant>/
│       ├── images/cam0/...         # cam1/, cam2/, ... when available
│       └── rec/                   # Matching calibration and reference poses
└── testsets/<dataset>/<scene>/<mode>.yaml

Dataset	Variant(s)
LaMAR, ETH3D-SLAM, CroCoDL	`native`
EuRoC	`rectified`
Oxford Spires	`native`, `rectified`
Hilti 360°	`panorama`

[Examples for each dataset - click to expand]

local/datasets/
├── lamar/CAB/ios_2021-06-02_14.21.49_000/
│   ├── preparation.json
│   └── native/
│       ├── images/cam0/...
│       └── rec/
├── euroc/machine_hall/MH_01_easy/
│   ├── preparation.json
│   └── rectified/
│       ├── images/cam0/...         # cam1/ when stereo is prepared
│       └── rec/
├── eth3d_slam/plant_1/0/
│   ├── preparation.json
│   └── native/
│       ├── images/cam0/...         # cam1/ when stereo is prepared
│       └── rec/
├── crocodl/ios-ARCHE_D2/ios_2023-07-12_20.46.14_000/
│   ├── preparation.json
│   └── native/
│       ├── images/cam0/...
│       └── rec/
├── spires/2024-03-12-keble-college-02/0/
│   ├── preparation.json
│   ├── native/
│   │   ├── images/cam0/...
│   │   └── rec/
│   └── rectified/
│       ├── images/
│       │   ├── cam0/...
│       │   ├── cam1/...
│       │   └── cam2/...
│       └── rec/
└── hilti/floor_1_2025-05-05_run_1/0/
    ├── preparation.json
    └── panorama/
        ├── images/cam0/...
        └── rec/

local/testsets/
├── lamar/CAB/sample.yaml
├── euroc/machine_hall/mono.yaml
├── eth3d_slam/plant_1/mono.yaml
├── crocodl/ios-ARCHE_D2/complete.yaml  # Also sample-*.yaml
├── spires/2024-03-12-keble-college-02/
│   ├── undistorted-mono.yaml
│   ├── fisheye-mono.yaml
│   └── undistorted-rig.yaml
└── hilti/floor_1_2025-05-05_run_1/panorama.yaml

Each rec/ contains calibration and camera poses, without 3D points. Select a benchmark with --mode; its testset specifies the recording, image variant and image IDs.

Run and evaluate

FRONTEND_CONF=uncalib/base
MAPPING_CONF=uncalib/base
# Omit --name below to derive the output name from the config pair.
OUTPUT_NAME=default

python -m vidmap.run_for_benchmark \
  --dataset "$DATASET" \
  --frontend-conf "$FRONTEND_CONF" \
  --mapping-conf "$MAPPING_CONF" \
  --name "$OUTPUT_NAME"

Inline Hydra overrides use the same frontend.<field>=<value> and mapping.<field>=<value> prefixes as vidmap.run.

The command reconstructs and evaluates every selected target, then prints the final aggregate report across all completed trajectories. Add --run-only to skip only this final report; per-trajectory evaluation still runs.

[LaMAR - click to expand]

LaMAR defaults to complete with wate_auc evaluation.

python -m vidmap.run_for_benchmark \
  --dataset lamar \
  --frontend-conf uncalib/gtkf-base \
  --mapping-conf uncalib/base

For the trajectories used in the paper, prepare sample-300 and run sample:

python -m vidmap.datasets.prepare -d lamar --profile sample-300

python -m vidmap.run_for_benchmark \
  --dataset lamar --mode sample \
  --frontend-conf uncalib/gtkf-base \
  --mapping-conf uncalib/base

Use separate dataset/testset roots to keep both profiles.

[EuRoC - click to expand]

EuRoC defaults to mono with pose_auc evaluation.

python -m vidmap.run_for_benchmark \
  --dataset euroc \
  --frontend-conf uncalib/base \
  --mapping-conf uncalib/base

For calibrated stereo:

python -m vidmap.run_for_benchmark \
  --dataset euroc --mode stereo \
  --frontend-conf calib/stereo \
  --mapping-conf calib/base

[ETH3D-SLAM - click to expand]

ETH3D-SLAM defaults to mono with pose_auc evaluation.

python -m vidmap.run_for_benchmark \
  --dataset eth3d_slam \
  --frontend-conf uncalib/base \
  --mapping-conf uncalib/base

For calibrated stereo:

python -m vidmap.run_for_benchmark \
  --dataset eth3d_slam --mode stereo \
  --frontend-conf calib/stereo \
  --mapping-conf calib/base

[CroCoDL - click to expand]

Evaluate the complete CroCoDL benchmark. CroCoDL selects wate_auc by default.

python -m vidmap.run_for_benchmark \
  --dataset crocodl \
  --frontend-conf uncalib/base \
  --mapping-conf uncalib/base

[Oxford Spires - click to expand]

Oxford Spires has 13 scenes and defaults to rectified monocular images (undistorted-mono). Select a mode with --mode.

Mode	Frontend config	Mapping config
`undistorted-mono` (default)	`uncalib/base`	`uncalib/base`
`undistorted-mono`	`calib/base`	`calib/base`
`fisheye-mono`	`uncalib/fisheye`	`uncalib/geocalib`
`undistorted-rig`	`calib/rig`	`calib/base`

python -m vidmap.datasets.prepare --dataset spires --mode undistorted-rig

python -m vidmap.run_for_benchmark \
  --dataset spires --mode undistorted-rig \
  --frontend-conf calib/rig --mapping-conf calib/base

Evaluation uses scale-aligned WATE-AUC on cam0.

[Hilti 360° - click to expand]

Hilti has 30 sequences and defaults to panorama mode. Select perspective views with rig.projection.

python -m vidmap.run_for_benchmark \
  --dataset hilti \
  --frontend-conf calib/panorama --mapping-conf calib/base

Evaluation uses scale-aligned WATE-AUC on view 0.

[Results - click to expand]

WATE-AUC with late principal point refinement, at 5% error thresholds (%, higher is better). Results are averaged equally across eligible sequences. ATE is the mean per-sequence translation RMSE after global Sim(3) alignment (meters, lower is better), evaluated at GT-supported estimated poses:

Dataset	ATE (m) ↓	10 m @ 0.5 m	25 m @ 1.25 m	50 m @ 2.5 m	100 m @ 5 m	Full @ 5%
LaMAR (sample-300, 50 sequences)	0.916	92.78	93.87	92.90	91.99	90.83
CroCoDL (64 sequences)	0.284	95.38	96.75	96.70	96.60	98.03
LaMAR (complete, 128 sequences)	2.095	91.47	92.77	91.85	90.90	89.71
Oxford Spires (rectified monocular, 10 sequences)	0.123	98.98	99.08	99.06	99.06	99.52
Oxford Spires (three-camera rig, 13 sequences)	0.088	99.17	99.30	99.36	99.43	99.68
Hilti (three-view, 30 sequences)	0.253	94.55	95.85	96.56	96.81	97.93

Pose AUC, averaged equally across sequences (%, higher is better):

Dataset	ATE (m) ↓	5 cm	10 cm	50 cm	1 m	10 m
EuRoC (11 sequences)	0.044	29.32	60.40	92.04	96.02	99.60
ETH3D-SLAM (55 sequences)	0.066	81.64	87.78	94.79	96.79	99.54

[Optionally re-evaluate saved results - click to expand]

Recompute evaluation reports without rerunning reconstruction:

python -m vidmap.evaluate_results \
  --dataset "$DATASET" \
  --name "$OUTPUT_NAME"

BibTeX citation

If you use any ideas from the paper or code from this repository, please consider citing:

@inproceedings{pataki2026vidmap,
  author    = {Zador Pataki and
               Paul-Edouard Sarlin and
               Marc Pollefeys},
  title     = {{VidMap: Exploiting Temporal Structure for Video-Based Structure-from-Motion}},
  booktitle = {ECCV},
  year      = {2026}
}

Acknowledgments

VidMap builds on COLMAP, MP-SfM, and hloc. For the video visualizations, dense matches were densely triangulated using ImLoc.

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
assets		assets
docs		docs
extensions/colmap		extensions/colmap
tests		tests
third_party		third_party
vidmap		vidmap
.gitattributes		.gitattributes
.gitignore		.gitignore
.gitmodules		.gitmodules
LICENSE		LICENSE
README.md		README.md
pyproject.toml		pyproject.toml

Navigation Menu

Repository navigation

About

Repository files navigation

VidMap

Exploiting Temporal Structure for Video-Based Structure-from-Motion

Quick Start

Setup

Execution

Camera rigs, panoramas, and fisheye

Visualization

Browser viewer

Rerun playback

Configuration

Recommended configurations

Speed up VidMap

Benchmarks

Prepare data

Run and evaluate

BibTeX citation

Acknowledgments

About

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

VidMap

Exploiting Temporal Structure for Video-Based Structure-from-Motion

Quick Start

Setup

Execution

Camera rigs, panoramas, and fisheye

Visualization

Browser viewer

Rerun playback

Configuration

Recommended configurations

Speed up VidMap

Benchmarks

Prepare data

Run and evaluate

BibTeX citation

Acknowledgments

About

Resources

Stars

Watchers

Forks

Contributors

Languages