3D reconstruction roundup 3: five releases, and what you can actually check

mdjsonmcpcopy↗ai

2026-11-09 · 16 min · 3d · slam · gaussian-splatting · optimization · datasets · benchmarks

Why read this

Notable· top 60%

Five 3D releases checked against their own tables, a recomputed Trainspotting split, the LaMAria board, and a certificate toy in numpy.

• Original analysis
• A new technique
• Concrete numbers to act on

·3D & spatial·Runs on a consumer GPU·Mixed licences·Research roundup

How this was scored

Is it new?

2 of 3: A real new idea, method or capability

Can I trust it?

2 of 3: Measures key facts from files, code or configs

Can I run it?

1 of 3: API-only, gated or restrictive licence

Will I understand it?

2 of 3: Mechanism from first principles with figures

Can I act on it?

2 of 3: A concrete recipe, numbers or comparison

Will it last?

1 of 3: Relevant for months

Does it affect many?

1 of 3: A specialist community

Only here?

2 of 3: A teardown or measurement few others did

Score 60 of 100, ranked 397 of 701 rated articles. Each question is answered 0–3 by hand, and a 3 is rare. How articles are scored

The first roundup sorted papers by the problem they attack, and the second by the old idea each one bolts onto a feed-forward model. This week's five were picked for a different reason: each one arrived as a post with a project page, and the question was how much of it a reader can check from outside. The answer runs from "the per-sample results are in the repository, recompute the table yourself" (Trainspotting) to "the only evidence is a public leaderboard row" (FPV-SLAM). For each: what is new, what is released, and what is checkable.

XM4D: a certificate for the camera, then one for the motion

What is new. Dynamic scene reconstruction from one video usually ends in a large nonconvex photometric optimisation, started from whatever the camera estimate gives it. XM4D, from Harvard's Computational Robotics group, splits the problem in two and makes each half a problem with a provable global optimum. Static point tracks are lifted to 3D with a monocular depth model, and a scaled bundle adjustment over camera poses, per-frame depth scales and landmarks is reduced, by eliminating translations and landmarks analytically, to a quadratically constrained quadratic program over scaled rotations. Its semidefinite relaxation is solved with a GPU Burer–Monteiro solver and a Riemannian staircase, which returns a certificate when the answer is globally optimal (reported). The dynamic half represents motion with an embedded deformation graph, a set of control nodes each with a rotation and a position per frame, and shows that tracking, rigidity and smoothness terms reduce to a rotation-only problem of the same shape, so the same solver certifies it. Only then does 4D Gaussian splatting refine appearance.

The conference version, XM, did the static half and won RSS 2025's best systems paper; the new pieces are the dynamic stage and a "uniform sphere" constraint on the depth scales that fixes global scale without anchoring the first frame (reported).

expand

What the tables show. Table 2 of the paper averages camera error per dataset over seven benchmarks. XM4D is best on Replica and TUM and in translation on Sintel. It is not best everywhere: on Syn4D, NVIDIA and DyCheck MegaSAM has the lower error (DyCheck ATE 0.005 against 0.008, NVIDIA 0.024 against 0.038), and on IMC's unordered photo collections COLMAP wins by a wide margin (ATE 0.003 against 0.047), all reported. The paper says this plainly. The runtime table is where it pulls ahead: the full dynamic reconstruction takes 34.6 minutes a DyCheck scene against 40.5 for MoSca and 512.4 for Shape of Motion, and on Replica room0 it reaches 2,000 frames in 55 minutes, where VGGT runs out of GPU memory at 200 frames (reported). Novel-view rendering is best on all three dynamic benchmarks, for example 19.51 mPSNR on DyCheck against MoSca's 18.91 (reported).

The paper's own limitations paragraph is the one to read: the certificate covers the optimisation, not its inputs. If the monocular depth or the point tracks are wrong, the solver returns the certified optimum of the wrong problem. Its memory also grows quadratically with the number of frames, because the data matrices are stored densely (reported).

What is released. The paper says "visualizations and code are available on the project page". The page's Code button points at ComputationalRobotics/XM4D-release, which git could not reach without credentials on 10 November, so it is private or not yet created (measured). The public ComputationalRobotics/XM4D repository is the project page's own HTML, viewer and assets, 294 files, last commit 9 November (measured). The static solver from the conference paper is public under MIT:

What is checkable, by running something small. A certificate is the unusual claim here, and it is easy to see on a toy. The smallest problem of XM4D's kind is rotation synchronisation: recover twenty 3D rotations from noisy measurements of the relative rotation between thirty pairs of them. I wrote it in numpy (2.5.3, Python 3.13): plain Riemannian descent on the rotations from ten random starts per problem, and separately the semidefinite relaxation solved the way XM does, by Burer–Monteiro factorisation with a staircase that starts at rank 3, checks the dual certificate (the smallest eigenvalue of the certificate matrix must be non-negative), and adds a row when it fails. Twenty random problems per noise level; noise is a rotation with the given standard deviation in radians per axis.

Three things come out of it (all measured, about four minutes of CPU per run of the sweep). First, local search fails even when the problem is easy. At 0.05 radians of noise, 27 of 200 random starts stop at a cost above the certified optimum, often far above it: the median stuck run ends at about 86 times the optimal cost. That is the kind of failure the paper's Figure 3 boxes in orange (reasoned). As noise grows the local solver fails more often, 59 of 200 at 0.3 and 185 of 200 at 0.6. Nothing in the local solver's output tells you which runs those are; the certificate does.

Second, the staircase behaves as the XM4D project page describes for its own solver ("the rank-3 solution fails the certificate, so the solver lifts to rank 4"). At low noise most problems certify at rank 3 straight away, 16 of 20 at 0.05; the rest climb to rank 4 or 5, and every problem at every noise level ended certified, 140 of 140.

Third, a certificate is only a proof of the solution you want when the relaxation is tight. Up to 0.3 radians all 20 certified solutions round back to true rotations at exactly the certified cost, so they are provably the global optimum. At 0.5 only 10 of 20 do, and at 0.6 only 5. In the others the semidefinite optimum genuinely has rank 4 or 5 (I checked the eigenvalues of the solution matrix on sixteen more problems), so the certificate certifies the relaxation, which is a lower bound, and the rounded rotations sit a few percent above it: 18.00 against 17.43 in one case. XM4D says its relaxation is "empirically tight" and rounds when the rank exceeds three, which is the same behaviour. The toy also makes the paper's limitation concrete: the certified optimum at 0.3 radians of noise is still 21 degrees from the true rotations on average, because the optimum of a noisy problem is not the truth.

Trainspotting: found photos as training data

What is new. Feed-forward reconstruction models, VGGT and its successors, are trained mostly on single-session captures: someone walks around an object and the object and its surroundings stay put. Hanyu Chen and Noah Snavely at Cornell built the other kind of multi-view data from Wikimedia Commons: photos of the same object taken by different people over years, in different places, sometimes of different physical copies of one design. Trainspotting is 12,068 objects and 311,886 photos, each reconstructed with a pipeline of SAM 3 gating, RoMa v2 matching, COLMAP and CasDiffMVS depth, and sorted into three regimes: fixed objects (a monument), roaming objects (one locomotive that moves between sites) and instance-pooled designs (many copies of a car model). Fine-tuning VGGT-Ω on it for 7,500 steps, about 12 hours on four B200s, gives Spotter (reported).

expand

What is released. Nearly everything. The repository holds the inference code, the reconstruction pipeline and the evaluation protocol; the checkpoint is on Hugging Face; the dataset is on Hugging Face under CC BY-SA 4.0. The model code and weights inherit VGGT-Ω's FAIR Noncommercial Research License, while the evaluation and reconstruction code is MIT (read in the repository's README). I read the checkpoint's safetensors header with two HTTP range requests: 1,437 tensors, 1,147,607,641 parameters, all float32, of which 908,139,552 are the aggregator, 203,589,641 the camera head and 3,547,664 the object-mask head (measured). That matches the paper's 1.14B.

What is checkable. The authors ship per-sample results for every model in their main table: 1,635 rows each, one 4-view and, where an object has enough photos, one 16-view sample from each of 1,000 held-out objects. That makes the headline table reproducible without a GPU. Their tables.py was refused by this sandbox's permission layer as downloaded code, so I wrote my own averaging over the same CSVs. It reproduces the README table to every printed digit: Spotter's pose AUC@30 is 81.2 against 25.0 for VGGT-Ω and 46.4 for π3, and AUC@5 is 44.2 against 7.2 (measured). The 1,000 evaluation objects are all marked eval in the split file, and none of the 11,068 training objects is among them (measured). Per sample, Spotter beats VGGT-Ω on AUC@5 in 93.1% of samples and ties in 3.9%; a bootstrap over objects puts the mean AUC@5 gain at 37.0 points, 95% interval 35.6 to 38.4 (measured).

Split by regime, the gain is not confined to the hard cases. Spotter's AUC@5 is 44.0 on fixed objects, 41.7 on roaming and 45.9 on instance-pooled, close to flat, while VGGT-Ω collapses from 20.4 to 2.9 and 2.1 (measured). The starting checkpoint is also the weakest of the four on pose in every regime; on fixed objects plain π3 has 26.9 to its 20.4 (measured), which says something about VGGT-Ω on photo collections before any fine-tuning.

Two caveats the paper states itself. The held-out ground truth is pseudo-ground truth from the same pipeline whose output Spotter was trained on, so part of the gain may be learning the pipeline's conventions. The independent check is NAVI in-the-wild, where AUC@30 rises from 18.8 to 76.0, but that is 35 samples (reported). And on conventional benchmarks the post's "it can also improve performance for normal workloads" is stronger than the paper's "largely unchanged". The paper bounds the change at 1.1 points of AUC@30 and 3.2 points of δ under 1.03. It does not mention AUC@5, which in its own appendix table drops from 91.1 to 86.1 on DTU, from 40.8 to 38.8 on 7-Scenes and from 66.9 to 65.4 on ETH3D (reasoned, from the reported table). Geometry accuracy improves slightly on 7-Scenes and Neural RGB-D. A wash on ordinary scenes, then, with a small cost in fine pose.

VoxLution: generation, not reconstruction from a scan

What is new. The post that circulated this one described it as generating detailed 3D spaces from noisy scans. It does not take a scan as input. VoxLution, from Angela Dai's group at TUM, is an unconditional (or layout-conditioned) generator of indoor scenes, trained on 3D-FRONT and on the real scans of ScanNet++, and the noise in its real-data results is noise it learned from those scans: its limitations section says the model "inherits the holes and noise of the training scans" (reported). What it adds is resolution. A coarse transformer lays out the room at 25 cm voxels; cascade transformers refine it chunk by chunk, each stage halving the voxel, down to 1.5625 cm, and a sparse VAE turns the occupancy into a mesh.

The tokenizer is the clever part, and it needs no training: each occupied voxel is described by which of its eight children are occupied, eight bits, one of 256 codes. The same 256-entry codebook means the same thing at every level, so all four cascade stages share it, and only occupied voxels produce tokens, so the sequence grows with the surface, not the volume (reported). Each chunk's sequence carries the tokens of coarser levels and the boundary of neighbours already generated, which is how adjacent chunks line up.

expand

What the numbers say. On 3D-FRONT FPD falls from LT3SD's 27.31 to 11.03 and FID from 4.91 to 2.40; on ScanNet++ from 34.33 to 10.14 and 7.50 to 1.96 (reported). The abstract's percentages follow from those: 59.6% and 51.1% on 3D-FRONT, 70.5% on ScanNet++ FPD, and 73.9% on FID where the paper prints 73.8 (reasoned). On ScanNet++ only two baselines are compared, both fine-tuned by the authors. Speed is the cost: on one H100 a room takes 22 minutes, a 4 m square takes 16 against 8 for LT3SD and 6 for WorldGrow, and 3.5 at the coarser 3.125 cm (reported). Training takes days per transformer on four H100s.

What is released. The paper and a project page; the Code button reads "coming soon". Nothing to run.

LucidLyra: understanding at the cost of reconstruction alone

What is new. NVIDIA's Lyra already turns one image into a 3D Gaussian scene by decoding the latents of a camera-controlled video model. LucidLyra gives each Gaussian two more things: a semantic feature aligned with language and an instance embedding, so the scene can be queried ("select the chair") and split into objects. The supervision comes from frozen 2D teachers run on the video model's own generated frames, SAM and SAM 2 for object tracks and a projected SigLIP 2 for semantics, and the decoder learns all of it in one feed-forward pass, without captured multi-view data or 3D labels (reported).

expand

What the headline is measured against. The abstract-level claim is a 31.7-point mIoU gain over "the strongest comparable baseline" on the authors' LucidLyra-SDG benchmark: 64.1 against 32.4 for Lyra with LangSplatV2 fitted per scene (reported). The row that matters is the one the paper calls a reference: "Lyra + Lifted Teachers", which assigns the teachers' features directly to Lyra's Gaussians. It scores 60.5 mIoU and 22.8 instance mTrackPR on SDG, against LucidLyra's 64.1 and 21.5; on the ADE20K-conditioned set 50.1 against 49.8 (reported). So LucidLyra roughly matches what you get by lifting the teachers into the scene, which is what distillation should give. The gain is time: teacher lifting takes 64 minutes a scene on an A100 and LucidLyra 2.1 (reported), about thirty times faster (reasoned). Rendering does not suffer, 25.02 dB against Lyra's 24.49. Each benchmark is 25 curated scenes, with reference labels propagated and then reviewed by people.

What is released. Paper and project page; "code coming soon", and "all data and benchmarks will be released" (reported). Nothing to check beyond the tables.

FPV-SLAM: a leaderboard row and a blog post

What is new. FPV Labs, a startup building egocentric data for robot learning, describes a sliding-window visual-inertial odometry adapted for head-mounted video, calibration estimated per recording from the footage itself, and a keyframe visual-inertial bundle adjustment with loop closure on top. It runs offline, about 10 frames per second on a 16-core cloud machine with refinement (reported). There is no paper and no code; the system is "open for early access".

What is checkable. LaMAria, ETH Zurich's egocentric SLAM benchmark, keeps its control points hidden and publishes a leaderboard, so the ranking claims can be read off it. Everything in the post's results section matches the board as of 10 November: 61.4 on Short against Mighty Camera's 57.1, a 12.2-point lead in low light and 10.1 on long walks over the next mono-inertial entry, long-walk recall within 5 m of 86.9% against 76.9% (measured). The "4.5x ORB-SLAM3" is the mean of the five category scores, 45.2 against 10.0 (measured from the board). Two things the post does not say. Its lead is narrow on moving platforms, 25.8 against 23.5, and on medium walks, 44.7 against 40.5. And its binocular entry, described as third, is now fourth by Short score, behind Aria's own SLAM, SLAM Dunk and GroundedSLAM, and behind RedComet and FlexVIO on medium, long and low-light walks (measured).

The post is also unusually frank about the benchmark. LaMAria aligns each walk with one similarity transform before scoring horizontal error at sparse control points, which forgives a uniform scale error; a robot needs metric pose in SE(3) straight out of the tracker, on short clips with hands in the way, which city walks do not test (reported). That is the right objection, and it applies to their own number too.

What to take from the week

Only one of the five can be rerun end to end by a reader, and it is the one whose contribution is a dataset. Trainspotting's checkable claim held: the table recomputes from the released rows, and the gain is spread across regimes rather than concentrated where the baseline fails. The caveats sit where the authors put them, on pseudo-ground truth and on 35 independent samples, plus one they did not, the AUC@5 cost on conventional data. XM4D's claim of certified optimality is the most interesting idea of the week and the easiest to misread: it certifies the solve, not the depth and tracks fed into it, and on three of its seven camera benchmarks MegaSAM still has lower error. VoxLution and LucidLyra are paper-only for now, and their numbers are consistent with themselves once you read which row they are compared against. FPV-SLAM is a product announcement whose one checkable claim checks out.