The 3DGS Training Pipeline — A Native Blender Reference Map

A detailed, end-to-end reference map of the 3D Gaussian Splatting training pipeline — annotated for what could run natively inside Blender’s geometry nodes versus what stays an irreducible custom kernel.

One of the open questions behind SkySplat is how much of the full reconstruction stack — Structure-from-Motion (SfM) all the way through 3DGS optimization — could live natively inside Blender. Blender’s geometry nodes already give us loop orchestration, point-cloud representation, and closed-form vector math for free. The interesting question is where that stops: which stages are genuinely irreducible kernels (feature matching, the robust geometric solvers, the differentiable rasterizer) that have to be written as a custom node, and which are just connective tissue that stock nodes already cover.

This reference lays out the entire pipeline as a single annotated build map. Each stage is tinted by whether it falls out of stock geometry nodes (native) or is an irreducible custom kernel (the fork), and every stage carries a precise statement of its fundamental operation, the shape of the data flowing in and out, and where the loops are.

What the reference covers

• Stage 00 — Structure-from-Motion as the upstream preprocess: feature extraction, descriptor matching, robust geometric solvers, and bundle adjustment, and why in practice you call COLMAP / GLOMAP (or feed-forward neural methods like DUSt3R / MASt3R, VGGSfM) rather than building it from nodes.
• A bird’s-eye schematic of the trainer — setup vs. the per-iteration loop.
• The setup stages (S0–S1) that run once.
• The training loop (×K ≈ 30k iterations) — projection, differentiable rasterization, loss, backprop, and the clone / split / prune densification logic.
• A data-shape ledger tracking what shape the data is in at every hop.
• The three loop scopes that structure the whole optimization.

This is intended as a working reference for anyone thinking about a native SfM + 3DGS implementation in Blender — a map of the terrain before building it.