OctoSplat: Hybrid OctoMap-Gaussian Splatting for Active Semantic Mapping and Phenotyping with Horticultural Robots

Jose Cuaran, Naveen K. Uppalapati, Girish Chowdhary
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign
📄 Paper 💻 Code

Abstract

Semantic reconstruction of agricultural scenes plays a vital role in tasks such as phenotyping and yield estimation. However, traditional approaches that rely on manual scanning or fixed camera setups remain a major bottleneck in this process, and existing active-mapping methods based solely on occupancy grids are too coarse to support accurate trait estimation. To close this gap, we propose an active 3D reconstruction framework for horticultural environments using a mobile manipulator. The proposed system integrates the classical OctoMap representation with 3D Gaussian Splatting to enable accurate and efficient target-aware mapping. While a low-resolution OctoMap provides probabilistic occupancy information for informative viewpoint selection and collision-free planning, 3D Gaussian Splatting leverages geometric, photometric, and semantic information to optimize a set of 3D Gaussians for high-fidelity scene reconstruction. We further introduce a robust mapping strategy that mitigates the effects of semantic segmentation and depth noise, together with a background pruning method that reduces memory consumption and computational cost. We validate our framework across simulated, laboratory, and real greenhouse scenes, and show that the improvements hold consistently across three state-of-the-art Gaussian Splatting backbones. In simulation, where ground-truth geometry is available, our approach outperforms occupancy-based mapping in both reconstruction accuracy and runtime efficiency: compared with a 0.01m-resolution OctoMap, our method doubles the fruit-level F1 score under noisy conditions while achieving up to a threefold reduction in runtime. Beyond simulation, novel-view synthesis quality also improves consistently in the laboratory and real greenhouse environments, with PSNR and mIoU improving by up to 1.5 dB and 18% respectively. Finally, the reconstructed semantic maps enable fruit counting and volume estimation with accuracies approaching 80%.

Method

OctoSplat system overview diagram
System overview. Our framework integrates two semantic representations: a low-resolution OctoMap for collision-free motion planning, frontier extraction, and viewpoint evaluation, and a 3DGS representation for high-fidelity reconstruction. Candidate exploitation viewpoints are sampled around semantic clusters, while exploration viewpoints are sampled around frontier centroids and along crop rows. A graph-based planner determines the optimal sequence of viewpoints by considering information gain and actuation cost.

Results

Simulation, laboratory, and real greenhouse environments, along with our simulated and physical mobile manipulator
Simulation, laboratory, and real greenhouse environments, along with our simulated and physical mobile manipulator.
Comparison of OctoSplat reconstructions against Semantic OctoMap under noiseless, segmentation noise, and segmentation plus depth noise conditions
Reconstruction quality vs. Semantic OctoMap. Under noiseless conditions and under segmentation and depth noise, our approach (left) preserves fine-grained plant and fruit geometry, while the 0.01 m-resolution Semantic OctoMap baseline (right) degrades substantially as noise increases.

Videos

Active Mapping in Laboratory 2x speed
Active mapping in real greenhouse 2x speed
Gaussian Reconstruction in simulation
Gaussian Reconstruction in laboratory environment
Gaussian Reconstruction in a real greenhouse — Scene 1
Gaussian Reconstruction in a real greenhouse — Scene 0