ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans

@inproceedings{dai2018scancomplete,
  title={Scancomplete: Large-scale scene completion and semantic segmentation
         for 3d scans},
  author={Dai, Angela and Ritchie, Daniel and Bokeloh, Martin and Reed, Scott
         and Sturm, J{\"u}rgen and Nie{\ss}ner, Matthias},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern
             Recognition},
  pages={4578--4587},
  year={2018}
}

• Incomplete scan -> Semantic Scene Segmentation
• Based fully on convolutions

To this end, we devise a fully-convolutional generative 3D CNN model whose filter kernels are invariant to the overall scene size.

Our network uses a fully-convolutional architecture with three-dimensional filter banks. Its key property is its invariance to input spatial extent, which is particularly critical for completing large 3D scenes whose sizes can vary significantly. That is, we can train the network using random spatial crops sampled from training scenes, and then test on different spatial extents at test time.

Our network first predicts the output at a low resolution in order to leverage more global information from the input. Subsequent hierarchy levels operate at a higher resolution and smaller context size. They condition on the previous level’s output in addition to the current-level incomplete TSDF. We use three hierarchy levels, with a large context of several meters