Joint Embedding of 3d Scan and CAD Objects

@inproceedings{dahnert2019joint,
  title={Joint embedding of 3d scan and cad objects},
  author={Dahnert, Manuel and Dai, Angela and Guibas, Leonidas J and Nie{\ss}ner, Matthias},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={8749--8758},
  year={2019}
}

• Similar to Joint Embeddings of Shapes and Images via CNN Image Purification
  • but 3D scans instead of images
  • but this one is End to End

Architecture #

Blue part

Segmenting foreground and background (supervised)

Green part

Encoder+Decoder -> try to predict complete shape

Yellow part

Training of the embedding space

• The reconstructed mesh should match in emedding space with the CAD model in embedding space
• At the same time it should be far away from a unrlated cad model

Loss #

Loss

Triplet loss

To minimize Loss: -> Distance to positive should be small -> Distance to negative should be big