Generating Shapes
Modeling by Example #
-
User in the loop
-
Idea:
- From an initial model:
- Select a part of the model to edit
- search 3D database for similar parts
- Composite selected parts into model
- Repeat
-
Requirement:
- Already have database of shapes with part segmentation annotations
Figure 1: User selecting a new left arm for the statue by placing placeholder boxes where the arm should be. A shape that fits the constraints is then looked up from the database and the relevant parts are copied over
Part Suggestion #
Based on desired attributes #
Object reconstruction #
- From 3D scan
- From RGB image
- From Depth image
-> Estimate geometric structure -> Estimate possible navigation -> Estimate possible interactions
Shape reconstruction #
| From distance field | |
|---|---|
| 3D-EPN for Shape Completion | distance field (voxels) -> distance field (voxels) |
| From RGB image Multi-View: | |
| 3D-R2N2 | images -> occupancy voxel |
| Implicit representations | |
| DeepSDF | output signed distance value for any point |
| Occupancy Networks | output occupancy probability for any point |
| Point clouds | |
| PSGN | image + segmentation -> points |
| Parametric models | |
| Deep Sketch-Based Modeling of Man-Made Shapes | 2D sketches -> 3D parametric shapes |
| Template based 3D Meshes | |
| Pixel2Mesh | image -> transfomed ellipsoid mesh |
| Mesh R-CNN | image -> predict template -> deform template |
| Direct Mesh generation | |
| Scan2Mesh | 3D scan (TSDF ) -> Indexed Face Set |
| Retrieval-based object reconstruction | |
| Database-assisted object retrieval for real-time 3d reconstruction | 3d Scan -> Object retrieval |
| Joint Embeddings of Shapes and Images via CNN Image Purification | Images / Shapes -> common latent space |
| Joint embedding of 3d scan and cad objects | 3d scans / cad objects -> common latent space |
| Mask2cad |