FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularization

@article{yang2022freenerf,
  author = {Jiawei Yang and Marco Pavone and Yue Wang},
  title = {FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularization},
  joural = {Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR)},
  year = {2023},
}

Publication Youtube Video

• Improvement over NeRF in the context of few-shot (only few images to train).
• 2 main contributions:

  1. Frequency regularization:

     • Mask off high frequency components of the position encoding (introduced by NeRF) for early training, and over time include more high frequency components. In original NeRF, the whole frequency spectrum is used for the entirety of the training.

       

     • This allows the model to first learn the low frequency information in color and shape and then iteratively train the high frequency information, which leads to an overall faster convergence. Which is needed in the few-shot context

  2. Occlusion regularization

training #

takes same time as regular nerf

• MipNeRF
• DietNeRF
• RegNeRF