Multi-view neural surface reconstruction has exhibited impressive results. However, a notable limitation is the prohibitively slow inference time when compared to traditional techniques, primarily attributed to the dense sampling, required to maintain the rendering quality.
We present TSDF-Sampling — a novel approach that substantially reduces the number of samplings by incorporating the Truncated Signed Distance Field (TSDF) of the scene. While prior works have proposed importance sampling, their dependence on initial uniform samples over the entire space makes them unable to avoid performance degradation when trying to use less number of samples. In contrast, our method leverages the TSDF volume generated only by the trained views, and it proves to provide a reasonable bound on the sampling from upcoming novel views. As a result, we achieve high rendering quality by fully exploiting the continuous neural SDF estimation within the bounds given by the TSDF volume.
Notably, our method is the first approach that can be robustly plug-and-play into a diverse array of neural surface field models, as long as they use the volume rendering technique. Our empirical results show an 11-fold increase in inference speed without compromising performance.
TSDF-Sampling integrates TSDF volume generated only by the train views, and the TSDF volume can provide sampling bound prior for upcoming novel views. Note that the model agnostic TSDF-Sampling can use any existing neural surface fields as a baseline. Below visualizes the coarse samples, and fine samples will be upsampled from the coarse samples.
(a) Hierarchical Sampling
(b) TSDF Sampling
In the synthetic Garage dataset, TSDF-Sampling enables high-fidelty renderings of color, depth, and normal, with as few as 4 samples per ray.
In the widely used Replica dataset, the model-agnostic TSDF-Sampling can render more complete and quality results with a limited number of neural network queries.
The PSNR drastically increases from 23 to 28, and the depth error decreases by more than 50%, when using the TSDF Sampling approach.
@article{
title = {Coming Soon},
journal = {},
year = {},
}