TL;DR: Our approach generates a mesh using a quadrature field. We associate texture with the mesh and render the mesh using ray tracing, which can produce volumetric effects such as fur.
We propose a novel Neural Radiance Field (NeRF) representation for non-opaque scenes that enables fast inference by utilizing textured polygons. Despite the high-quality novel view rendering that NeRF provides, a critical limitation is that it relies on volume rendering that can be computationally expensive and does not utilize the advancements in modern graphics hardware. Many existing methods fall short when it comes to modelling volumetric effects as they rely purely on surface rendering. We thus propose to model the scene with polygons, which can then be used to obtain the quadrature points required to model volumetric effects, and also their opacity and colour from the texture. To obtain such polygonal mesh, we train a specialized field whose zero-crossings would correspond to the quadrature points when volume rendering, and perform marching cubes on this field. We then perform ray-tracing and utilize the ray-tracing shader to obtain the final colour image. Our method allows an easy integration with existing graphics frameworks allowing rendering speed of over 100 frames-per-second for a 1920×1080 image, while still being able to represent non-opaque objects.
The core of our approach consists of learning a quadrature field \(\sin (\omega \mathcal{F}(x)\), the zero-crossings of which gives us quadrature points deterministically. We train this field such that the field gives more quadrature points in regions where volumetric weights are higher. The relative number of zero crossings can be controlled by the frequency term omega as shown above.
Visualization of reconstructed nerf-synthetic dataset.
Visualization of reconstructed nerf-synthetic dataset.
Our approach can represent transparency using continuous opacity.
Our approach can represent hair like material better than the baselines.
Our approach can represent transparency better than surface based baking approaches like MobileNeRF.
@article{quadfields,
author = {Gopal Sharma, Daniel Rebain, Andrea Tagliasacchi, Kwang Moo Yi},
title = {Volumetric Rendering with Baked Quadrature Fields},
journal = {ECCV},
year = {2024},
}