Auto-Riggable Gaussian Characters
Credits: Evan Zhang, Sean Brynjólfsson, Justin Tien-Smith
Recently, techniques for solving gaussian splats of dynamic scenes (Dynamic3DGaussians, 2024) have found success in using local rigidity constraints to enforce spatial and temporal consistency.
We use this detailed representation and decompose it into the rigid parts and joints which describe their movement. This procedure makes no assumptions about the anatomy of the dynamic entities within the scene and therefore should work equally well for all people, animals, or machines—anything that moves about a discrete set of joints. We're currently implementing the joint solver after obtaining promising results for our clustering algorithm to find the bones.
Our final deliverable will be an animation-ready gaussian rig and a portable format for it. Clustering also significantly downsizes the storage requirements because local rigidity means gaussians are predictive of their neighbors—no need to track all of them. We are also developing more visualizations to better understand the limitations of the representation. In doing so, we have observed some new failure modes of the original method, such as how some regions gradually creep into neighboring areas over time.