Weakly-Supervised Mesh-Convolutional
Hand Reconstruction in the Wild
CVPR 2020 (Oral)

  • Imperial College London
  • Università della Svizzera italiana

Abstract

We introduce a simple and effective network architecture for monocular 3D hand pose estimation consisting of an image encoder followed by a mesh convolutional decoder that is trained through a direct 3D hand mesh reconstruction loss. We train our network by gathering a large-scale dataset of hand action in YouTube videos and use it as a source of weak supervision. Our weakly-supervised mesh convolutions-based system largely outperforms state-of-the-art methods, even halving the errors on the in the wild benchmark.

Dataset

The dataset will be released later in 2020.

It consists of around 50,000 hand meshes aligned with frames selected from over a hundred in the wild YouTube videos.

BibTeX