The process of modeling a 3D object from a series of 2D images is a very well-studied computer vision problem called multiview stereo reconstruction.