Media.Player

Inti Multiview - Real-Time Stereo Reconstruction Integration For 3D Teleimmersion

Technology Benefits

45 frames per second1 cm accuracy for 1-2m distancesruns on any newer desktop computerforeground/background segmentationmarkerless 3D reconstructionscalable matching & global optimization

Technology Application

computer games & 3D video conferencingsurveillance and trackingautonomous vehicle controlremote dance choreographygeoscientific and archeological

Detailed Technology Description

None

Others

Tech ID/UC Case

21121/2011-030-0

Related Cases

2011-030-0

*Abstract

While teleimmersion has great potential, available algorithms for real-time stereo reconstruction require several seconds to several minutes to process two images and produce accurate stereo output.  Available FPGA and GPU implementations have inherent drawbacks in ability to reconstruct homogenous regions or regions with repeating patterns. The time-of-flight cameras have low resolution, limited range, high noise, and albedo sensitivity. Therefore,  a practical real-time stereo reconstruction is needed for a system enabling geographically distributed users to interact with each other in a shared virtual space.  

University of California investigators have responded to this challenge by developing INTI Multiview, a real-time stereo reconstitution integration for 3D telleimmersion.  With INTI Multiview, each user is presented by their 3D avatar generated in real time. INTI Multiview focuses primarily on integrating multiple stereo reconstructions from different views.  Levels of accuracy comparable to other methods are achieved at a much faster speed on CPU by taking a hybrid approach: performing a local optimization technique (the region matching) and using a global optimization approximation to improve the initial results (anisotropic diffusion).  The investigators have implemented a novel multiscale representation that allows for the highly accurate reconstruction of a scene.  The investigators have successfully tested INTI Multiview in many application areas, such as remote dance choreography, shared geoscientific and archeological applications, and training. This work has further extensions in other applications where real-time stereo data is necessary, e.g. full body motion capture, surveillance and tracking, foreground/background segmentation, autonomous vehicle control.  Markerless 3D reconstruction for human movement analysis (motion capture, visual feedback for gaming, rehabilitation etc.)  

*Principal Investigator

Name: Gregorij Kurillo

Department:

Name: Edgar Lobaton

Department:

Name: Ramanarayan Vasudevan

Department:

Country/Region

USA