Occlusion-aware Intermediate View Reconstruction

Intermediate view reconstruction has many applications, especially in the area of 3D displays and communication. One of the most important applications is that it can be used to reconstruct additional views from stereo pairs, so that any stereo pair can be displayed on emerging automultiscopic displays that require many views of a scene as input. It is in this context that we demonstrate applications of the proposed methods in different areas. First, in a biomedical application, the proposed view reconstruction algorithm is embedded into a neuromuscular training system that utilizes automultiscopic 3D displays. Second, the proposed view reconstruction method is applied to monoscopic video sequences to increase frame rate and, therefore, enhance low frame-rate videos captured by mobile phones. Third, the proposed occlusion-aware optical flow method is used to solve a real-world problem of NASA, namely the recovery of a missing color component in stereo images. Finally, disparity estimation in a multiview video codec that uses view reconstruction is shown to benefit from our methods.