Portland State University. Department of Computer Science
Term of Graduation
Date of Publication
Doctor of Philosophy (Ph.D.) in Computer Science
Computer vision, Computational photography, Image processing -- Digital techniques
1 online resource (xi, 89 pages)
Novel view synthesis is an important research problem in computer vision and computational photography. It enables a wide range of applications including re-cinematography, video enhancement, virtual reality, etc. These algorithms leverage a pre-acquired set of images taken from a set of viewpoints to synthesize another image at a novel viewpoint as if it was captured by a real camera. To synthesize a high-quality novel view, these algorithms often assume a static scene, or the images were captured synchronously. However, the scenes in practice are often dynamic, and taking a dense set of images of these scenes at the same moment is challenging because it often requires a complicated setup of dedicated equipment. It will be more useful to have a novel view synthesis algorithm that can handle the cases when the number of input images is small and they are captured asynchronously from distant viewpoints.
However, developing such technologies faces several challenging technical problems. First, it is difficult to obtain a high-quality 3D scene structure using only a small set of input images. Secondly, images captured from sparse viewpoints often have small overlapping areas but with significant occlusion and parallax, which is difficult to resolve. Thirdly, asynchronously captured images often have dynamic contents such as local object motion across the images. This often leads to severe occlusion and pixel inconsistency. As a result, it is challenging to align input views together to synthesize an image at a novel view using those input images. Besides, a novel view synthesis algorithm often requires a substantial analysis of the input images, such as an optical flow that captures pixel correspondences between them, which strongly affects the efficiency of the algorithm and may limit its practical applications.
This thesis presents a set of technical contributions to address the problem of novel view synthesis for input images captured asynchronously from sparse viewpoints. First, to address the challenge of sparse input viewpoints and missing high-quality dense 3D scene structure, this thesis introduces an appearance flow completion algorithm for novel view synthesis. This algorithm directly incorporates a sparse 3D scene structure and uses it to guide the estimation of appearance flows between input and target views. These appearance flows are then used to warp and blend the input views to synthesize a target view. Secondly, this thesis introduces a data-driven homography estimation method to address the problem of object motion in the scene, which is common with asynchronously captured images. Finally, this thesis presents a learned upsampling method for fast optical flow estimation, which enhances the efficiency of novel view synthesis methods.
In Copyright. URI: http://rightsstatements.org/vocab/InC/1.0/ This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).
Le, Hoang, "Novel View Synthesis - a Neural Network Approach" (2020). Dissertations and Theses. Paper 5535.