First Advisor

Feng Liu

Term of Graduation

Spring 2020

Date of Publication

8-7-2020

Document Type

Dissertation

Degree Name

Doctor of Philosophy (Ph.D.) in Computer Science

Department

Computer Science

Language

English

Subjects

Computer vision, Computational photography, Image processing -- Digital techniques

DOI

10.15760/etd.7409

Physical Description

1 online resource (xi, 89 pages)

Abstract

Novel view synthesis is an important research problem in computer vision and computational photography. It enables a wide range of applications including re-cinematography, video enhancement, virtual reality, etc. These algorithms leverage a pre-acquired set of images taken from a set of viewpoints to synthesize another image at a novel viewpoint as if it was captured by a real camera. To synthesize a high-quality novel view, these algorithms often assume a static scene, or the images were captured synchronously. However, the scenes in practice are often dynamic, and taking a dense set of images of these scenes at the same moment is challenging because it often requires a complicated setup of dedicated equipment. It will be more useful to have a novel view synthesis algorithm that can handle the cases when the number of input images is small and they are captured asynchronously from distant viewpoints.

However, developing such technologies faces several challenging technical problems. First, it is difficult to obtain a high-quality 3D scene structure using only a small set of input images. Secondly, images captured from sparse viewpoints often have small overlapping areas but with significant occlusion and parallax, which is difficult to resolve. Thirdly, asynchronously captured images often have dynamic contents such as local object motion across the images. This often leads to severe occlusion and pixel inconsistency. As a result, it is challenging to align input views together to synthesize an image at a novel view using those input images. Besides, a novel view synthesis algorithm often requires a substantial analysis of the input images, such as an optical flow that captures pixel correspondences between them, which strongly affects the efficiency of the algorithm and may limit its practical applications.

This thesis presents a set of technical contributions to address the problem of novel view synthesis for input images captured asynchronously from sparse viewpoints. First, to address the challenge of sparse input viewpoints and missing high-quality dense 3D scene structure, this thesis introduces an appearance flow completion algorithm for novel view synthesis. This algorithm directly incorporates a sparse 3D scene structure and uses it to guide the estimation of appearance flows between input and target views. These appearance flows are then used to warp and blend the input views to synthesize a target view. Secondly, this thesis introduces a data-driven homography estimation method to address the problem of object motion in the scene, which is common with asynchronously captured images. Finally, this thesis presents a learned upsampling method for fast optical flow estimation, which enhances the efficiency of novel view synthesis methods.

Rights

In Copyright. URI: http://rightsstatements.org/vocab/InC/1.0/ This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).

Persistent Identifier

https://archives.pdx.edu/ds/psu/33643

Download

Share

COinS