First Advisor

Feng Liu

Term of Graduation

Winter 2025

Date of Publication

3-13-2025

Document Type

Dissertation

Degree Name

Doctor of Philosophy (Ph.D.) in Computer Science

Department

Computer Science

Language

English

Subjects

High resolution imaging, Computer vision, Neural networks (Computer science)

DOI

10.15760/etd.3903

Physical Description

1 online resource (xi, 94 pages)

Abstract

Efficient visual synthesis has played an indirect yet important role in the development of neural networks. Graphics processing units (GPUs) were initially designed for visual synthesis tasks such as rendering graphics. However, their parallel computing capabilities later made GPUs the ideal hardware for training neural networks. Since neural networks have greatly benefited from GPUs, a compelling question arises: How can neural networks, in turn, improve efficient visual synthesis?

Existing visual synthesis methods still require improvement for higher efficiency. For example, Monte Carlo rendering takes over 15 minutes to generate a megapixel frame at a high-quality setting on a GPU, as computing thousands of samples per pixel is time-consuming yet essential for an offline path tracer. This thesis presents three technical contributions based on neural networks for efficient visual synthesis. We explore the performance improvement that neural networks can bring to different applications of visual synthesis, including graphics, 4D Gaussians, and videos.

First, we propose speeding up Monte Carlo rendering by significantly reducing the number of pixels required for sampling. We introduce a neural network to predict future frames instead of rendering future frames. Second, we introduce a feature splatting method based on 4D Gaussians, named Spacetime Gaussians. It is composed of three important components for dynamic view synthesis in a volumetric rendering way. The proposed Spacetime Gaussian is formulated by adapting 3D Gaussians with temporal opacity and parametric motion/rotation. Last, we apply neural visual synthesis to the general video prediction task. To produce high-quality results, we explore a trending architecture, Vision Transformers (ViTs).

Rights

In Copyright. URI: http://rightsstatements.org/vocab/InC/1.0/ This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).

Persistent Identifier

https://archives.pdx.edu/ds/psu/43150

Additional Files
chapt-3.mp4 (191630 kB)
Supplementary material for Chapter 3
chapt-4.pdf (13115 kB)
Supplementary material for Chapter 4
chapt-5.mp4 (77772 kB)
Supplementary material for Chapter 5
Download

Share

COinS