Sponsor
Portland State University. Department of Computer Science
First Advisor
Banfsheh Rekabdar
Term of Graduation
Spring 2023
Date of Publication
6-13-2023
Document Type
Thesis
Degree Name
Master of Science (M.S.) in Computer Science
Department
Computer Science
Language
English
Subjects
Deep Hierarchical Variational Autoencoder, Likelihood-based Generative Models, Model-Based RL, World Models
DOI
10.15760/etd.3607
Physical Description
1 online resource (xii, 48 pages)
Abstract
Model-based reinforcement learning (MBRL) approaches leverage learned models of the environment to plan and make optimal decisions, reducing the need for extensive real-world interactions and enabling more efficient learning in complex domains such as robotics, autonomous systems, and resource allocation problems. They also provide interpretability and insight into the underlying dynamics, facilitating better decision-making and system understanding.
The world model is a model-based RL approach that employs generative neural network models to learn a compressed spatial and temporal representation of the environment. This work explores world models and a simple single-layered RNN model to learn a simple policy based on the representations to solve tasks in complex RL environments. A traditional variational autoencoder (VAE) encodes environment features to latent representations in the world model approach. Recent research on generative models reveals that traditional VAE constraints cause information loss or distortion during compression and impede the world model based- agent's ability to learn accurate representations of complex environments. This thesis proposes a deep hierarchical variational autoencoder (NVAE) as the visual component of the world model to overcome the challenge of modeling complex data and long-range correlations and improve an agent's performance in complex RL environments such as car racing-v2 and panda-gym.
Rights
©2023 Sriharshitha Ayyalasomayajula
In Copyright. URI: http://rightsstatements.org/vocab/InC/1.0/
This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).
Persistent Identifier
https://archives.pdx.edu/ds/psu/40839
Recommended Citation
Ayyalasomayajula, Sriharshitha, "A Deep Hierarchical Variational Autoencoder for World Models in Complex Reinforcement Learning Environments" (2023). Dissertations and Theses. Paper 6485.
https://doi.org/10.15760/etd.3607