Portland State University. Department of Computer Science
Term of Graduation
Date of Publication
Master of Science (M.S.) in Computer Science
Deep Hierarchical Variational Autoencoder, Likelihood-based Generative Models, Model-Based RL, World Models
1 online resource (xii, 48 pages)
Model-based reinforcement learning (MBRL) approaches leverage learned models of the environment to plan and make optimal decisions, reducing the need for extensive real-world interactions and enabling more efficient learning in complex domains such as robotics, autonomous systems, and resource allocation problems. They also provide interpretability and insight into the underlying dynamics, facilitating better decision-making and system understanding.
The world model is a model-based RL approach that employs generative neural network models to learn a compressed spatial and temporal representation of the environment. This work explores world models and a simple single-layered RNN model to learn a simple policy based on the representations to solve tasks in complex RL environments. A traditional variational autoencoder (VAE) encodes environment features to latent representations in the world model approach. Recent research on generative models reveals that traditional VAE constraints cause information loss or distortion during compression and impede the world model based- agent's ability to learn accurate representations of complex environments. This thesis proposes a deep hierarchical variational autoencoder (NVAE) as the visual component of the world model to overcome the challenge of modeling complex data and long-range correlations and improve an agent's performance in complex RL environments such as car racing-v2 and panda-gym.
©2023 Sriharshitha Ayyalasomayajula
Ayyalasomayajula, Sriharshitha, "A Deep Hierarchical Variational Autoencoder for World Models in Complex Reinforcement Learning Environments" (2023). Dissertations and Theses. Paper 6485.