First Advisor

Banfsheh Rekabdar

Term of Graduation

Spring 2023

Date of Publication


Document Type


Degree Name

Master of Science (M.S.) in Computer Science


Computer Science




Deep Hierarchical Variational Autoencoder, Likelihood-based Generative Models, Model-Based RL, World Models



Physical Description

1 online resource (xii, 48 pages)


Model-based reinforcement learning (MBRL) approaches leverage learned models of the environment to plan and make optimal decisions, reducing the need for extensive real-world interactions and enabling more efficient learning in complex domains such as robotics, autonomous systems, and resource allocation problems. They also provide interpretability and insight into the underlying dynamics, facilitating better decision-making and system understanding.

The world model is a model-based RL approach that employs generative neural network models to learn a compressed spatial and temporal representation of the environment. This work explores world models and a simple single-layered RNN model to learn a simple policy based on the representations to solve tasks in complex RL environments. A traditional variational autoencoder (VAE) encodes environment features to latent representations in the world model approach. Recent research on generative models reveals that traditional VAE constraints cause information loss or distortion during compression and impede the world model based- agent's ability to learn accurate representations of complex environments. This thesis proposes a deep hierarchical variational autoencoder (NVAE) as the visual component of the world model to overcome the challenge of modeling complex data and long-range correlations and improve an agent's performance in complex RL environments such as car racing-v2 and panda-gym.


©2023 Sriharshitha Ayyalasomayajula

In Copyright. URI:

This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).

Persistent Identifier