First Advisor

Christof Teuscher

Term of Graduation

Winter 2022

Date of Publication

2-2-2022

Document Type

Thesis

Degree Name

Master of Science (M.S.) in Computer Science

Department

Computer Science

Language

English

Subjects

Audio-visual materials -- Evaluation, Time-series analysis, Teaching -- Aids and devices, Zoom (Electronic resource), Distance education

DOI

10.15760/etd.7787

Physical Description

1 online resource (xi, 122 pages)

Abstract

The recent shift towards remote education has presented new challenges for instructors with respect to teaching evaluation. Students in traditional classrooms send signals to instructors which provide feedback for the effectiveness of a given lecture. Virtual learning environments lack some of these communication channels and require new ways of collecting feedback. This work presents a suite of analysis tools for the virtual instructor. Given the transcript and video files for a Zoom meeting, this tool summarizes student sentiment and speaking characteristics. Sentiment scores are derived using state of the art Natural Language Processing (NLP) models. The video file is used to extract interesting features about the lecture content, such as the number of slides, pace of slide changes, and number of words per slide. All metrics were experimentally tested with data from four Zoom meetings, each of which included ground-truth annotations for the slide changes. Transcript-based metrics were validated by comparing to the output produced by Meeting Measures, the project this tool is based on.

A time series outlier detection model was developed for the purpose of identifying slide changes during a presentation. Initially, a percentile-based model performed well on the annotated videos when the optimal threshold was known. However, the process of finding this threshold turned out to be non-trivial for videos without annotations. A Kalman filter model was then tested to alleviate the need for an optimization step. Ultimately, the percentile-based model was replaced by an HMM-based (Hidden Markov Model) model because of its ability to generalize. When tested on annotated videos, the HMM-based detector performed within a reasonable tolerance of the optimal percentile-based model. Furthermore, for each slide detected an open source Optical Character Recognition (OCR) framework was used to extract text content for computing word counts.

This tool outputs a dashboard containing a set of visualizations for the instructor. These are intended to both identify pain points in the lecture as well as provide a bird's-eye view of general class interaction characteristics. The broader impact goal of this work is to increase remote teaching effectiveness by helping instructors optimize education delivery throughout the academic year.

Rights

In Copyright. URI: http://rightsstatements.org/vocab/InC/1.0/ This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).

Persistent Identifier

https://archives.pdx.edu/ds/psu/37351

Download

Share

COinS