First Advisor

Karen L. Karavanic

Term of Graduation

Spring 2020

Date of Publication

3-27-2020

Document Type

Thesis

Degree Name

Master of Science (M.S.) in Computer Science

Department

Computer Science

Language

English

Subjects

High performance computing, Critical path analysis, Cloud computing

DOI

10.15760/etd.7369

Physical Description

1 online resource (vii, 90 pages)

Abstract

Optimizing scientific application performance in HPC environments is a complicated task which has motivated the development of many performance analysis tools over the past decades. These tools were designed to analyze the performance of a single parallel code using common approaches such as message passing (MPI), multithreading (OpenMP), acceleration (CUDA), or a hybrid approach. However, current trends in HPC such as the push to exascale, convergence with Big Data, and growing complexity of HPC applications and scientific workflows, have created gaps that these performance tools do not cover, particularly involving end-to-end data movement through an end-to-end HPC workflow comprising multiple codes, paradigms, or platforms.

To address this performance monitoring gap, we define a new metric called Workflow Critical Path (WCP), a data-oriented critical path metric for Holistic HPC Workflows. Using cloud-based technologies, we implement a prototype called Crux, a distributed analysis tool for calculating and visualizing WCP. Crux takes a novel, data-oriented approach by constructing program activity graphs (PAGs) using data states as vertices and data mutations as edges. Our experiments with a workflow simulator on Amazon Web Services show Crux is scalable and capable of calculating WCP for common Holistic HCP workflow patterns. We discuss how Crux and WCP could be used with production HPC applications.

Rights

In Copyright. URI: http://rightsstatements.org/vocab/InC/1.0/ This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).

Persistent Identifier

https://archives.pdx.edu/ds/psu/33448

Download

Share

COinS