Portland State University. Department of Computer Science
Date of Award
Master of Science (M.S.) in Computer Science
1 online resource (ix, 54 pages)
Electronic data processing -- Distributed processing, Machine learning, Cloud computing, Computer science
There is a rise in demand among machine learning researchers for powerful computational resources to train complex machine learning models, e.g., deep learning models. In order to train these models in a reasonable amount of time, the training is often distributed among multiple machines; yet paying for such machines (either through renting them on cloud data centers or building a local infrastructure) is costly. DeepMarket attempts to reduce these costs by creating a marketplace that integrates multiple computational resources over a distributed TensorFlow framework. Instead of requiring users to rent expensive GPU/CPUs from a third-party cloud provider, DeepMarket allows users to lend their edge computing resources to each other when they are available. Such a marketplace, however, requires a credit mechanism that ensures users receive resources in proportion to the resources they lend to others. Moreover, DeepMarket must respect users' needs to use their own resources and the resulting limits on when resources can be lent to others. In this thesis, I present the design and implementation of DeepMarket, an architecture that addresses these challenges and allows users to securely lend and borrow computing resources. I also present preliminary experimental evaluation results that show DeepMarket's performance, in terms of job completion time, is comparable to third-party cloud providers. However, DeepMarket can achieve this performance with reduced cost and increased data privacy.
Kim, Soyoung, "Design and Experimental Evaluation of DeepMarket: an Edge Computing Marketplace with Distributed TensorFlow Execution Capability" (2019). Dissertations and Theses. Paper 5120.