A Community Platform for Research on Pricing and Distributed Machine Learning
Sponsor
This research was supported by NSF grants CNS-1910517, CNS-1942305, CNS-1751075 and CNS-1909306, and ARO W911NF1910036.
Published In
2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS)
Document Type
Citation
Publication Date
3-2021
Abstract
Data generated by increasingly pervasive and intelligent devices has led to an explosion in the use of machine learning (ML) and artificial intelligence, with ever more complex models trained to support applications in fields as diverse as healthcare, finance, and robotics. In order to train these models in a reasonable amount of time, the training is often distributed among multiple machines. However, paying for these machines (either by constructing a local cloud infrastructure or renting machines through an external provider such as Amazon AWS) is very costly. We propose to reduce these costs by creating a marketplace of computing resources designed to support distributed machine learning algorithms. Through our marketplace (coined “DeepMarket”), users can lend their spare computing resources (when not needed) or augment their resources with available DeepMarket machines to train their ML models. Such a marketplace directly provides several benefits for two groups of researchers: (i) ML researchers would be able to train their models with much reduced cost, and (ii) network economics researchers would be able to experiment with different compute pricing mechanisms. The focus of this Demo is to introduce the audience to DeepMarket and its user interface (named “PLUTO”). In particular, we will bring a few laptops with pre-installed PLUTO applications so that users can see how they can create an account on DeepMarket servers, lend their resource, borrow available resources, submit ML jobs, and retrieve the results. Our overall goal is to encourage the conference audience to install PLUTO on their own machines and create a user and developer community around DeepMarket.
Rights
Copyright © 2021 American Speech-Language-Hearing Association
Locate the Document
DOI
10.1109/ICDCS47774.2020.00117
Persistent Identifier
https://archives.pdx.edu/ds/psu/35007
Publisher
IEEE
Citation Details
Li, X., Gomena, S., Ballard, L., Li, J., Aryafar, E., & Joe-Wong, C. (2020). A Community Platform for Research on Pricing and Distributed Machine Learning. Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/icdcs47774.2020.00117