Search · four archives
Search · four archives
8 papers · ranked by Valyu relevance
Kangwook Lee, Maximilian Lam, Ramtin Pedarsani, Dimitris Papailiopoulos + 1 more
'Dimitris Papailiopoulos' 'Kannan Ramchandran'] Codes are widely used in many engineering applications to offer robustness against noise. In large-scale systems there are several types of noise that can affect the performance of distributed machine learning algorithms – straggler nodes, system failures, or…
Songze Li, Mohammad Ali Maddah-Ali, A. Salman Avestimehr
More specifically, a general distributed computing framework, motivated by commonly used structures like MapReduce, is considered, where the overall computation is decomposed into computing a set of "Map" and "Reduce" functions distributedly across multiple computing nodes. A coded scheme, named "Coded Distributed…
Kai Wan, Mingyue Ji, Giuseppe Caire
—This paper considers the MapReduce-like coded distributed computing framework originally proposed by Li et al., which uses coding techniques when distributed computing servers exchange their computed intermediate values, in order to reduce the overall traffic load. Their original model servers are connected via an…
Yingjie Cheng, Gaojun Luo, Xiwang Cao, Martianus Frederic Ezerman + 1 more
'San Ling'] A coded distributed computing (CDC) system aims to reduce the communication load in the MapReduce framework. Such a system has K nodes, N input files, and Q Reduce functions. Each input file is mapped by r nodes and each Reduce function is computed by s nodes. The objective is to achieve the maximum…
Yingjie Cheng, Gaojun Luo, Xiwang Cao, Martianus Frederic Ezerman + 1 more
'San Ling'] Coded distributed computing (CDC) was introduced to greatly reduce the communication load for MapReduce computing systems. Such a system has K nodes, N input files, and Q Reduce functions. Each input file is mapped by r nodes and each Reduce function is computed by s nodes. The architecture must allow for…
Songze Li, Sucha Supittayapornpong, Mohammad Ali Maddah-Ali, A. Salman Avestimehr
'A. Salman Avestimehr'] Abstract—We focus on sorting, which is the building block of many machine learning algorithms, and propose a novel distributed sorting algorithm, named CodedTeraSort, which substantially improves the execution time of the TeraSort benchmark in Hadoop MapReduce. The key idea of CodedTeraSort is…
Jer Shyuan Ng, Wei Yang Bryan Lim, Nguyen Cong Luong, Zehui Xiong + 4 more
'Alia Asheralieva' 'Dusit Niyato' 'Cyril Leung' 'Chunyan Miao'] Abstract—Distributed computing has become a common approach for large-scale computation of tasks due to benefits such as high reliability, scalability, computation speed, and costeffectiveness. However, distributed computing faces critical issues related…
Authors not listed
Machine learning models are transforming data-driven research across scientific disciplines, yet their deployment as accessible and reliable web services remains a significant challenge. We introduce the NERDD framework, a scalable, maintainable, and secure microservices platform designed to support the sustainable…