Search · four archives
Search · four archives
11 papers · ranked by Valyu relevance
Kangwook Lee, Maximilian Lam, Ramtin Pedarsani, Dimitris Papailiopoulos + 1 more
'Dimitris Papailiopoulos' 'Kannan Ramchandran'] Codes are widely used in many engineering applications to offer robustness against noise. In large-scale systems there are several types of noise that can affect the performance of distributed machine learning algorithms – straggler nodes, system failures, or…
Songze Li, Mohammad Ali Maddah-Ali, A. Salman Avestimehr
More specifically, a general distributed computing framework, motivated by commonly used structures like MapReduce, is considered, where the overall computation is decomposed into computing a set of "Map" and "Reduce" functions distributedly across multiple computing nodes. A coded scheme, named "Coded Distributed…
Kai Wan, Mingyue Ji, Giuseppe Caire
—This paper considers the MapReduce-like coded distributed computing framework originally proposed by Li et al., which uses coding techniques when distributed computing servers exchange their computed intermediate values, in order to reduce the overall traffic load. Their original model servers are connected via an…
Yingjie Cheng, Gaojun Luo, Xiwang Cao, Martianus Frederic Ezerman + 1 more
'San Ling'] A coded distributed computing (CDC) system aims to reduce the communication load in the MapReduce framework. Such a system has K nodes, N input files, and Q Reduce functions. Each input file is mapped by r nodes and each Reduce function is computed by s nodes. The objective is to achieve the maximum…
Yingjie Cheng, Gaojun Luo, Xiwang Cao, Martianus Frederic Ezerman + 1 more
'San Ling'] Coded distributed computing (CDC) was introduced to greatly reduce the communication load for MapReduce computing systems. Such a system has K nodes, N input files, and Q Reduce functions. Each input file is mapped by r nodes and each Reduce function is computed by s nodes. The architecture must allow for…
Songze Li, Sucha Supittayapornpong, Mohammad Ali Maddah-Ali, A. Salman Avestimehr
'A. Salman Avestimehr'] Abstract—We focus on sorting, which is the building block of many machine learning algorithms, and propose a novel distributed sorting algorithm, named CodedTeraSort, which substantially improves the execution time of the TeraSort benchmark in Hadoop MapReduce. The key idea of CodedTeraSort is…
Jer Shyuan Ng, Wei Yang Bryan Lim, Nguyen Cong Luong, Zehui Xiong + 4 more
'Alia Asheralieva' 'Dusit Niyato' 'Cyril Leung' 'Chunyan Miao'] Abstract—Distributed computing has become a common approach for large-scale computation of tasks due to benefits such as high reliability, scalability, computation speed, and costeffectiveness. However, distributed computing faces critical issues related…
Jia Lu, Ryan Tsoi, Nan Luo, Yuanchi Ha + 8 more
Dynamical systems often generate distinct outputs according to different initial conditions, and one can infer the corresponding input configuration given an output. This property captures the essence of information encoding and decoding. Here, we demonstrate the use of self-organized patterns, combined with machine…
Timothy C. Haas
Models of political-ecological systems can inform policies for managing ecosystems that contain endangered species. One way to increase the credibility of these models is to subject them to a rigorous suite of data-based statistical assessments. Doing so involves statistically estimating the model’s parameters…
Pierre Carrier, Bill Long, Richard Walsh, Jef Dawson + 4 more
High Performance Computing (HPC) Best Practice offers opportunities to implement lessons learned in areas such as computational chemistry and physics in genomics workflows, specifically Next-Generation Sequencing (NGS) workflows. In this study we will briefly describe how distributed-memory parallelism can be an…
Cláudia Brito, Pedro Ferreira, João Paulo
Breakthroughs in sequencing technologies led to an exponential growth of genomic data, providing unprecedented biological in-sights and new therapeutic applications. However, analyzing such large amounts of sensitive data raises key concerns regarding data privacy, specifically when the information is outsourced to…