Search · four archives
Search · four archives
17 papers · ranked by Valyu relevance
Yuhan Wang, Youlong Wu
Coded distributed computing (CDC) can reduce the communication load for distributed computing systems by introducing redundant computation and creating multicasting opportunities. The optimal computation-communication tradeoff has been well studied for homogeneous systems, and some results have been obtained under…
Yongcheng Yang, Yifei Huang, Xiaohuan Qin, Shenglian Lu + 3 more
'Yanlin Geng' 'Youlong Wu' 'Ling Liu'] Coded distributed computing (CDC) is a powerful approach to reduce the communication overhead in distributed computing frameworks by utilizing coding techniques. In this paper, we focus on the CDC problem in $(H,L)$-combination networks, where H APs act as intermediate pivots and…
Federico Brunero, Petros Elia
Coded distributed computing (CDC) is a new technique proposed with the purpose of decreasing the intense data exchange required for parallelizing distributed computing systems. Under the famous MapReduce paradigm, this coded approach has been shown to decrease this communication overhead by a factor that is linearly…
Yingjie Cheng, Gaojun Luo, Xiwang Cao, Martianus Frederic Ezerman + 1 more
'San Ling'] A coded distributed computing (CDC) system aims to reduce the communication load in the MapReduce framework. Such a system has K nodes, N input files, and Q Reduce functions. Each input file is mapped by r nodes and each Reduce function is computed by s nodes. The objective is to achieve the maximum…
Mingming Zhang, Youlong Wu, Minquan Cheng, Dianhua Wu
Coded distributed computing, proposed by Li et al., offers significant potential for reducing the communication load in MapReduce computing systems. In the setting of the cascaded coded distributed computing that consisting of K nodes, N input files, and Q output functions, the objective is to compute each output…
Qicheng Zeng, Zhaojun Nan, Sheng Zhou, T. Aaron Gulliver
Coded computing is recognized as a promising solution to address the privacy leakage problem and the straggling effect in distributed computing. This technique leverages coding theory to recover computation tasks using results from a subset of workers. In this paper, we propose the adaptive privacy-preserving coded…
Minquan Cheng, Yongkang Wang, Lingyu Zhang, Youlong Wu
Distributed multi-task learning (DMTL) effectively improves model generalization performance through the collaborative training of multiple related models. However, in large-scale learning scenarios, communication bottlenecks severely limit practical system performance. In this paper, we investigate the communication…
Yingjie Cheng, Gaojun Luo, Xiwang Cao, Martianus Frederic Ezerman + 1 more
'San Ling'] Coded distributed computing (CDC) was introduced to greatly reduce the communication load for MapReduce computing systems. Such a system has K nodes, N input files, and Q Reduce functions. Each input file is mapped by r nodes and each Reduce function is computed by s nodes. The architecture must allow for…
Youlong Wu, Zhen-hao Huang, Kai Yuan, Shuai Ma + 1 more
—Distributed computing frameworks such as MapReduce and Spark are often used to process large-scale data computing jobs. In wireless scenarios, exchanging data among distributed nodes would seriously suffer from the communication bottleneck due to limited communication resources such as bandwidth and power. To address…
Derya Malak
Our work addresses the well-known open problem of distributed computing of bilinear functions of two correlated sources A and B. In a setting with two nodes, with the first node having access to A and the second to B, we establish bounds on the optimal sum-rate that allows a receiver to compute an important class of…
Bin Fan, Bin Tang, Zhihao Qu, Baoliu Ye + 1 more
In wireless distributed computing systems, worker nodes connect to a master node wirelessly and perform large-scale computational tasks that are parallelized across them. However, the common phenomenon of straggling (i.e., worker nodes often experience unpredictable slowdown during computation and communication) and…
Ming Xiao, Mikael Skoglund, H. Vincent Poor, Onur Günlü + 2 more
'Rafael F. Schaefer' 'Holger Boche'] This article aims to give a comprehensive and rigorous review of the principles and recent development of coding for large-scale distributed machine learning (DML). With increasing data volumes and the pervasive deployment of sensors and computing machines, machine learning has…
Derya Malak, Mohammad Reza Deylam Salehi, Berksan Serbetci, Petros Elia + 2 more
'Petros Elia' 'Chintha Tellambura' 'Jun Chen'] The work here studies the communication cost for a multi-server multi-task distributed computation framework, as well as for a broad class of functions and data statistics. Considering the framework where a user seeks the computation of multiple complex (conceivably…
Jia Lu, Ryan Tsoi, Nan Luo, Yuanchi Ha + 8 more
Dynamical systems often generate distinct outputs according to different initial conditions, and one can infer the corresponding input configuration given an output. This property captures the essence of information encoding and decoding. Here, we demonstrate the use of self-organized patterns, combined with machine…
Adina S. Wagner, Laura K. Waite, Małgorzata Wierzba, Felix Hoffstaedter + 4 more
Large-scale datasets present unique opportunities to perform scientific investigations with unprecedented breadth. However, they also pose considerable challenges for the findability, accessibility, interoperability, and reusability (FAIR) of research outcomes due to infrastructure limitations, data usage constraints…
Cláudia Brito, Pedro Ferreira, João Paulo
Breakthroughs in sequencing technologies led to an exponential growth of genomic data, providing unprecedented biological in-sights and new therapeutic applications. However, analyzing such large amounts of sensitive data raises key concerns regarding data privacy, specifically when the information is outsourced to…
Authors not listed
Machine learning models are transforming data-driven research across scientific disciplines, yet their deployment as accessible and reliable web services remains a significant challenge. We introduce the NERDD framework, a scalable, maintainable, and secure microservices platform designed to support the sustainable…