Search · four archives
Search · four archives
19 papers · ranked by Valyu relevance
Yuhan Wang, Youlong Wu
Coded distributed computing (CDC) can reduce the communication load for distributed computing systems by introducing redundant computation and creating multicasting opportunities. The optimal computation-communication tradeoff has been well studied for homogeneous systems, and some results have been obtained under…
Jinbao Zhu, Songze Li
We consider the problem of evaluating arbitrary multivariate polynomials over a massive dataset containing multiple inputs, on a distributed computing system with a master node and multiple worker nodes. Generalized Lagrange Coded Computing (GLCC) codes are proposed to simultaneously provide resiliency against…
Minquan Cheng, Youlong Wu, Xianxian Li
—Coded distributed computing (CDC) introduced by Li et al. can greatly reduce the communication load for MapReduce computing systems. In the cascaded CDC with K workers, N input files and Q output functions, each input file will be mapped by r workers and each output function will be computed by s workers such that…
Federico Brunero, Petros Elia
Coded distributed computing (CDC) is a new technique proposed with the purpose of decreasing the intense data exchange required for parallelizing distributed computing systems. Under the famous MapReduce paradigm, this coded approach has been shown to decrease this communication overhead by a factor that is linearly…
Yingjie Cheng, Gaojun Luo, Xiwang Cao, Martianus Frederic Ezerman + 1 more
'San Ling'] A coded distributed computing (CDC) system aims to reduce the communication load in the MapReduce framework. Such a system has K nodes, N input files, and Q Reduce functions. Each input file is mapped by r nodes and each Reduce function is computed by s nodes. The objective is to achieve the maximum…
Zhenhao Huang, Minquan Cheng, Kai Wan, Qifu Tyler Sun + 1 more
We consider a coded distributed computing problem in a ring-based communication network, where N computing nodes are arranged in a ring topology and each node can only communicate with its neighbors within a constant distance d. To mitigate the communication bottleneck in exchanging intermediate values, we propose new…
Homa Esfahanizadeh, Alejandro Cohen, Muriel Médard, Shlomo Shamai
—Modern computationally-heavy applications are often time-sensitive, demanding distributed strategies to accelerate them. On the other hand, distributed computing suffers from the bottleneck of slow workers in practice. Distributed coded computing is an attractive solution that adds redundancy such that a subset of…
Mingming Zhang, Youlong Wu, Minquan Cheng, Dianhua Wu
Coded distributed computing, proposed by Li et al., offers significant potential for reducing the communication load in MapReduce computing systems. In the setting of the cascaded coded distributed computing that consisting of K nodes, N input files, and Q output functions, the objective is to compute each output…
Minquan Cheng, Yongkang Wang, Lingyu Zhang, Youlong Wu
Distributed multi-task learning (DMTL) effectively improves model generalization performance through the collaborative training of multiple related models. However, in large-scale learning scenarios, communication bottlenecks severely limit practical system performance. In this paper, we investigate the communication…
Yingjie Cheng, Gaojun Luo, Xiwang Cao, Martianus Frederic Ezerman + 1 more
'San Ling'] Coded distributed computing (CDC) was introduced to greatly reduce the communication load for MapReduce computing systems. Such a system has K nodes, N input files, and Q Reduce functions. Each input file is mapped by r nodes and each Reduce function is computed by s nodes. The architecture must allow for…
Youlong Wu, Zhen-hao Huang, Kai Yuan, Shuai Ma + 1 more
—Distributed computing frameworks such as MapReduce and Spark are often used to process large-scale data computing jobs. In wireless scenarios, exchanging data among distributed nodes would seriously suffer from the communication bottleneck due to limited communication resources such as bandwidth and power. To address…
Derya Malak
Our work addresses the well-known open problem of distributed computing of bilinear functions of two correlated sources A and B. In a setting with two nodes, with the first node having access to A and the second to B, we establish bounds on the optimal sum-rate that allows a receiver to compute an important class of…
Mohammad Reza Deylam Salehi, Vijith Kumar Kizhakke Purakkal, Derya Malak
'Derya Malak'] Abstract—This work addresses the K-user computation broadcast problem consisting of a master node, that holds all datasets and users for a general class of function demands, including linear and non-linear functions, over finite fields. The master node sends a broadcast message to enable each of K…
Derya Malak, Mohammad Reza Deylam Salehi, Berksan Serbetci, Petros Elia
'Petros Elia'] The work here studies the communication cost for a multi-server multi-task distributed computation framework, and does so for a broad class of functions and data statistics. Considering the framework where a user seeks the computation of multiple complex (conceivably non-linear) tasks from a set of…
Jiepeng Tang, Navneet Agrawal, Sławomir Stańczak, Jingge Zhu
In this paper, we present a coded computation (CC) scheme for distributed computation of the inference phase of machine learning (ML) tasks, specifically, the task of image classification. Building upon Agrawal et al. 2022, the proposed scheme combines the strengths of deep learning and Lagrange interpolation technique…
Zhongfu Guo, Xinsheng Ji, Wei You, Yu Zhao + 2 more
Offloading Method within Edge Networks Authors: ['Zhongfu Guo' 'Xinsheng Ji' 'Wei You' 'Yu Zhao' 'Yi Bai' 'Lingwei Wang'] Abstract—This paper introduces REDC, a comprehensive strategy for offloading computational tasks within mobile Edge Networks (EN) to Distributed Computing (DC) after Rateless Encoding (RE). Despite…
Canran Wang, Netanel Raviv
—We consider the problem of encoding information in a system of N = K + R processors that operate in a decentralized manner, i.e., without a central processor which orchestrates the operation. The system involves K source processors, each holding some data modeled as a vector over a finite field. The remaining R…
Wei Geng, Yulong Zhang, Dirk Kutscher, Abhishek Kumar + 2 more
'Pan Hui'] Information-Centric Networking (ICN), with its data-oriented operation and generally more powerful forwarding layer, provides an attractive platform for distributed computing. This paper provides a systematic overview and categorization of different distributed computing approaches in ICN encompassing…
Xiuzhen Cheng, Minghui Xu, Runyu Pan, Dongxiao Yu + 3 more
'Xiao Xue' 'Weifeng Lyu'] Abstract—With the continuous improvement of information infrastructures, academia and industry have been constantly exploring new computing paradigms to fully exploit computing powers. In this paper, we propose Meta Computing, a new computing paradigm that aims to utilize all available…