20 papers · ranked by Valyu relevance
Leo D’Amato, Gian Luca Lancia, Giovanni Pezzulo, Tatyana O. Sharpee
Living organisms rely on internal models of the world to act adaptively. These models, because of resource limitations, cannot encode every detail and hence need to compress information. From a cognitive standpoint, information compression can manifest as a distortion of latent representations, resulting in the…
Henry Pinkard, Laura Waller
Though originally developed for communications engineering, information theory contains mathematical tools with numerous applications in science and engineering. These tools can be used to characterize the fundamental limits of data compression and transmission in the presence of noise. Here, we present a practical…
Jiayang Zou, Luyao Fan, Jiayang Gao, Jia Wang
—In this paper, we study rate-distortion theory for general sources with an emphasis on the existence of optimal reconstruction distributions. Classical existence results rely on compactness assumptions with continuous distortion that are often violated in general settings. By introducing the concentrationcompactness…
Jingxuan Chai, Huixiang Zhu, Yong Xiao, Guangming Shi + 2 more
'Shenghui Song'] Semantic communication has attracted considerable interest due to its potential to support emerging human-centric services, such as holographic communications, extended reality (XR), and human-machine interactions. Different from traditional communication systems that focus on minimizing the…
David G. Nagy, Balázs Török, Gergő Orbán
It has extensively been documented that human memory exhibits a wide range of systematic distortions, which have been associated with resource constraints. Resource constraints on memory can be formalised in the normative framework of lossy compression, however traditional lossy compression algorithms result in…
Peter Harremoës, Raúl Alcaraz, Leandro Pardo, Luca Faes + 1 more
'Boris Ryabko'] Rate distortion theory was developed for optimizing lossy compression of data, but it also has applications in statistics. In this paper, we illustrate how rate distortion theory can be used to analyze various datasets. The analysis involves testing, identification of outliers, choice of compression…
Rodrick Wallace
The living state is cognitive at every scale and level of organization. Since it is possible to associate a broad class of cognitive processes with ‘dual’ information sources, many pathologies can be addressed using statistical models based on the Shannon Coding, the Shannon-McMillan Source Coding, the Rate Distortion…
Jiayang Zou, Luyao Fan, Jiayang Gao, Jia Wang
—This paper revisits the rate-distortion theory from the perspective of optimal weak transport, as recently introduced by Gozlan et al. While the conditions for optimality and the existence of solutions are well-understood in the case of discrete alphabets, the extension to abstract alphabets requires more intricate…
Gangtao Xin, Pingyi Fan, Khaled B. Letaief, Mateu Sbert
In recent years, semantic communication has received significant attention from both academia and industry, driven by the growing demands for ultra-low latency and high-throughput capabilities in emerging intelligent services. Nonetheless, a comprehensive and effective theoretical framework for semantic communication…
Jerry Gibson
Shannon introduced the fields of information theory and rate distortion theory in his landmark 1948 paper [1], where he defined “The Rate for a Source Relative to a Fidelity Evaluation.” Shannon officially coined the term “rate distortion function” in his seminal contribution in 1959 [2]. The 1950s, 1960s and 1970s…
Anthony M.V. Jakob, Samuel J. Gershman
Rate-distortion theory provides a powerful framework for understanding the nature of human memory by formalizing the relationship between information rate (the average number of bits per stimulus transmitted across the memory channel) and distortion (the cost of memory errors). Here we show how this abstract…
Authors not listed
—Semantic communication has emerged as a novel communication paradigm that focuses on conveying the user's intended meaning rather than the bit-wise transmission of source signals. One of the key challenges is to effectively represent and extract the semantic meaning of any given source signals. While deep learning…
Li Xie, Liangyan Li, Jun Chen, Lei Yu + 2 more
'Luca Barletta'] A constrained version of Talagrand’s transportation inequality is established, which reveals an intrinsic connection between the Gaussian distortion-rate-perception functions with limited common randomness under the Kullback-Leibler divergence-based and squared Wasserstein-2 distance-based perception…
Mehdi Salehifar, Tejaswi Nanjundaswamy, Kenneth Rose
—This paper studies a layered coding framework with a relaxed hierarchical structure. Advances in wired/wireless communication and consumer electronic devices have created a requirement for serving the same content at different quality levels. The key challenge is to optimally encode all the required quality levels…
Artemy Kolchinsky, Bernat Corominas-Murtra
In many real-world systems, information can be transmitted in two qualitatively different ways: by copying or by transformation. Copying occurs when messages are transmitted without modification, e.g., when an offspring receives an unaltered copy of a gene from its parent. Transformation occurs when messages are…
Meixia Tao, Kai Niu, Youlong Wu
Traditional information theory provides a rigorous foundation for information compression and reliable symbol transmission. However, in emerging applications such as autonomous driving, remote healthcare, and industrial Internet of Things (IoT), the key communication challenge has been shifted from accurate delivery of…
Mark M. Wilde, Nilanjana Datta, Min-Hsiu Hsieh, Andreas Winter
—We extend quantum rate distortion theory by considering auxiliary resources that might be available to a sender and receiver performing lossy quantum data compression. The first setting we consider is that of quantum rate distortion coding with the help of a classical side channel. Our result here is that the…
Samuel Espley, Samuel Allsop, David Buttar, Simone Tomasi + 1 more
Machine learning (ML) models have provided a highly efficient pathway to quantum mechanical accurate reaction barrier predictions. Previous approaches have, however, stopped at prediction of these barriers instead of developing predictive capabilities in reactivity analysis tasks such as…
James V. Stone
Shannon's mathematical theory of communication defines fundamental limits on how much information can be transmitted between the different components of any man-made or biological system. This paper is an informal but rigorous introduction to the main ideas implicit in Shannon's theory. An annotated reading list is…
Authors not listed
Gibbs’ paradox—the apparent discontinuity in mixing entropy for gases of varying similarity and the seeming reversibility of mixing-separation cycles—has resisted fully satisfactory resolution for 150 years. We present a solution based on categorical state theory, which posits that physical con f igurations are…