21 papers · ranked by Valyu relevance
El Mahdi Mouloua, Essaid Mohamed
Information theory is introduced in this lecture note with a particular emphasis on its relevance to algebraic coding theory. The document develops the mathematical foundations for quantifying uncertainty and information transmission by building upon Shannon's pioneering formulation of information, entropy, and channel…
James V. Stone
Shannon's mathematical theory of communication defines fundamental limits on how much information can be transmitted between the different components of any man-made or biological system. This paper is an informal but rigorous introduction to the main ideas implicit in Shannon's theory. An annotated reading list is…
Henry Pinkard, Laura Waller
Though originally developed for communications engineering, information theory contains mathematical tools with numerous applications in science and engineering. These tools can be used to characterize the fundamental limits of data compression and transmission in the presence of noise. Here, we present a practical…
Thomas F. Varley
In the 21st century, many of the crucial scientific and technical issues facing humanity can be understood as problems associated with understanding, modelling, and ultimately controlling complex systems: systems comprised of a large number of non-trivially interacting components whose collective behaviour can be…
Steven M. Silverstein, Michael Wibral, William A. Phillips
Information theory provides a formal framework within which information processing and its disorders can be described. However, information theory has rarely been applied to modeling aspects of the cognitive neuroscience of schizophrenia. The goal of this article is to highlight the benefits of an approach based on…
Rosanna C. G. Smith, Ben D. MacArthur
To outline how ideas from Information Theory may be used to analyze single cell data and better understand stem cell behaviour. Recent technological breakthroughs in single cell profiling have made it possible to interrogate cell-to-cell variability in a multitude of contexts, including the role it plays in stem cell…
Sergio Verdú
We give a brief survey of the literature on the empirical estimation of entropy, differential entropy, relative entropy, mutual information and related information measures. While those quantities are of central importance in information theory, universal algorithms for their estimation are increasingly important in…
Christoph Adami
Information is a precise concept that can be defined mathematically, but its relationship to what we call ‘knowledge’ is not always made clear. Furthermore, the concepts ‘entropy’ and ‘information’, while deeply related, are distinct and must be used with care, something that is not always achieved in the literature.…
Yair Lapin
The Turing machine halting problem can be explained by several factors, including arithmetic logic irreversibility and memory erasure, which contribute to computational uncertainty due to information loss during computation. Essentially, this means that an algorithm can only preserve information about an input, rather…
Yunus A. Çengel, Milivoje M. Kostic, Jean-Noël Jaubert
The term entropy is used in different meanings in different contexts, sometimes in contradictory ways, resulting in misunderstandings and confusion. The root cause of the problem is the close resemblance of the defining mathematical expressions of entropy in statistical thermodynamics and information in the…
Artemy Kolchinsky, Bernat Corominas-Murtra
In many real-world systems, information can be transmitted in two qualitatively different ways: by copying or by transformation. Copying occurs when messages are transmitted without modification, e.g., when an offspring receives an unaltered copy of a gene from its parent. Transformation occurs when messages are…
David J. Galas, James Kunert-Graf, Lisa Uechi, Nikita A. Sakhanenko
Quantitative genetics has evolved dramatically in the past century, and the proliferation of genetic data enables the characterization of complex interactions beyond the scope of its theoretical foundations. In this paper, we lay the foundations of an alternative formulation of quantitative genetics based on…
Olimpia Lombardi, Cristian López
Integrated Information Theory (IIT) intends to provide a principled theoretical approach able to characterize consciousness both quantitatively and qualitatively. By starting off identifying the fundamental properties of experience itself, IIT develops a formal framework that relates those properties to the physical…
William Sulis
Information is the fundamental currency of naturally occurring complex adaptive systems, whether they are individual organisms or collective social insect colonies. Information appears to be more important than energy in determining the behavior of these systems. However, it is not the quantity of information but…
Rodrick Wallace
The living state is cognitive at every scale and level of organization. Since it is possible to associate a broad class of cognitive processes with ‘dual’ information sources, many pathologies can be addressed using statistical models based on the Shannon Coding, the Shannon-McMillan Source Coding, the Rate Distortion…
Haengjin Choe
Since the publication of the Shannon’s article about information theory, there have been many attempts to apply information theory to the field of neuroscience. Meanwhile, the Weber-Fechner law describes the relationship between the external physical stimulus and the human response to that. It is not surprising that we…
Authors not listed
Understanding the electronic structure of amino acids is crucial for understanding protein structure and function. We interfaced quantum information indices and ab initio quantum chemistry methods to quantify the locality correlation between neighboring amino acids. Quantum chemical information from the 1-, 2- and…
Authors not listed
Chemical bonding, despite being fundamental to chemistry, lacks a universally agreed-upon quantum mechanical definition that captures all its facets across diverse molecular systems. Traditional models—Lewis structures, Valence Bond theory, Molecular Orbital theory, and the Quantum Theory of Atoms in Molecules…
Jürgen Köfinger, Gerhard Hummer
The proper balancing of information from experiment and theory is a long-standing problem in the analysis of noisy and incomplete data. Viewed as a Pareto optimization problem, improved agreement with the experimental data comes at the expense of growing inconsistencies with the theoretical reference model. Here, we…
Jürgen Köfinger, Gerhard Hummer
The proper balancing of information from experiment and theory is a long-standing problem in the analysis of noisy and incomplete data. Viewed as a Pareto optimization problem, improved agreement with the experimental data comes at the expense of growing inconsistencies with the theoretical reference model. Here, we…
Authors not listed
This paper develops a comprehensive theoretical framework for designing quantum memory systems with enhanced resilience to thermal decoherence through engineered lattice geometries and protective structures. We formulate a unified mathematical description connecting material properties, geometric configurations, and…