13 papers · ranked by Valyu relevance
Julius Vetter, Guy Moss, Cornelius Schröder, Richard Gao + 1 more
'Jakob H. Macke'] Scientific modeling applications often require estimating a distribution of parameters consistent with a dataset of observations—an inference task also known as source distribution estimation. This problem can be ill-posed, however, since many different source distributions might produce the same…
Vikash Singh, Debargha Ganguly, Weicong Chen, Sabyasachi Sahoo + 7 more
Continual test-time adaptation (CTTA) updates a pretrained model online on an unlabeled, non-stationary stream while anchoring it to a frozen source checkpoint. This anchor is useful only when the source remains reliable. On CCC-Hard, however, a ResNet-50 source falls to approximately $1.3\%$ top-$1$ accuracy, while…
Ravishankar Ramanathan
The extraction of randomness from weakly random seeds is a problem of central importance with multiple applications. In the device-independent setting, this problem of quantum randomness amplification has been mainly restricted to specific weak sources of Santha-Vazirani type, while extraction from the general…
Xin Li
We continue the study of constructing explicit extractors for independent general weak random sources. The ultimate goal is to give a construction that matches what is given by the probabilistic method — an extractor for two independent n-bit weak random sources with minentropy as small as log n + O(1). Previously, the…
Naveen Kumar, Ambesh Dixit, Vivek Vijay
Entropy has emerged as a dynamic, interdisciplinary, and widely accepted quantitative measure of uncertainty across different disciplines. A unified understanding of entropy measures, supported by a detailed review of their theoretical foundations and practical applications, is crucial to advance research across…
Marko Popovic
Entropy concept was introduced by Clausius 160 years ago, and has been continually enriched, developed and interpreted by the researchers in many different scientific disciplines ever since. Thermodynamics and other scientific disciplines face several simple but crucial questions concerning the entropy concept. Thus…
Merab Gogberashvili
We describe the universe as a single entangled ensemble of quantum particles. The total entropy of this world ensemble, which can be expressed as a sum of information, thermodynamic and entanglement components, is assumed to be always zero. This condition suggests information quantization, which we associate with the…
Tommaso Toffoli
Here we deconstruct, and then in a reasoned way reconstruct, the concept of "entropy of a system," paying particular attention to where the randomness may be coming from. We start with the core concept of entropy as a count associated with a description; this count (traditionally expressed in logarithmic form for a…
Rafael D. Sorkin
We express the entropy of a scalar field φ directly in terms of its spacetime correlation function W(x, y) = hφ(x)φ(y)i, assuming that the higher correlators are of "Gaussian" form. The resulting formula associates an entropy S ( R) to any spacetime region R ; and when R is globally hyperbolic with Cauchy surface Σ …
Phil Attard
In 1948 Claude E. Shannon published a paper on the mathematical theory of communications that quickly became a classic. It established the field of information theory, which is vital to disciplines such as genomics, cryptology, computer science, network engineering, signal processing, data storage, quantum informatics…
Phil Attard
It is shown that the standard expression for the information entropy, originally due to Shannon, is only valid for a particular set of states. For the general case of statistical mechanics, one needs to include an additional term in the expression for the entropy as a function of the probability. A simple derivation of…
Andreas Schlatter, R. E. Kastner
A key tenet of the Transactional Interpretation of Quantum Mechanics is the idea that photon absorption localizes the absorbing material system. In doing so, it measures the location of the absorber and hence reduces information entropy which in turn needs to be balanced by appropriate entropy production, if there is a…
Tarek Tohme, William Bialek
At the 2023 Les Houches Summer School on Theoretical Biological Physics, several students asked for some background on information theory, and so we added a tutorial to the scheduled lectures. This is largely a transcript of that tutorial, lightly edited. It covers basic definitions and context rather than detailed…