Extended Information Geometry: Large Deviation Theory, Statistical Thermodynamics, and Empirical Counting Frequencies

Abstract

Thermodynamics, and Empirical Counting Frequencies Authors: ['Viswa Virinchi Muppirala' 'Hong Qian'] Combinatorics, probabilities, and measurements are fundamental to understanding information. This work explores how the application of large deviation theory (LDT) in counting phenomena leads to the emergence of various entropy functions, including Shannon's entropy, mutual information, and relative and conditional entropies. In terms of these functions, we reveal an inherent geometrical structure through operations, including contractions, lift, change of basis, and projections. Legendre-Fenchel (LF) transform that is central to both LDT and Gibbs' method of thermodynamics offers a novel energetic description of data. The manifold of empirical mean values of statistical data ad infinitum has a parametrization using LF conjugates w.r.t. an entropy function; this gives rise to the additivity known in statistical thermodynamic energetics. This work extends current information geometry to information projection as defined through conditional expectations in Kolmogorov's probability theory.