22 papers · ranked by Valyu relevance
Giorgio Taricco, Miguel Rubi
Entropy regularization is a recurring mechanism in reinforcement learning (RL), but its meaning changes across algorithmic settings. In classical online RL, entropy encourages exploration and smooths policy improvement; in inverse RL and imitation learning, maximum-entropy resolves ambiguity among expert-consistent…
David Peer, Bart Keulen, Sebastian Stabinger, Justus Piater + 1 more
'Antonio Rodríguez–Sánchez'] Training deep neural networks is a very demanding task, especially challenging is how to adapt architectures to improve the performance of trained models. We can find that sometimes, shallow networks generalize better than deep networks, and the addition of more layers results in higher…
Authors not listed
Electrochemical impedance spectroscopy (EIS) coupled with distribution of relaxation times (DRT) analysis is a robust framework for characterizing electrochemical systems. However, DRT deconvolution is often plagued by spurious peaks, hindering accurate process identification and quantitative parameter estimation. To…
Daniele Musso
Local entropic loss functions provide a versatile framework to define architecture-aware regularization procedures. Besides the possibility of being anisotropic in the synaptic space, the local entropic smoothening of the loss function can vary during training, thus yielding a tunable model complexity. A scoping…
Koffka Khan, Ercan Kuruoglu
Personalized Federated Learning (PFL) seeks to train client-specific models across distributed data silos with heterogeneous distributions. We introduce Sign-Entropy Regularization (SER), a novel entropy-based regularization technique that penalizes excessive directional variability in client-local optimization.…
Authors not listed
Inverse molecular design aims to generate novel chemical structures that satisfy multiple property constraints, yet reinforcement-learning (RL) fine-tuning can be sensitive to how objectives are converted into a scalar reward. Here, we systematically analyze how scalarization choices and stabilization mechanisms shape…
Zenglan Su, Manuchehr Soleimani, Yandan Jiang, Haifeng Ji + 2 more
'Baoliang Wang' 'Lei Wang'] Regularization with priors is an effective approach to solve the ill-posed inverse problem of electrical tomography. Entropy priors have been proven to be promising in radiation tomography but have received less attention in the literature of electrical tomography. This work aims to…
Zahra Amini Farsani, Volker J Schmid
This paper aims to solve the arterial input function (AIF) determination in dynamic contrast-enhanced MRI (DCE-MRI), an important linear ill-posed inverse problem, using the maximum entropy technique (MET) and regularization functionals. In addition, estimating the pharmacokinetic parameters from a DCE-MR image…
Abdulrahman Oladipupo Ibraheem
I introduce two novel loss functions for classification in deep learning. The two loss functions extend standard cross entropy loss by regularizing it with minimum entropy and Kullback-Leibler (K-L) divergence terms. The first of the two novel loss functions is termed mixed entropy loss (MIX-ENT for short), while the…
Hovig Tigran Bayandorian
Lp-norm regularization schemes such as L0, L1, and L2-norm regularization and Lp-norm-based regularization techniques such as weight decay, LASSO, and elastic net compute a quantity which depends on model weights considered in isolation from one another. This paper introduces a regularizer based on minimizing a novel…
Lena-Marie Woelk, Sukanya A. Kannabiran, Valerie Brock, Christine E. Gee + 4 more
Live cell Ca^2+^ fluorescence microscopy is a cornerstone of cellular signaling analysis and imaging. The demand for high spatial and temporal imaging resolution is, however, intrinsically linked to a low signal-to-noise ratio (SNR) of the acquired spatio-temporal image data, which impedes subsequent image analysis.…
Trong-Tuan Nguyen, Van-Dat Thang, Nguyen Van Thin, Phuong T. Nguyen
for neural networks Authors: ['Trong-Tuan Nguyen' 'Van-Dat Thang' 'Nguyen Van Thin' 'Phuong T. Nguyen'] Abstract The entropy error function has been widely used in neural networks. Nevertheless, the network training based on this error function generally leads to a slow convergence rate, and can easily be trapped in a…
Mackenzie J. Meni, Ryan T. White, Michael Mayo, Kevin R. Pilkiewicz
Neural networks have dramatically increased our capacity to learn from large, high-dimensional datasets across innumerable disciplines. However, their decisions are not easily interpretable, their computational costs are high, and building and training them are not straightforward processes. To add structure to these…
Antony Mizzi, David M. Walker, Michael Small, José F. F. Mendes
We derive a penalty strength criterion for ridge regression using stochastic complexity, which is a refined variant of the minimum description length principle. Since stochastic complexity does not typically account for the effect of regularization on complexity, despite its ability to simplify models, we are required…
Authors not listed
Gibbs’ paradox—the apparent discontinuity in mixing entropy for gases of varying similarity and the seeming reversibility of mixing-separation cycles—has resisted fully satisfactory resolution for 150 years. We present a solution based on categorical state theory, which posits that physical con f igurations are…
Illia Horenko, Davide Bassetti, Lukáš Pospíšil
Shannon entropy (SE) and its quantum mechanical analogue von Neumann entropy are key components in many tools used in physics, information theory, machine learning (ML) and quantum computing. Besides of the significant amounts of SE computations required in these fields, the singularity of the SE gradient is one of the…
Stephanie Wankowicz, James Fraser
During protein folding, proteins transition from a disordered polymer into a globular structure, markedly decreasing their conformational degrees of freedom and consequently leading to a substantial reduction in entropy. Nonetheless, folded proteins still retain significant entropy as they fluctuate between the…
Salomé A. Sepúlveda-Fontaine, José M. Amigó, Sotiris Kotsiantis
Since its origin in the thermodynamics of the 19th century, the concept of entropy has also permeated other fields of physics and mathematics, such as Classical and Quantum Statistical Mechanics, Information Theory, Probability Theory, Ergodic Theory and the Theory of Dynamical Systems. Specifically, we are referring…
Guanghui Zhang, Xinran Wang, Steven J. Luck
Regularization has been extensively used in multivariate pattern classification (MVPA; decoding) of EEG data to mitigate the risk of overfitting. N-fold cross-validation is also used to mitigate this risk, and it is often combined with averaging across trials to improve the signal-to-noise ratio. However, the impact of…
Griffin S. Hampton, Ryan Neff, Zezheng Song, Mustapha Bouhrara + 2 more
Myelin water fraction (MWF) mapping in the central nervous system is a topic of intense research activity. One framework for this requires parameter estimation from a decaying biexponential signal. However, this is often an ill-posed nonlinear problem resulting in unreliable parameter estimates. For linear…
Giovanna Maria Dimitri
In this work, done in collaboration with Prof. Michelangelo Diligenti (department of Engineering and Mathematics, University of Siena) we present the use of Semantic Based Regularization Kernel based machine learning method to predict protein function. We initially build the protein functions ontology, given an initial…
Authors not listed
Phase equilibrium calculations are crucial in chemical engineering design and optimization processes. The PC-SAFT equation of state (EoS) can precisely calculate phase equilibrium, but is relatively complex and computationally intensive. Surrogate models are mathematically simple models that map or regress the…