22 papers · ranked by Valyu relevance
Khoat Than
In this work, we develop a theoretical framework that elucidates the role of normalization through the lens of capacity control. We prove that an unnormalized DNN can exhibit exponentially large Lipschitz constants with respect to either its parameters or inputs, implying excessive functional capacity and potential…
Yang Qi, Zhichao Zhu, Yiming Wei, Lu Cao + 5 more
The abundance of both input and process noises in the brain suggests that stochasticity is an integral part of neural computing, but how spiking neural networks (SNN) can learn general tasks under correlated variability remain unclear. In this work, we propose a stochastic neural computing (SNC) theory to implement…
Xuan Qi, Yi Wei, Fanqi Yu, Furao shen + 2 more
Batch normalization (BN) is central to modern deep networks, but its effect on the realized function during training remains less understood than its optimization benefits. We study training-time BN in continuous piecewise-affine (CPA) networks through the geometry of switching hyperplanes and the induced affine-region…
Yun Bu, Wenbo Jiang, Gang Lu, Qiang Zhang
When training a neural network, the choice of activation function can greatly impact its performance. A function with a larger derivative may cause the coefficients of the latter layers to deviate further from the calculated direction, making deep learning more difficult to train. However, an activation function with a…
Reto Gerber, Jake Griner, Daniel Incicau, Silvia Guglietta + 2 more
Accurate cell type annotation in imaging mass cytometry (IMC) and related technologies critically depends on preprocessing steps such as normalization, segmentation, and marker aggregation. More distinct separation between negative and positive signals enables more precise cell boundary inference and more robust marker…
Peifeng Gao, Wenyi Fang, Yang Zheng, Difan Zou
Delayed loss spikes have been reported in neural-network training, but existing theory mainly explains earlier non-monotone behavior caused by overly large fixed learning rates. We study one stylized hypothesis: normalization can postpone instability by gradually increasing the effective learning rate during otherwise…
Zou, Xiandong, Zhou Pan
Normalization is fundamental to deep learning, but existing approaches such as BatchNorm, LayerNorm, and RMSNorm are variance-centric by enforcing zero mean and unit variance, stabilizing training without controlling how representations capture task-relevant information. We propose IB-Inspired Normalization (IBNorm), a…
Jeffrey Uhlmann
Deep neural networks constructed from linear maps and positively homogeneous nonlinearities (e.g., ReLU) possess a fundamental gauge symmetry: the network function is invariant to node-wise diagonal rescalings. However, standard gradient descent is not equivariant to this symmetry, causing optimization trajectories to…
Mingzhi Chen, Taiming Lu, Jiachen Zhu, Mingjie Sun + 1 more
Figure 1 We introduce Dynamic erf (Derf), a point-wise function, that outperforms normalization layers and other point-wise functions. (a) We identify the feasible function shape for replacing the normalization layer and propose a large set of point-wise functions within this space. Evaluating all candidates, we…
Jessica K Anderson, Jiwei Zhang, Xinshou Ge, Howard Fan + 10 more
Batch effect correction is a common and often necessary step in data analysis to reduce bias due to technical and experimental factors when combining multiple batches of data. The severity of the batch effects dictates the correction strategy; therefore, a careful assessment of each dataset’s batch effects is…
Jatsada Singthongchai, Tanachapong Wangkhamhan, Zhiming Luo
This study presents a controlled benchmarking analysis of min-max scaling, Z-score normalization, and an adaptive preprocessing pipeline that combines percentile-based ROI cropping with histogram standardization. The evaluation was conducted across four public chest X-ray (CXR) datasets and three convolutional neural…
Authors not listed
Machine learning models are increasingly applied to heterogeneous materials datasets spanning different synthesis routes, measurement protocols, and structural classes. Although multi-task and representation-learning approaches are commonly used to improve predictive performance, the latent representations learned by…
MOHAMED HASSAN, ALEKSANDAR VAKANSKI, BOYU ZHANG, MIN XIAN
The generalization performance of deep neural networks (DNNs) is a critical factor in achieving robust model behavior on unseen data. Recent studies have highlighted the importance of sharpness-based measures in promoting generalization by encouraging convergence to flatter minima. Among these approaches…
Angus F. Chapman, Rachel N. Denison, Christopher Pack
How does the visual system process dynamic inputs? Perception and neural activity are shaped by the spatial and temporal context of sensory input, which has been modeled by divisive normalization over space or time. However, theoretical work has largely treated normalization separately within these dimensions and has…
Qinyue Chen, Yunxin Xie, Zhihui Lai, Zijian Qiao
Intelligent mechanical fault diagnosis plays a key role in maintaining rotating machinery. Although data-driven unsupervised domain adaptation methods have achieved considerable progress, their industrial applications are often restricted by low-quality sensor data. Non-stationary vibration signals and background noise…
Daniel E. Schäffer, Helen Kang, Ekin Deniz Aksu, Daniel Edelman + 1 more
Data from single-cell RNA sequencing (scRNA-seq) and the Assay for Transposase-Accessible Chromatin (scATAC-seq) are high-dimensional, sparse, and undesirably capture technical variability between experiments or batches. Many analysis methods thus seek to produce a low-dimensional cell-by-feature embedding space that…
Deying Song, Douglas Ruff, Marlene Cohen, Chengcheng Huang
Neurons in higher-order visual areas integrate information through a canonical computation called normalization. The strength of normalization is highly heterogeneous across neurons, and this heterogeneity correlates with attention-mediated modulations in neural responses. However, the circuit mechanism underlying the…
Authors not listed
The integration of artificial intelligence and machine learning is rapidly transforming the landscape of materials discovery, facilitating unprecedented acceleration in the exploration of vast chemical spaces and the prediction of material properties. However, the adoption of these advanced techniques has not been…
Mekan Myradov, Aissa Houdjedj, Oznur Tastan, Hilal Kazan
Single-cell RNA sequencing (scRNA-seq) datasets generated across laboratories and experimental conditions often exhibit batch effects that obscure biological variation. Numerous computational methods have been developed to integrate such datasets, making robust benchmarking essential. Evaluation metrics play a central…
Matteo De Matola, Giorgio Arcara
Convolutional neural networks (CNNs) are a class of artificial neural networks (ANNs). Since the early 2010s, they have been widely adopted as models of primate vision and classifiers of neuroimaging data, becoming relevant for a wealth of neuroscientific fields. However, the majority of neuroscience researchers come…
William Fishell, Suraj Honnuraiah
Spiking neural networks (SNNs) have attracted growing interest for their ability to operate efficiently on low-power neuromorphic hardware, offering a biologically grounded route toward energy-efficient computation. However, despite advances in large-scale neuromorphic systems capable of simulating millions of spiking…
Zhiqian Zhai, Changhu Wang, Chengfeng Jiang, Ziqi Rong + 1 more
Integrating single-cell and spatial transcriptomics data across batches is essential for recovering comparable cell identities—including cell types, subtypes, and states—as a prerequisite for downstream analyses in multi-condition and large-scale studies. This task remains challenging because between-batch variation…