19 papers · ranked by Valyu relevance
Bryant, Dustin, Jim Woodcock, Simon J. Foster
This paper presents a formalized analysis of the sigmoid function and a fully mechanized proof of the Universal Approximation Theorem (UAT) in Isabelle/HOL, a higher-order logic theorem prover. The sigmoid function plays a fundamental role in neural networks; yet, its formal properties, such as differentiability…
Chintan Panchal, Ankur Changela, Mohendra Roy
Efficient hardware implementation of nonlinear activation functions is a crucial task in deploying artificial neural networks on resource-constrained and edge devices such as Field-Programmable Gate Arrays (FPGAs). The sigmoid activation function is widely used for probabilistic output, binary classification, and…
Andreas Skiadopoulos, Maria Knikou
Recruitment input-output curves of transspinal evoked potentials that represent the net spinal motor output of alpha motor neurons, and those of cortically, spinally or peripherally induced responses are used extensively in research as neurophysiological biomarkers to establish physiological or pathological behavior of…
Qian Feng, Zhihua Xia, Zhifeng Xu, Jiasi Weng + 1 more
Activation Functions in Recurrent Neural Network Authors: ['Qian Feng' 'Zhihua Xia' 'Zhifeng Xu' 'Jiasi Weng' 'Jian Weng'] Abstract—Deep neural network (DNN) typically involves convolutions, pooling, and activation function. Due to the growing concern about privacy, privacy-preserving DNN becomes a hot research topic.…
Samuel Chevalier, Duncan Starkenburg, Krishnamurthy Dvijotham
In the field of formal verification, Neural Networks (NNs) are typically reformulated into equivalent mathematical programs which are optimized over. To overcome the inherent non-convexity of these reformulations, convex relaxations of nonlinear activation functions are typically utilized. Common relaxations (i.e.…
Andreas Skiadopoulos, Maria Knikou, Elvan Wiyarta
Recruitment input-output curves of transspinal evoked potentials that represent the net output of spinal neuronal networks during which cortical, spinal and peripheral inputs are integrated as well as motor evoked potentials and H-reflexes are used extensively in research as neurophysiological biomarkers to establish…
Lisa Buche, Lauren G. Shoemaker, Lauren M. Hallett, Peter Vesk + 2 more
The ability of species to form diverse communities is not fully understood. Species are known to interact in various ways with their neighborhood. Despite this, common models of species coexistence assume that per capita interactions are constant and competitive, even as the environment changes. In this study, we…
Changsoo Shin
Modern AI systems excel at pattern recognition and task execution, but they often fall short of replicating the layered, self-referential structure of human thought that unfolds over time. In this paper, we present a mathematically grounded and conceptually simple framework based on smoothed step functions-sigmoid…
Hyenkyun Woo
This article presents a new polynomial parameterized sigmoid called SIGTRON, which is an extended asymmetric sigmoid with Perceptron, and its companion convex model called SIGTRON-imbalanced classification (SIC) model that employs a virtual SIGTRON-induced convex loss function. In contrast to the conventional…
Peter Buchwald, Sung-Ting Chuang, Brandon Watts, Oscar Alcazar
Insulin released in response to a stepwise increase in glucose (square wave) is biphasic with a transient first-phase peak of 5-10 min and a more sustained second phase. While it is usually assumed that the first phase is rate- and the second is concentration-dependent, there are no detailed investigations into the…
Siyao Huang, Yong Deng, Małgorzata Przybyła-Kasperek
Conflict management is crucial in information fusion. One of the efficient algorithms to address conflicting data fusion is discounting method. However, how to determine the discounting coefficient in conflict management remains an open issue. A heuristic method to determine discounting coefficient is presented based…
J. T. Barron
Power transforms, such as the Box-Cox transform [5] and Tukey's ladder of powers [3], are a fundamental tool in mathematics and statistics. These transforms are primarily used for normalizing and standardizing datasets, effectively by raising values to a power. In this work I present a novel power transform, and I show…
Andrew R Auty
The conventional interpretation^1^ of a logistic curve in biology is that it indicates the presence of an infection-like mechanism bounded by a saturation limit e.g. saturation occurs when the whole population has been infected. However, a logistic form may be generated by the fluctuating approach of biological systems…
Yuru Song, Marcus K. Benna
Cortical neurons often establish multiple synaptic contacts with the same postsynaptic neuron. To avoid functional redundancy of these parallel synapses, it is crucial that each synapse exhibits distinct computational properties. Here we model the current to the soma contributed by each synapse as a sigmoidal…
Batoul Saab, Jihad Fahs, Arij Daou
Conductance-based models of neuronal excitability depend critically on the mathematical form used to describe voltage-dependent ion channel gating. The classical Hodgkin-Huxley (HH) formalism employs empirically derived rate expressions fitted to squid giant axon data that are not readily transferable across cell types…
Yusuke Watanabe, Hiroshi Ban, Nobuhiro Hagura, Yuji Ikegaya
A neural network is a machine learning algorithm that can learn and make predictions by adjusting the strength of the connections between nodes. The sigmoid function is commonly used as an activation function in these nodes. This study explores the potential applicability of biological materials in the development of…
Lane Yoder, Sunder Ali Khowaja
The networks proposed here show how neurons can be connected to form flip-flops, the basic building blocks in sequential logic systems. The novel neural flip-flops (NFFs) are explicit, dynamic, and can generate known phenomena of short-term memory. For each network design, all neurons, connections, and types of…
Huixiong Chen, Qi Ye
In this paper, we use composite optimization algorithms to solve sigmoid networks. We equivalently transfer the sigmoid networks to a convex composite optimization and propose the composite optimization algorithms based on the linearized proximal algorithms and the alternating direction method of multipliers. Under the…
Bartłomiej Szczepan Olek, Lars Ehm
In the present study, various shapes of laboratory consolidation curves were numerically reproduced using a four-parametric sigmoid function. Sixteen consolidation curves were selected based on one-dimensional oedometer tests to statistically evaluate the sigmoid model and to determine the appropriate deviation…