18 papers · ranked by Valyu relevance
Misha Denil, Nando de Freitas
It has recently been observed that certain extremely simple feature encoding techniques are able to achieve state of the art performance on several standard image classification benchmarks including deep belief networks, convolutional nets, factored RBMs, mcRBMs, convolutional RBMs, sparse autoencoders and several…
Bailey Kong, Charless C. Fowlkes
In this paper, we explore an efficient variant of convolutional sparse coding with unit norm code vectors where reconstruction quality is evaluated using an inner product (cosine distance). To use these codes for discriminative classification, we describe a model we term Energy-Based Spherical Sparse Coding (EB-SSC) in…
Walt Woods, Christof Teuscher
—Memristive crossbars have become a popular means for realizing unsupervised and supervised learning techniques. In previous neuromorphic architectures with leaky integrate-andfire neurons, the crossbar itself has been separated from the neuron capacitors to preserve mathematical rigor. In this work, we sought to…
Nicole L. Carlson, Vivienne L. Ming, Michael R. DeWeese
We have developed a sparse mathematical representation of speech that minimizes the number of active model neurons needed to represent typical speech sounds. The model learns several well-known acoustic features of speech such as harmonic stacks, formants, onsets and terminations, but we also find more exotic…
Peter Loxley
Sparse codes in neuroscience have been suggested to offer certain computational advantages over other neural representations of sensory data. To explore this viewpoint, a sparse code is used to represent natural images in an optimal control task solved with neuro-dynamic programming, and its computational properties…
Katrina Evtimova, Yann LeCun
Sparse representations of images are useful in many computer vision applications. Sparse coding with an l1 penalty and a learned linear dictionary requires regularization of the dictionary to prevent a collapse in the l1 norms of the codes. Typically, this regularization entails bounding the Euclidean norms of the…
Joshua Bowren
Neural networks, specifically deep convolutional neural networks, have achieved unprecedented performance in various computer vision tasks, but the rationale for the computations and structures of successful neural networks is not fully understood. Theories abound for the aptitude of convolutional neural networks for…
Joshua Bowren, Luis Gonzalo Sánchez-Giraldo, Odelia Schwartz
Sparse coding has been incorporated in models of the visual cortex for its computational advantages and connection to biology. But how the level of sparsity contributes to performance on visual tasks is not well understood. In this work, sparse coding has been integrated into an existing hierarchical V2 model (Hosoya…
Alireza Makhzani, Brendan J. Frey
Recently, it has been observed that when representations are learnt in a way that encourages sparsity, improved performance is obtained on classification tasks. These methods involve combinations of activation functions, sampling steps and different kinds of penalties. To investigate the effectiveness of sparsity by…
Maximilian Karl, Christian Osendorfer
A process centric view of robust PCA (RPCA) allows its fast approximate implementation based on a special form of a deep neural network with weights shared across all layers. However, empirically this fast approximation to RPCA fails to find representations that are parsemonious. We resolve these bad local minima by…
Laurent Perrinet
``` @inbook{Perrinet15sparse, Author = {Perrinet, Laurent U.}, Chapter = {Sparse models for Computer Vision}, Citeulike-Article-Id = {13514904}, Editor = {Crist{\'{o}}bal, Gabriel and Keil, Matthias S. and Perrinet, Laurent U.}, Keywords = {bicv-sparse, sanz12jnp, vacher14}, month = nov, chapter = {14}, Priority = {0}…
Andrew P. Magyar
Single unit recordings in the human medial temporal lobe (MTL) have revealed a population of cells with conceptually based, highly selective activity, indicating the presence of a sparse neural code. Building off previous work by the author and J.C. Collins, this paper develops a statistical model for analyzing this…
Laurent Perrinet
Natural images follow statistics inherited by the structure of our physical (visual) environment. In particular, a prominent facet of this structure is that images can be described by a relatively sparse number of features. We designed a sparse coding algorithm biologically-inspired by the architecture of the primary…
Hugo Ladret, Christian Casanova, Laurent Perrinet
Both biological and artificial neural networks inherently balance their performance with their operational cost, which balances their computational abilities. Typically, an efficient neuromorphic neural network is one that learns representations that reduce the redundancies and dimensionality of its input. This is for…
Vicente Botella‐Soler, Stéphane Deny, Olivier Marre, Gašper Tkačik
Retinal circuitry transforms spatiotemporal patterns of light into spiking activity of ganglion cells, which provide the sole visual input to the brain. Recent advances have led to a detailed characterization of retinal activity and stimulus encoding by large neural populations. The inverse problem of decoding, where…
Wilkie Olin-Ammentorp
It has been well-established that within conventional neural networks, many of the values produced at each layer are zero. In this work, I demonstrate that spiking neural networks can prevent the transmission of spikes representing values close to zero using local information. This can reduce the amount of energy…
Lingyun Ke, M. Hu
Encoding static images into spike trains is a crucial step for enabling Spiking Neural Networks (SNNs) to process visual information efficiently. However, existing schemes such as rate coding, Poisson encoding, and time-to-first-spike (TTFS) often ignore spatial relationships and yield temporally inconsistent spike…
Ahmad El Ferdaoussi, Éric Plourde, Jean Rouat
—The problem of spike encoding of sound consists in transforming a sound waveform into spikes. It is of interest in many domains, including the development of audio-based spiking neural networks, where it is the first and most crucial stage of processing. Many algorithms have been proposed to perform spike encoding of…