22 papers · ranked by Valyu relevance
Maliheh Miri, Mohammad Taghi Sadeghi, Vahid Abootalebi
Sparse representation of signals has achieved satisfactory results in classification applications compared to the conventional methods. Microarray data, which are obtained from monitoring the expression levels of thousands of genes simultaneously, have very high dimensions in relation to the small number of samples.…
Mohammad Abbasi, Connor Sanderford, Narendian Raghu, Benjamin B Bartelle
We have developed representation learning methods, specifically to address the constraints and advantages of complex spatial data. Sparse filtering (SFt), uses principles of sparsity and mutual information to build representations from both global and local features from a minimal list of samples. Critically, the…
Alicia Zeng, Jack Gallant
Encoding models based on word embeddings or artificial neural network (ANN) features reliably predict brain responses to naturalistic stimuli but remain difficult to interpret. A central limitation is superposition—the entanglement of distinct semantic features along correlated directions in dense embeddings, which…
Mohammad Abbasi, Connor R. Sanderford, Narendiran Raghu, Mirjeta Pasha + 2 more
'Mirjeta Pasha' 'Benjamin B. Bartelle' 'Gennady S. Cymbalyuk'] Unsupervised learning methods are commonly used to detect features within transcriptomic data and ultimately derive meaningful representations of biology. Contributions of individual genes to any feature however becomes convolved with each learning step…
Kavi Gupta, Osbert Bastani, Armando Solar-Lezama
Real-world processes often contain intermediate state that can be modeled as an extremely sparse tensor. We introduce SPARLING, a technique that allows you to learn models with intermediate layers that match this state from only end-to-end labeled examples (i.e., no supervision on the intermediate state). SPARLING uses…
Αλέξανδρος Γκίλλας, Dimitris Ampeliotis, Kostas Berberidis
—In this study, the problem of computing a sparse representation of multi-dimensional visual data is considered. In general, such data e.g., hyperspectral images, color images or video data consists of signals that exhibit strong local dependencies. A new computationally efficient sparse coding optimization problem is…
Mohammad Mahdi Jahani Yekta
We show, based on the following three grounds, that the primary visual cortex (V1) is a biological direct-shortcut deep residual learning neural network (ResNet) for sparse visual processing: (1) We first highlight that Gabor-like sets of basis functions, which are similar to the receptive fields of simple cells in the…
Costa, Valérie, Fel, Thomas + 6 more
Sparse autoencoders (SAEs) have recently become central tools for interpretability, leveraging dictionary learning principles to extract sparse, interpretable features from neural representations whose underlying structure is typically unknown. This paper evaluates SAEs in a controlled setting using MNIST, which…
Yuxiao Li, Eric J. Michaud, David D. Baek, Joshua Engels + 4 more
'Xiaoqing Sun' 'Max Tegmark' 'Michael L. Mayo' 'Kevin R. Pilkiewicz'] Sparse autoencoders have recently produced dictionaries of high-dimensional vectors corresponding to the universe of concepts represented by large language models. We find that this concept universe has interesting structure at three levels: (1) The…
Kyle Luther, H. Sebastian Seung
Sparse coding has been proposed as a theory of visual cortex and as an unsupervised algorithm for learning representations. We show empirically with the MNIST dataset that sparse codes can be very sensitive to image distortions, a behavior that may hinder invariant object recognition. A locally linear analysis suggests…
Shane K Chu, Gary D Stormo, Pier Luigi Martelli
A key distinction between traditional and more recent approaches to motif discovery is in how motifs are represented during optimization. Traditional methods, such as STREME and HOMER, typically use local representations, such as PWMs, for motif characterization during optimization. In contrast, recent approaches often…
Yanbo Lian, Anthony N. Burkitt
Sparse coding, predictive coding and divisive normalization have each been found to be principles that underlie the function of neural circuits in many parts of the brain, supported by substantial experimental evidence. However, the connections between these related principles are still poorly understood. In this…
Thomas Fel, Matthew Kowal, Mozes Jacobs, Dron Hazra + 21 more
What is the geometry of a visual percept? The most widely used protocols for decomposing neural network representations into interpretable parts treat concepts as isolated directions, yet recent work shows that concepts are often realized as geometric structures in low dimensional regions of activation space. We turn…
Madhuri Gadwal, Atul Negi, Venkata Rangarao Kaluri
To overcome difficulties in classifying large dimensionality data with a large number of classes, we propose a novel approach called JLSPCADL. This paper uses the Johnson-Lindenstrauss (JL) Lemma to select the dimensionality of a transformed space in which a discriminative dictionary can be learned for signal…
Sunil Kumar Prabhakar, Young-Gi Ju, Harikumar Rajaguru, Dong-Ok Won
In comparison to other biomedical signals, electroencephalography (EEG) signals are quite complex in nature, so it requires a versatile model for feature extraction and classification. The structural information that prevails in the originally featured matrix is usually lost when dealing with standard feature…
Sanjar Adilov
Machine learning models for molecular-property prediction typically work with molecular representations in the form of fingerprints, descriptors, or graphs. In case of fingerprints and descriptors, molecular representations usually comprise thousands of features, which causes the curse of dimensionality for many…
G. Madhuri, Atul Negi
Sparse Representation (SR) of signals or data has a well founded theory with rigorous mathematical error bounds and proofs. SR of a signal is given by superposition of very few columns of a matrix called Dictionary, implicitly reducing dimensionality. Training dictionaries such that they represent each class of signals…
Kelsey Hatzell, Yanjie Zheng
X-ray Computed Tomography (CT) is a non-invasive, non-destructive approach to imaging materials, material systems and engineered components in two- and three- dimensions. Acquisition of 3D images requires the collection of hundreds or thousands of through-thickness X-ray radiographic images from different angles. Such…
David Buterez, Jon Paul Janet, Steven Kiddle, Pietro Liò
We investigate the potential of graph neural networks for transfer learning and improving molecular property prediction on sparse and expensive to acquire high-fidelity data by leveraging low-fidelity measurements as an inexpensive proxy for a targeted property ofinterest. This problem arises in discovery processes…
Anthony Onwuli, Keith T. Butler, Aron Walsh
High-dimensional representations of the elements have become common within the field of materials informatics to build useful, structure-agnostic models for the chemistry of materials. However, the characteristics of elements change when they adopt a given oxidation state, with distinct structural preferences and…
Stamatia Zavitsanou, Zonghua Bo, Emanuele Casali, Matthew Langton + 1 more
Machine learning (ML) is currently transforming the field of chemistry by offering unparalleled efficiency in addressing complex challenges. Despite the progress made, a notable gap persists in the availability of user-friendly tools tailored to chemical problems involving small and sparse datasets. Here, we introduce…
Ping Yang, E. Adrian Henle, Cory M. Simon, Xiaoli Fern
Pesticides benefit agriculture by increasing crop yield, quality, and security. However, pesticides may inadvertently harm bees, which are valuable as pollinators. Thus, candidate pesticides in development pipelines must be assessed for toxicity to bees. Leveraging a data set of 382 molecules with toxicity labels from…