18 papers · ranked by Valyu relevance
Yang Liu, Wentao Feng, Zhuoyao Liu, Shudong Huang + 1 more
Enabling Visual Semantic Models to effectively handle multi-view description matching has been a longstanding challenge. Existing methods typically learn a set of embeddings to find the optimal match for each view's text and compute similarity. However, the visual and text embeddings learned through these approaches…
Wenpeng Hu, Mengyu Wang, Bing Liu, Feng Ji + 4 more
'Dongyan Zhao' 'Jinwen Ma' 'Rui Yan'] Sparsity is regarded as a desirable property of representations, especially in terms of explanation. However, its usage has been limited due to the gap with dense representations. Most NLP research progresses in recent years are based on dense representations. Thus the desirable…
Liping Yuan, Jiehang Zeng, Xiaoqing Zheng
It is still a challenging task to learn a neural text generation model under the framework of generative adversarial networks (GANs) since the entire training process is not differentiable. The existing training strategies either suffer from unreliable gradient estimations or imprecise sentence representations.…
V. E. Prokhorov, Yingzhen Li, Ehsan Shareghi, Nigel Collier
It has been long known that sparsity is an effective inductive bias for learning efficient representation of data in vectors with fixed dimensionality, and it has been explored in many areas of representation learning. Of particular interest to this work is the investigation of the sparsity within the VAE framework…
Simon Van de Vyver, Tibo Vande Moortele, Peter Dawyndt, Bart Mesuere + 1 more
Pattern matching is a fundamental challenge in bioinformatics, especially in the fields of genomics, transcriptomics and proteomics. Efficient indexing structures, such as suffix arrays, are critical for searching large datasets. While sparse suffix arrays offer significant memory savings compared to full suffix…
Gunther Eysenbach, Amirabbas Azizi, Eradah Hamad, Simon Lin + 5 more
Background It is difficult to synthesize the vast amount of textual data available from social media websites. Capturing real-world discussions via social media could provide insights into individuals’ opinions and the decision-making process. Objective We conducted a sequential mixed methods study to determine the…
Simon Van de Vyver, Tibo Vande Moortele, Peter Dawyndt, Bart Mesuere + 1 more
Background Pattern matching is a fundamental challenge in bioinformatics, especially in the fields of genomics, transcriptomics and proteomics. Efficient indexing structures, such as suffix arrays, are critical for searching large datasets. A sparse suffix array (SSA) retains only suffixes at every k-th position in the…
Alpo HONKAPOHJA, Jacob Thaisen, Anders Nøklestad
Non-standardised early vernaculars present a problem for search tools due to the high degree of variation. The challenge lies in the variation found in orthography, syntax, and lexicon between titles, incipits, and explicits in manuscript copies of the same work. Traditional search methods relying on exact string…
Siddhartha Brahma, Polina Zablotskaia, David Mimno
Transformers allow attention between all pairs of tokens, but there is reason to believe that most of these connections—and their quadratic time and memory—may not be necessary. But which ones? We evaluate the impact of sparsification patterns with a series of ablation experiments. First, we compare masks based on…
Yuxiao Li, Eric J. Michaud, David D. Baek, Joshua Engels + 4 more
'Xiaoqing Sun' 'Max Tegmark' 'Michael L. Mayo' 'Kevin R. Pilkiewicz'] Sparse autoencoders have recently produced dictionaries of high-dimensional vectors corresponding to the universe of concepts represented by large language models. We find that this concept universe has interesting structure at three levels: (1) The…
Rongbo Chen, Haojun Sun, Lifei Chen, Jianfei Zhang + 1 more
Markov models are extensively used for categorical sequence clustering and classification due to their inherent ability to capture complex chronological dependencies hidden in sequential data. Existing Markov models are based on an implicit assumption that the probability of the next state depends on the preceding…
Alexey V. Orlov, Yulia V. Makus, German A. Ashniev, Natalia N. Orlova + 1 more
Foundation models trained on protein and DNA sequences are increasingly deployed for variant interpretation, drug design, and gene regulation prediction, yet their internal representations remain opaque – limiting both biological insight and trust in model-guided decisions. Existing interpretation approaches establish…
Jingyao Li, Dongdong Lin, Hongbao Cao, Yu-Ping Wang
Background Multicolour Fluorescence In-Situ Hybridization (M-FISH) images are employed for detecting chromosomal abnormalities such as chromosomal translocations, deletions, duplication and inversions. This technique uses mixed colours of fluorochromes to paint the whole chromosomes for rapid detection of chromosome…
Chao Lou, Zixia Jia, Zilong Zheng, Kewei Tu
Long-Range Transformers Authors: ['Chao Lou' 'Zixia Jia' 'Zilong Zheng' 'Kewei Tu'] Accommodating long sequences efficiently in autoregressive Transformers, especially within an extended context window, poses significant challenges due to the quadratic computational complexity and substantial KV memory requirements…
Ko Sugawara
Deep learning is becoming more prominent in cell image analysis. However, collecting the annotated data required to train efficient deep-learning models remains a major obstacle. I demonstrate that functional performance can be achieved even with sparsely annotated data. Furthermore, I show that the selection of sparse…
Xingyu Liu, Zonglei Zhen, Jia Liu
Recently, deep convolutional neural networks (DCNNs) have attained human-level performances on challenging object recognition tasks owing to their complex internal representation. However, it remains unclear how objects are represented in DCNNs with an overwhelming number of features and non-linear operations. In…
Hasan M. Sayeed, Sterling G. Baird, Taylor D. Sparks
Capturing structure-property relationships of materials for property prediction using machine learning requires the representation or featurization of the structural aspects of materials at different levels, including atomic, crystal, and microscales. While crystal structure-based modeling techniques are effective for…
Kelsey Hatzell, Yanjie Zheng
X-ray Computed Tomography (CT) is a non-invasive, non-destructive approach to imaging materials, material systems and engineered components in two- and three- dimensions. Acquisition of 3D images requires the collection of hundreds or thousands of through-thickness X-ray radiographic images from different angles. Such…