22 papers · ranked by Valyu relevance
Emir Öztürk, Altan Mesut, Stefano Cirillo
Learning-based data compression methods have gained significant attention in recent years. Although these methods achieve higher compression ratios compared to traditional techniques, their slow processing times make them less suitable for compressing large datasets, and they are generally more effective for short…
Zhaoyuan Su, Ammar Ahmed, Zirui Wang, Ali Anwar + 1 more
Pre-Trained ML Models but Were Afraid to Ask Authors: ['Zhaoyuan Su' 'Ammar Ahmed' 'Zirui Wang' 'Ali Anwar' 'Yue Cheng'] As the number of pre-trained machine learning (ML) models is growing exponentially, data reduction tools are not catching up. Existing data reduction techniques are not specifically designed for…
Ziguang Li, Chao Huang, Xuliang Wang, Haibo Hu + 6 more
'Dongbo Bu' 'Quan Yu' 'Wen Gao' 'Xingwu Liu' 'Ming Li'] The LLMs may be seen to approximate the uncomputable Solomonoff induction. Therefore, under this new uncomputable paradigm, we present LMCompress. LMCompress shatters all previous lossless compression algorithms, doubling the lossless compression ratios of JPEG-XL…
Marek Kokot, Amitava Roy, Travis J Wheeler, Sebastian Deorowicz
Molecular dynamics (MD) simulations model the physical movements of atoms in biomolecular systems over time, providing atomic-resolution insight into conformational changes, binding events, and dynamic behaviors that cannot be captured by static structures alone. As such, MD simulations are playing an increasingly…
Oren Rippel, Lubomir Bourdev
Our algorithm typically produces files 2.5 times smaller than JPEG and JPEG 2000, 2 times smaller than WebP, and 1.7 times smaller than BPG on datasets of generic images across all quality levels. At the same time, our codec is designed to be lightweight and deployable: for example, it can encode or decode the Kodak…
Robert Underwood, Jon C. Calhoun, Sheng Di, Franck Cappello
Training Sets Authors: ['Robert Underwood' 'Jon C. Calhoun' 'Sheng Di' 'Franck Cappello'] Abstract—Machine Learning and Artificial Intelligence (ML/AI) techniques have become increasingly prevalent in high performance computing (HPC). However, these methods depend on vast volumes of floating point data for training and…
Sonain Jamil, Md. Jalil Piran, MuhibUrRahman
—In the realm of image processing and computer vision (CV), machine learning (ML) architectures are widely applied. Convolutional neural networks (CNNs) solve a wide range of image processing issues and can solve image compression problem. Compression of images is necessary due to bandwidth and memory constraints.…
Hyunbin Kim, Milot Mirdita, Martin Steinegger
Highly accurate protein structure predictors have generated hundreds of millions of protein structures; these pose a challenge in terms of storage and processing. Here we present Foldcomp, a novel lossy structure compression algorithm and indexing system to address this challenge. By using a combination of internal and…
Herbert J. Bernstein, Alexei S. Soares, Kimberly Horvat, Jean Jakoncic
New higher-count-rate, integrating, large area X-ray detectors with framing rates as high as 17,400 images per second are beginning to be available. These will soon be used for specialized MX experiments but will require optimal lossy compression algorithms to enable systems to keep up with data throughput. Some…
Huabin Diao, Yuexing Hao, Shaoyun Xu, Gongyan Li + 1 more
Convolutional neural networks (CNNs) have achieved significant breakthroughs in various domains, such as natural language processing (NLP), and computer vision. However, performance improvement is often accompanied by large model size and computation costs, which make it not suitable for resource-constrained devices.…
Yicong Li, Core Francisco Park, Daniel Xenes, Caitlyn Bishop + 7 more
The ongoing pursuit to map detailed brain structures at high resolution using electron microscopy (EM) has led to advancements in imaging that enable the generation of connectomic volumes that have reached the petabyte scale and are soon expected to reach the exascale for whole mouse brain collections. To tackle the…
Authors not listed
Efficient lossless compression is essential for minimizing storage costs and transmission overhead while preserving data integrity. Traditional compression techniques, such as dictionary-based and statistical methods, often struggle to optimally exploit the structure and redundancy in complex data formats. Recent…
David Podgorelec, Damjan Strnad, Ivana Kolingerová, Borut Žalik + 1 more
'Jun Chen'] After a boom that coincided with the advent of the internet, digital cameras, digital video and audio storage and playback devices, the research on data compression has rested on its laurels for a quarter of a century. Domain-dependent lossy algorithms of the time, such as JPEG, AVC, MP3 and others…
Authors not listed
Chemical reactions in solution are central to biological function, synthetic chemistry, and materials design. Accurate modeling of these systems is essential for obtaining mechanistic insights, but remains computationally demanding. Hybrid machine-learned/molecular mechanics (ML/MM) simulations offer a promising…
Yuhang Dong, W. David Pan, Dongsheng Wu
Malaria is a severe public health problem worldwide, with some developing countries being most affected. Reliable remote diagnosis of malaria infection will benefit from efficient compression of high-resolution microscopic images. This paper addresses a lossless compression of malaria-infected red blood cell images…
Yong-Yeon Jo, Young Sang Choi, Hyun Woo Park, Jae Hyeok Lee + 6 more
'Hyojung Jung' 'Hyo-Eun Kim' 'Kyounglan Ko' 'Chan Wha Lee' 'Hyo Soung Cha' 'Yul Hwangbo'] Image compression is used in several clinical organizations to help address the overhead associated with medical imaging. These methods reduce file size by using a compact representation of the original image. This study aimed to…
Miaoshan Lu, Junjie Tong, Ruimin Wang, Shaowei An + 2 more
Mass spectrum (MS) data volumes increase with an improved ion acquisition ratio and a highly accurate mass spectrometer. However, the most widely used data format, mzML, does not take advantage of compression methods and improved read performances. Several compression algorithms have been proposed in recent years, and…
Felix Zeller, Chieh-Min Hsieh, Wilke Dononelli, Tim Neudecker
The field of liquid-phase and solid-state high-pressure chemistry has exploded since the advent of the diamond anvil cell, an experimental technique that allows the application of pressures up to several hundred gigapascal. To complement high-pressure experiments, a large number of computational tools have been…
Gregory P. Way, Michael Zietz, Daniel S. Himmelstein, Casey S. Greene
Unsupervised machine learning algorithms applied to gene expression data extract latent, or hidden, signals representing technical and biological sources of variation. However, these algorithms require a user to select a biologically-appropriate latent dimensionality. We compressed gene expression data from three large…
Matthew Witman, Sanliang Ling, Matthew Wadge, Anis Bouzidi + 12 more
The ability to rapidly screen material performance in the vast space of compositionally complex (high entropy) alloys is of critical importance to efficiently identify optimal hydride candidates for various use cases. Given the prohibitive complexity of first principles simulations and large-scale sampling required to…
Yuhang Dong, W. David Pan, Jonathan Wu, Thangarajah Akilan + 2 more
'Jitendra Kumar' 'Chengsheng Yuan'] Digital images are usually stored in compressed format. However, image classification typically takes decompressed images as inputs rather than compressed images. Therefore, performing image classification directly in the compression domain will eliminate the need for decompression…
Chen-Hsiu Huang, Ja-Ling Wu, Jun Chen
End-to-end learned image compression codecs have notably emerged in recent years. These codecs have demonstrated superiority over conventional methods, showcasing remarkable flexibility and adaptability across diverse data domains while supporting new distortion losses. Despite challenges such as computational…