22 papers · ranked by Valyu relevance
Heming Xia, Zhe Yang, Qingxiu Dong, Peiyi Wang + 5 more
'Tianyu Liu' 'Wenjie Li' 'Zhifang Sui'] To mitigate the high inference latency stemming from autoregressive decoding in Large Language Models (LLMs), Speculative Decoding has emerged as a novel decoding paradigm for LLM inference. In each decoding step, this method first drafts several future tokens efficiently and…
Kimonas Provatas, Aris Karatzikos, Charalampos Koilakos, Michail Patsakis + 5 more
Genomic and protein foundation models (GFMs and PFMs) have demonstrated strong performance in learning the language of DNA and proteins, but their use in large-scale sequence generation is limited by the latency of autoregressive decoding. Because every token triggers a forward pass of a large Transformer, whose…
Kimonas Provatas, Aris Karatzikos, Charalampos Koilakos, Michail Patsakis + 5 more
Genomic and protein foundation models (GFMs and PFMs) have demonstrated strong performance in learning the language of DNA and proteins, but their use in large-scale sequence generation is limited by the latency of autoregressive decoding. Because every token triggers a forward pass of a large Transformer, whose…
Minghao Yan, Saurabh Agarwal, Shivaram Venkataraman
Speculative Decoding is a widely used technique to speed up inference for Large Language Models (LLMs) without sacrificing quality. When performing inference, speculative decoding uses a smaller draft model to generate speculative tokens and then uses the target LLM to verify those draft tokens. The speedup provided by…
Hyun Ryu, Eric Kim
Decoding Authors: ['Hyun Ryu' 'Eric Kim'] Inference in Large Language Models (LLMs), such as those used in GPT-3 and LaMDA, has relied heavily on autoregressive decoding, which has yielded effective results. However, with LLMs growing in size and complexity, so has the need for improving inference efficiency. The…
Szymon Kobus, Deniz Gündüz
Speculative decoding accelerates large language model inference using a smaller draft model. In this paper, we establish a surprising connection between speculative decoding and channel simulation, which aims at simulating a noisy channel using as few bits as possible. This connection allows us to provide an…
Xiaoxuan Liu, Cade Daniel, Langxiang Hu, Woosuk Kwon + 6 more
Goodput Authors: ['Xiaoxuan Liu' 'Cade Daniel' 'Langxiang Hu' 'Woosuk Kwon' 'Zhuohan Li' 'Xiangxi Mo' 'Alvin Cheung' 'Zhijie Deng' 'Ion Stoica' 'Hao Zhang'] Reducing the inference latency of large language models (LLMs) is crucial, and speculative decoding (SD) stands out as one of the most effective techniques. Rather…
Hao Ding, Nannan Wu, Tianyi Qiu
DNA foundation models such as Evo2 7B adopt hybrid Hyena/attention architectures (Striped-Hyena2) whose single-stream autoregressive decoding is bounded by weight bandwidth at ∼45 tok/s. Speculative decoding on such hybrids faces a systems problem that prior SSM work solves only partially: after a draft is verified…
Harikrishna Narasimhan, Wittawat Jitkrittum, Ankit Singh Rawat, Seung‐Yeon Kim + 3 more
'Seung‐Yeon Kim' 'Neha Gupta' 'Aditya Krishna Menon' 'Sanjiv Kumar'] Cascades and speculative decoding are two common approaches to improving language models' inference efficiency. Both approaches involve interleaving models of different sizes, but via fundamentally distinct mechanisms: cascades employ a deferral rule…
Jingjian Li, Wei Wang, Hong Mo, Mengting Zhao + 3 more
'Derya Malak' 'Song-Nam Hong'] A distributed arithmetic coding algorithm based on source symbol purging and using the context model is proposed to solve the asymmetric Slepian-Wolf problem. The proposed scheme is to make better use of both the correlation between adjacent symbols in the source sequence and the…
Rithea Ngeth, Brian M. Kurkoski, Yuto Lim, Yasuo Tan
This paper investigates the design of overlapped chunked codes (OCC) for multi-source multi-relay networks where a physical-layer network coding approach, compute-and-forward (CF) based on nested lattice codes (NLC), is applied for the simultaneous transmissions from the sources to the relays. This code is called…
Ethan M. Meyers
Neural decoding is a powerful method to analyze neural activity. However, the code needed to run a decoding analysis can be complex, which can present a barrier to using the method. In this paper we introduce a package that makes it easy to perform decoding analyses in the R programing language. We describe how the…
Cameron Higgins, Mats W.J. van Es, Andrew Quinn, Diego Vidaurre + 1 more
Decoding of high temporal resolution, stimulus-evoked neurophysiological data is increasingly used to test theories about how the brain processes information. However, a fundamental relationship between the frequency spectra of the neural signal and the subsequent decoding accuracy timecourse is not widely recognised.…
David Kracht, Steffen Schober
Background Barcode multiplexing is a key strategy for sharing the rising capacity of next-generation sequencing devices: Synthetic DNA tags, called barcodes, are attached to natural DNA fragments within the library preparation procedure. Different libraries, can individually be labeled with barcodes for a joint…
Xiumin Wang, Jinlong He, Jun Li, Liang Shan
A traditional successive cancellation (SC) decoding algorithm produces error propagation in the decoding process. In order to improve the SC decoding performance, it is important to solve the error propagation. In this paper, we propose a new algorithm combining reinforcement learning and SC flip (SCF) decoding of…
Mingkang Li, Ruixue Wang, Guihua Wan, Yuqi Yang + 1 more
Calcium imaging has gained extensive application in neural decoding tasks because of its high precision in observing cortical neural activity. Nevertheless, the immense data volume and complexity of automated signal extraction algorithms in calcium imaging result in significant delays in extracting neuronal calcium…
Niklas Gassner, Julia Lieb, Abhinaba Mazumder, Michael Schaller
In this paper, we present a framework for generic decoding of convolutional codes, which allows us to do cryptanalysis of code-based systems that use convolutional codes as public keys. We then apply this framework to information set decoding, study success probabilities and give tools to choose variables. Finally, we…
Jia Lu, Ryan Tsoi, Nan Luo, Yuanchi Ha + 8 more
Dynamical systems often generate distinct outputs according to different initial conditions, and one can infer the corresponding input configuration given an output. This property captures the essence of information encoding and decoding. Here, we demonstrate the use of self-organized patterns, combined with machine…
Eleni Litsa, Vijil Chenthamarakshan, Payel Das, Lydia Kavraki
Elucidating the structure of a chemical compound is a fundamental task in chemistry with application in multiple domains including the emerging field of metabolomics, with promising applications in drug discovery, precision medicine, and biomarker discovery. The common practice for elucidating the structure of a…
Islam S. Badreldin, Karim G. Oweiss
Brain-machine interfaces rely on extracting motor control signals from brain activity in real time to actuate external devices such as robotic limbs. Whereas biomimetic approaches to neural decoding use motor imagery/observation signals, non-biomimetic approaches assign an arbirary transformation that maps neural…
David M. Benton, You Yang
Concurrent coding is an encoding scheme with ‘holographic’ type properties that are shown here to be robust against a significant amount of noise and signal loss. This single encoding scheme is able to correct for random errors and burst errors simultaneously, but does not rely on cyclic codes. A simple and practical…
Authors not listed
Structure elucidation --- determining molecular structures from spectroscopic data -- remains one of chemistry's most fundamental and challenging tasks, essential for advancing fields from drug discovery to materials science. While machine learning approaches have attempted to automate this process, they typically…