Search · four archives

Results for parameter-efficient fine-tuning

or ask it as a questionAsk the stacks →

ArchivesarXiv PubMed bioRxiv chemRxiv

WhenAny time Past month Past year Past 5 years

OrderMost relevant Most cited Newest

28 papers · ranked by Valyu relevance

01
PPubMed17 Nov 2025
Parameter-efficient fine-tuning for low-resource text classification: a comparative study of LoRA, IA 3 , and ReFT
Steve Nwaiwu
The successful application of large-scale transformer models in Natural Language Processing (NLP) is often hindered by the substantial computational cost and data requirements of full fine-tuning. This challenge is particularly acute in low-resource settings, where standard fine-tuning can lead to catastrophic…
92relevance
02
BbioRxiv30 May 2025
Parameter-Efficient Fine-Tuning of a Supervised Regulatory Sequence Model
Han Yuan, Johannes Linder, David R Kelley
DNA sequence deep learning models accurately predict epigenetic and transcriptional profiles, enabling analysis of gene regulation and genetic variant effects. While large-scale training models like Enformer and Borzoi are trained on abundant data, they cannot cover all cell states and assays, necessitating new model…
92relevance
03
PPubMed3 May 2024
Comparison between parameter-efficient techniques and full fine-tuning: A case study on multilingual news article classification Comparison between parameter-efficient techniques and full fine-tuning
Olesya Razuvayevskaya, Ben Wu, João A. Leite, Freddy Heppell + 5 more
'Ivan Srba' 'Carolina Scarton' 'Kalina Bontcheva' 'Xingyi Song' 'Mario Graff-Guerrero'] Adapters and Low-Rank Adaptation (LoRA) are parameter-efficient fine-tuning techniques designed to make the training of language models more efficient. Previous results demonstrated that these methods can even improve performance on…
92relevance
04
PPubMed9 Feb 2026
InfoMSD: an information-maximization self-distillation framework for parameter-efficient fine-tuning on artwork images
Feng Guan, Hao Hong, Yong Wang
In recent years, despite the remarkable performance of large-scale vision language models across various visual classification tasks, their substantial parameter counts and high fine-tuning costs have hindered deployment in resource-constrained cultural and artwork settings. This work specifically addresses the task of…
91relevance
05
PPubMed28 Dec 2024
Parameter-efficient fine-tuning of large language models using semantic knowledge tuning
Nusrat Jahan Prottasha, Asif Mahmud, Md. Shohanur Islam Sobuj, Prakash Bhat + 3 more
'Prakash Bhat' 'Md Kowsher' 'Niloofar Yousefi' 'Ozlem Ozmen Garibay'] Large Language Models (LLMs) are gaining significant popularity in recent years for specialized tasks using prompts due to their low computational cost. Standard methods like prefix tuning utilize special, modifiable tokens that lack semantic meaning…
91relevance
06
AarXiv2022
Efficient Methods for Natural Language Processing: A Survey
Marcos Treviso, Tianchu Ji, Ji-Ung Lee, Betty van Aken + 14 more
'Qingqing Cao' 'Manuel R. Ciosici' 'Michael Hassid' 'Kenneth Heafield' 'Sara Hooker' 'Pedro H. Martins' 'André F. T. Martins' 'Peter Milder' 'Colin Raffel' 'Edwin Simpson' 'Noam Slonim' 'Niranjan Balasubramanian' 'Leon Derczynski' 'Roy Schwartz'] Recent work in natural language processing (NLP) has yielded appealing…
90relevance
07
AarXiv18 May 2025
Adaptive parameter-efficient fine-tuning via Hessian-informed subset selection
Shiyun Xu, Zhiqi Bu
Parameter-efficient fine-tuning (PEFT) is a highly effective approach for adapting large pre-trained models to downstream tasks with minimal computational overhead. At the core, PEFT methods freeze most parameters and only trains a small subset (say < 0.1% of total parameters). Notably, different PEFT methods select…
90relevance
08
AarXiv2023
DePT: Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning
Zhengxiang Shi, Aldo Lipani
Prompt tuning (PT), where a small amount of trainable soft (continuous) prompt vectors is affixed to the model input, has shown promising results across various tasks and model architecture for parameter-efficient fine-tuning (PEFT). PT stands out from other PEFT approaches because it maintains competitive performance…
90relevance
09
BbioRxiv20 Feb 2025
Parameter Efficient Fine-tuning of Transformer-based Masked Autoencoder Enhances Resource Constrained Neuroimage Analysis
Nikhil J. Dhinagar, Saket S. Ozarkar, Ketaki U. Buwa, Sophia I. Thomopoulos + 10 more
Recent innovations in artificial intelligence (AI) have increasingly focused on large-scale foundational models that are more general purpose in contrast to conventional models trained to perform specialized tasks. Transformer-based architectures have become the standard backbone in foundation models across data…
90relevance
10
BbioRxiv10 Nov 2023
Democratizing Protein Language Models with Parameter-Efficient Fine-Tuning
Samuel Sledzieski, Meghana Kshirsagar, Minkyung Baek, Bonnie Berger + 2 more
Proteomics has been revolutionized by large pre-trained protein language models, which learn unsupervised representations from large corpora of sequences. The parameters of these models are then fine-tuned in a supervised setting to tailor the model to a specific downstream task. However, as model size increases, the…
90relevance
11
AarXiv2022
On the Effectiveness of Parameter-Efficient Fine-Tuning
Zihao Fu, Haoran Yang, Anthony Man–Cho So, Wai Lam + 2 more
'Nigel Collier'] Fine-tuning pre-trained models has been ubiquitously proven to be effective in a wide range of NLP tasks. However, fine-tuning the whole model is parameter inefficient as it always yields an entirely new model for each task. Currently, many research works propose to only fine-tune a small portion of…
90relevance
12
PPubMed8 Jul 2026
Sample-efficient fine-tuning with textual prompts for time series forecasting Fine-tuning with textual prompts for time series forecasting
Kaibin Wei, Jianqiang Jing, Jiawei Liu, Qing Liu + 2 more
Time series forecasting models often face challenges in cross-domain fine-tuning, such as high training costs and limited adaptability. To address these issues, we propose a Cue-driven Feature Fusion Network (CFF-Net), which combines semantic cues from textual prompts with numerical time series features for…
90relevance
13
PPubMed8 Jun 2026
Parameter-Efficient Fine-Tuning for Photovoltaic Cell Defect Classification: A Systematic Comparison of LoRA, QLoRA, and Full Fine-Tuning on ConvNeXt-Tiny
Seda Bayat Toksöz, Gültekin Işık, Gökhan Şahin, Erdal Akin + 2 more
Automated visual inspection of photovoltaic (PV) cells is an important component of solar-module quality assurance. However, adapting modern pre-trained vision backbones to PV defect classification remains challenging because full fine-tuning requires substantial memory, naturally imbalanced datasets can reduce…
90relevance
14
BbioRxiv5 Nov 2023
PEFT-SP: Parameter-Efficient Fine-Tuning on Large Protein Language Models Improves Signal Peptide Prediction
Shuai Zeng, Duolin Wang, Dong Xu
Signal peptides (SP) play a crucial role in protein translocation in cells. The development of large protein language models (PLMs) provides a new opportunity for SP prediction, especially for the categories with limited annotated data. We present a Parameter-Efficient Fine-Tuning (PEFT) framework for SP prediction…
88relevance
15
AarXiv2022
Efficient Fine-Tuning of Compressed Language Models with Learners
Danilo Vucetic, Mohammadreza Tayaranian, Maryam Ziaeefard, James J. Clark + 2 more
'James J. Clark' 'Brett H. Meyer' 'Warren J. Gross'] Fine-tuning BERT-based models is resourceintensive in memory, computation, and time. While many prior works aim to improve inference efficiency via compression techniques, e.g., pruning, these works do not explicitly address the computational challenges of training…
88relevance
16
AarXiv31 Jan 2025
Sparse Gradient Compression for Fine-Tuning Large Language Models
Dong Yang, M. Amiri, Tejaswini Pedapati, Subhajit Chaudhury + 1 more
'Pin‐Yu Chen'] Fine-tuning large language models (LLMs) for downstream tasks has become increasingly crucial due to their widespread use and the growing availability of open-source models. However, the high memory costs associated with fine-tuning remain a significant challenge, especially as models increase in size.…
86relevance
17
CchemRxiv10 Jul 2025
LifeLong Learning for Large Language Models in Predicting Chemical Reaction Yields
Authors not listed
Large Language Models (LLMs) based on transformer architectures excel at internet-scale tasks. However, real-world scientific scenarios—such as synthetic chemistry laboratories and autonomous experimental setups—typically involve incremental data generation in batches as new chemical reactions are conducted, unlike…
86relevance
18
AarXiv2 Jul 2025
LoRA Fine-Tuning Without GPUs: A CPU-Efficient Meta-Generation Framework for LLMs
Reza Arabpour, Haitz Sáez de Ocáriz Borde, Anastasis Kratsios
Low-Rank Adapters (LoRAs) have transformed the fine-tuning of Large Language Models (LLMs) by enabling parameter-efficient updates. However, their widespread adoption remains limited by the reliance on GPU-based training. In this work, we propose a theoretically grounded approach to LoRA fine-tuning designed…
85relevance
19
BbioRxiv11 Apr 2025
Ontology-based Protein-Protein Interaction Explanation Using Large Language Models
Nur Bengisu Çam, Hasin Rehana, Jie Zheng, Benu Bansal + 3 more
Protein-protein interactions (PPIs) play a crucial role in various biological processes, and understanding these interactions is essential for advancing biomedical research. Automated extraction and analysis of PPI information from the rapidly growing scientific literature remains an important challenge. We present a…
85relevance
20
PPubMed3 Sep 2025
Efficient inference, training, and fine-tuning of protein language models
Muhammed Hasan Çelik, Xiaohui Xie
Title: Summary Protein language models (PLMs) have shown great promise in protein structure and function predictions, but their adoption is limited by computational cost. We address this challenge by enhancing the efficiency of evolutionary scale modeling (ESM). Using FlashAttention and sequence packing, we achieve…
85relevance
21
BbioRxiv8 Jan 2025
Fine-tuning molecular mechanics force fields to experimental free energy measurements
Dominic Rufa, Joshua Fass, John D. Chodera
Alchemical free energy methods using molecular mechanics (MM) force fields are essential tools for predicting thermodynamic properties of small molecules, especially via free energy calculations that can estimate quantities relevant for drug discovery such as affinities, selectivities, the impact of target mutations…
83relevance
22
BbioRxiv14 Dec 2023
Fine-tuning protein language models boosts predictions across diverse tasks
Robert Schmirler, Michael Heinzinger, Burkhard Rost
Prediction methods inputting embeddings from protein Language Models (pLMs) have reached or even surpassed state-of-the-art (SOTA) performance on many protein prediction tasks. In natural language processing (NLP) fine-tuning large Language Models (LLMs) has become the de facto standard. In contrast, most pLM-based…
79relevance
23
CchemRxiv3 Nov 2025
CrysText: A Generative AI Approach for Text-Conditioned Crystal Structure Generation using LLM
Authors not listed
The ability to generate crystal structures directly from textual descriptions marks a pivotal advancement in materials informatics and underscores the emerging role of large language models (LLMs) in inverse design. In this work, we introduce CrysText, a text-conditioned framework that generates crystal structures in…
78relevance
24
AarXiv18 Jul 2024
Crafting Efficient Fine-Tuning Strategies for Large Language Models
Michael Oliver, G. Wang
This paper addresses the challenges of efficiently fine-tuning large language models (LLMs) by exploring data efficiency and hyperparameter optimization. We investigate the minimum data required for effective fine-tuning and propose a novel hyperparameter optimization method that leverages early-stage model…
76relevance
25
CchemRxiv18 May 2022
Multi-fidelity machine learning models for improved high-throughput screening predictions
David Buterez, Jon Paul Janet, Steven Kiddle, Pietro Liò
We investigate the potential of graph neural networks for transfer learning and improving molecular property prediction on sparse and expensive to acquire high-fidelity data by leveraging low-fidelity measurements as an inexpensive proxy for a targeted property ofinterest. This problem arises in discovery processes…
73relevance
26
CchemRxiv22 Dec 2025
The Impact of Reward Scalarization and Weight Scheduling on Optimization Dynamics in Multi-Objective Molecular Design
Authors not listed
Inverse molecular design aims to generate novel chemical structures that satisfy multiple property constraints, yet reinforcement-learning (RL) fine-tuning can be sensitive to how objectives are converted into a scalar reward. Here, we systematically analyze how scalarization choices and stabilization mechanisms shape…
65relevance
27
CchemRxiv16 May 2022
Neural Scaling of Deep Chemical Models
Nathan Frey, Ryan Soklaski, Simon Axelrod, Siddharth Samsi + 3 more
Massive scale, both in terms of data availability and computation, enables significant breakthroughs in key application areas of deep learning such as natural language processing (NLP) and computer vision. There is emerging evidence that scale may be a key ingredient in scientific deep learning, but the importance of…
63relevance
28
CchemRxiv6 Sep 2022
The most compact search space is not always the most efficient: A case study on maximizing solid rocket fuel packing fraction via constrained Bayesian optimization
Sterling Baird, Jason R. Hall, Taylor D. Sparks
Would you rather search for a line inside a cube or a point inside a square? This type of solution degeneracy often exists in physics-based simulations and wet-lab experiments, but constraining these degeneracies is often unsupported or difficult to implement in many optimization packages, requiring additional time and…
55relevance

Results for parameter-efficient fine-tuning

Parameter-efficient fine-tuning for low-resource text classification: a comparative study of LoRA, IA 3 , and ReFT

Parameter-Efficient Fine-Tuning of a Supervised Regulatory Sequence Model

Comparison between parameter-efficient techniques and full fine-tuning: A case study on multilingual news article classification Comparison between parameter-efficient techniques and full fine-tuning

InfoMSD: an information-maximization self-distillation framework for parameter-efficient fine-tuning on artwork images

Parameter-efficient fine-tuning of large language models using semantic knowledge tuning

Efficient Methods for Natural Language Processing: A Survey

Adaptive parameter-efficient fine-tuning via Hessian-informed subset selection

DePT: Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning

Parameter Efficient Fine-tuning of Transformer-based Masked Autoencoder Enhances Resource Constrained Neuroimage Analysis

Democratizing Protein Language Models with Parameter-Efficient Fine-Tuning

On the Effectiveness of Parameter-Efficient Fine-Tuning

Sample-efficient fine-tuning with textual prompts for time series forecasting Fine-tuning with textual prompts for time series forecasting

Parameter-Efficient Fine-Tuning for Photovoltaic Cell Defect Classification: A Systematic Comparison of LoRA, QLoRA, and Full Fine-Tuning on ConvNeXt-Tiny

PEFT-SP: Parameter-Efficient Fine-Tuning on Large Protein Language Models Improves Signal Peptide Prediction

Efficient Fine-Tuning of Compressed Language Models with Learners

Sparse Gradient Compression for Fine-Tuning Large Language Models

LifeLong Learning for Large Language Models in Predicting Chemical Reaction Yields

LoRA Fine-Tuning Without GPUs: A CPU-Efficient Meta-Generation Framework for LLMs

Ontology-based Protein-Protein Interaction Explanation Using Large Language Models

Efficient inference, training, and fine-tuning of protein language models

Fine-tuning molecular mechanics force fields to experimental free energy measurements

Fine-tuning protein language models boosts predictions across diverse tasks

CrysText: A Generative AI Approach for Text-Conditioned Crystal Structure Generation using LLM

Crafting Efficient Fine-Tuning Strategies for Large Language Models

Multi-fidelity machine learning models for improved high-throughput screening predictions

The Impact of Reward Scalarization and Weight Scheduling on Optimization Dynamics in Multi-Objective Molecular Design

Neural Scaling of Deep Chemical Models

The most compact search space is not always the most efficient: A case study on maximizing solid rocket fuel packing fraction via constrained Bayesian optimization

Parameter-efficient fine-tuning for low-resource text classification: a comparative study of LoRA, IA 3 , and ReFT

Parameter-Efficient Fine-Tuning of a Supervised Regulatory Sequence Model

Comparison between parameter-efficient techniques and full fine-tuning: A case study on multilingual news article classification Comparison between parameter-efficient techniques and full fine-tuning

InfoMSD: an information-maximization self-distillation framework for parameter-efficient fine-tuning on artwork images

Parameter-efficient fine-tuning of large language models using semantic knowledge tuning

Efficient Methods for Natural Language Processing: A Survey

Adaptive parameter-efficient fine-tuning via Hessian-informed subset selection

DePT: Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning

Parameter Efficient Fine-tuning of Transformer-based Masked Autoencoder Enhances Resource Constrained Neuroimage Analysis

Democratizing Protein Language Models with Parameter-Efficient Fine-Tuning

On the Effectiveness of Parameter-Efficient Fine-Tuning

Sample-efficient fine-tuning with textual prompts for time series forecasting Fine-tuning with textual prompts for time series forecasting

Parameter-Efficient Fine-Tuning for Photovoltaic Cell Defect Classification: A Systematic Comparison of LoRA, QLoRA, and Full Fine-Tuning on ConvNeXt-Tiny

PEFT-SP: Parameter-Efficient Fine-Tuning on Large Protein Language Models Improves Signal Peptide Prediction

Efficient Fine-Tuning of Compressed Language Models with Learners

Sparse Gradient Compression for Fine-Tuning Large Language Models

LifeLong Learning for Large Language Models in Predicting Chemical Reaction Yields

LoRA Fine-Tuning Without GPUs: A CPU-Efficient Meta-Generation Framework for LLMs

Ontology-based Protein-Protein Interaction Explanation Using Large Language Models

Efficient inference, training, and fine-tuning of protein language models

Fine-tuning molecular mechanics force fields to experimental free energy measurements

Fine-tuning protein language models boosts predictions across diverse tasks

CrysText: A Generative AI Approach for Text-Conditioned Crystal Structure Generation using LLM

Crafting Efficient Fine-Tuning Strategies for Large Language Models

Multi-fidelity machine learning models for improved high-throughput screening predictions

The Impact of Reward Scalarization and Weight Scheduling on Optimization Dynamics in Multi-Objective Molecular Design

Neural Scaling of Deep Chemical Models

The most compact search space is not always the most efficient: A case study on maximizing solid rocket fuel packing fraction via constrained Bayesian optimization