26 papers · ranked by Valyu relevance
Timothée Lesort, Vincenzo Lomonaco, Andrei Stoian, Davide Maltoni + 2 more
'David Filliat' 'Natalia Díaz-Rodríguez'] Continual learning (CL) is a particular machine learning paradigm where the data distribution and learning objective change through time, or where all the training data and objective criteria are never available at once. The evolution of the learning process is modeled by a…
Gido M. van de Ven, Tinne Tuytelaars, Andreas S. Tolias
Incrementally learning new information from a non-stationary stream of data, referred to as ‘continual learning’, is a key feature of natural intelligence, but a challenging problem for deep neural networks. In recent years, numerous deep learning methods for continual learning have been proposed, but comparing their…
Muhammad Burhan Hafez, Kerim Erekmen
Central to the development of universal learning systems is the ability to solve multiple tasks without retraining from scratch when new data arrives. This is crucial because each task requires significant training time. Addressing the problem of continual learning necessitates various methods due to the complexity of…
Tameem Adel
Continual learning is an online paradigm where a learner continually accumulates knowledge from different tasks encountered over sequential time steps. Importantly, the learner is required to extend and update its knowledge without forgetting about the learning experience acquired from the past, and while avoiding the…
Liyuan Wang, Xingxing Zhang, Hang Su, Jun Zhu
—To cope with real-world dynamics, an intelligent system needs to incrementally acquire, update, accumulate, and exploit knowledge throughout its lifetime. This ability, known as continual learning, provides a foundation for AI systems to develop themselves adaptively. In a general sense, continual learning is…
Eli Verwimp, Rahaf Aljundi, Shai Ben-David, Matthias Bethge + 16 more
'Andrea Cossu' 'Alexander Gepperth' 'Tyler L. Hayes' 'Eyke Hüllermeier' 'Christopher Kanan' 'Dhireesha Kudithipudi' 'Christoph H. Lampert' 'Martin Mundt' 'Razvan Pascanu' 'Adrian Popescu' 'Andreas S. Tolias' 'Joost van de Weijer' 'Bing Liu' 'Vincenzo Lomonaco' 'Tinne Tuytelaars' 'Gido M. van de Ven'] Eli Verwimp∗ KU…
Gido M. van de Ven, Nicholas Soures, Dhireesha Kudithipudi
This book chapter delves into the dynamics of continual learning, which is the process of incrementally learning from a non-stationary stream of data. Although continual learning is a natural skill for the human brain, it is very challenging for artificial neural networks. An important reason is that, when learning…
Karan Grewal, Jeremy Forest, Benjamin P. Cohen, Subutai Ahmad
Biological neurons integrate their inputs on dendrites using a diverse range of non-linear functions. However the majority of artificial neural networks (ANNs) ignore biological neurons’ structural complexity and instead use simplified point neurons. Can dendritic properties add value to ANNs? In this paper we…
Timo Flesch, David G. Nagy, Andrew Saxe, Christopher Summerfield + 1 more
'Alireza Soltani'] Humans can learn several tasks in succession with minimal mutual interference but perform more poorly when trained on multiple tasks at once. The opposite is true for standard deep neural networks. Here, we propose novel computational constraints for artificial neural networks, inspired by earlier…
Martin Mundt, Iuliia Pliushch, Sagnik Majumder, Yongwon Hong + 3 more
Modern deep neural networks are well known to be brittle in the face of unknown data instances and recognition of the latter remains a challenge. Although it is inevitable for continual-learning systems to encounter such unseen concepts, the corresponding literature appears to nonetheless focus primarily on alleviating…
Tanvi Verma, Liyuan Jin, Jun Zhou, Jia Huang + 7 more
'Benjamin Chen Ming Choong' 'Ting Fang Tan' 'Fei Gao' 'Xinxing Xu' 'Daniel S. Ting' 'Yong Liu'] Background The implementation of deep learning models for medical image classification poses significant challenges, including gradual performance degradation and limited adaptability to new diseases. However, frequent…
Saba Aslam, Abdur Rasool, Hongyan Wu, Xiaoli Li
Continual learning, the ability of a model to learn over time without forgetting previous knowledge and, therefore, be adaptive to new data, is paramount in dynamic fields such as disease outbreak prediction. Deep neural networks, i.e., LSTM, are prone to error due to catastrophic forgetting. This study introduces a…
Zihan Liu, Anno Kurth, Yuma Osako, Toshitake Asabuki
Humans and animals can learn and seamlessly perform a vast repertoire of behaviors. However, how neural populations incorporate new skills without disrupting previously learned ones remains poorly understood. This challenge is known as catastrophic forgetting in artificial neural networks and is especially severe in…
Authors not listed
Large Language Models (LLMs) based on transformer architectures excel at internet-scale tasks. However, real-world scientific scenarios—such as synthetic chemistry laboratories and autonomous experimental setups—typically involve incremental data generation in batches as new chemical reactions are conducted, unlike…
Andrea Cossu, Gabriele Graffieti, Lorenzo Pellegrini, Davide Maltoni + 3 more
'Davide Bacciu' 'Antonio Carta' 'Vincenzo Lomonaco'] The ability of a model to learn continually can be empirically assessed in different continual learning scenarios. Each scenario defines the constraints and the opportunities of the learning environment. Here, we challenge the current trend in the continual learning…
Gideon Kowadlo, Abdelrahman Ahmed, Amir Mayan, David Rawlinson + 1 more
'T. Ganesh Kumar'] Continual learning and few-shot learning are important frontiers in progress toward broader Machine Learning (ML) capabilities. Recently, there has been intense interest in combining both. One of the first examples to do so was the Continual few-shot Learning (CFSL) framework of Antoniou et al.…
Timothée Lesort, M. Caccia, Irina Rish
Classical machine learning algorithms often assume that the data are drawn i.i.d. from a stationary probability distribution. Recently, continual learning emerged as a rapidly growing area of machine learning where this assumption is relaxed, i.e. where the data distribution is non-stationary and changes over time.…
Timo Flesch, Jan Balaguer, Ronald Dekker, Hamed Nili + 1 more
Humans can learn to perform multiple tasks in succession over the lifespan (“continual” learning), whereas current machine learning systems fail. Here, we investigated the cognitive mechanisms that permit successful continual learning in humans. Unlike neural networks, humans that were trained on temporally…
Yihan Zhao, Wenqing Su, Ying Yang
Continual learning is motivated by the need to adapt to real-world dynamics in tasks and data distribution while mitigating catastrophic forgetting. Despite significant advances in continual learning techniques, the theoretical understanding of their generalization performance lags behind. This paper examines the…
Danil Tyulmankov, Guangyu Robert Yang, LF Abbott
Over the course of a lifetime, a continual stream of information is encoded and retrieved from memory. To explore the synaptic mechanisms that enable this ongoing process, we consider a continual familiarity detection task in which a subject must report whether an image has been previously encountered. We design a…
N. Menghi, S. Vigano’, W. J. Johnston, S. Elnagar + 2 more
Learning depends not only on the content of what we learn, but also on how we learn and on how experiences are structured over time. To investigate how task similarity and training regime interact during learning, we trained participants on spatial and conceptual learning tasks that shared either similar or distinct…
Authors not listed
Machine olfaction—the artificial replication of the sense of smell—faces significant challenges due to the absence of large, standardized training datasets. Unlike vision, language, and audio models, which benefit from extensive corpora such as ImageNet, GLUE, and AudioSet, olfaction lacks scaled equivalents and…
Hua-Dong Xiong, Li Ji-An, Robert C. Wilson, Marcelo G. Mattar
A hallmark of intelligence is the ability to adapt behavior to changing environments, which requires adapting one’s own learning strategies. This phenomenon is known as learning to learn or meta-learning. Although well established in humans and animals, a computational framework that characterizes how biological agents…
Derek van Tilborg, Helena Brinkmann, Emanuele Criscuolo, Luke Rossen + 2 more
Deep learning is becoming increasingly relevant in drug discovery, from de novo design to protein structure prediction and synthesis planning. However, it is often challenged by the small data regimes typical of certain drug discovery tasks. In such scenarios, deep learning approaches – which are notoriously…
Yannick Ureel, Maarten R. Dobbelaere, Yi Ouyang, Kevin De Ras + 3 more
By combining machine learning with design of experiments, so-called active machine learning, more efficient and cheaper research can be conducted. Machine learning algorithms are more flexible, and are better at investigating the processes spanning all length scales of chemical engineering. While the active machine…
Muhammad Hanzla, Abdul Rehman Shinwari
Machine Learning (ML) can be defined as a class of Artificial Intelligence for automated data analysis, which is capable of detecting patterns in data. The extracted patterns can be used to predict un-known data or to assist in decision-making processes under uncertainty. Recent advances in experimental and…