24 papers · ranked by Valyu relevance
Tom J. Viering, Marco Loog
—Learning curves provide insight into the dependence of a learner's generalization performance on the training set size. This important tool can be used for model selection, to predict the effect of more training data, and to reduce the computational complexity of model training and hyperparameter tuning. This review…
Alex Nguyen, David J. Schwab, Vudtiwat Ngampruetikorn
Lossy data transformations by definition lose information. Yet, in modern machine learning, methods like data pruning and lossy data augmentation can help improve generalization performance. We study this paradox using a solvable model of high-dimensional, ridge-regularized linear regression under data coarse graining.…
Joseph Scott German, Guofeng Cui, Chenliang Xu, Robert A. Jacobs + 1 more
'Ming Bo Cai'] We propose the “runtime learning” hypothesis which states that people quickly learn to perform unfamiliar tasks as the tasks arise by using task-relevant instances of concepts stored in memory during mental training. To make learning rapid, the hypothesis claims that only a few class instances are used…
Stephen Purpura, Dustin Hillard, Mark Hubenthal, Jim Walsh + 2 more
'Scott A. Golder' 'S. Alex Smith'] We present a system that enables rapid model experimentation for tera-scale machine learning with trillions of non-zero features, billions of training examples, and millions of parameters. Our contribution to the literature is a new method (SA L-BFGS) for changing batch L-BFGS to…
Swaraj Dube, Yee Wan Wong, Hermawan Nugroho, Yilun Shang
Incremental learning evolves deep neural network knowledge over time by learning continuously from new data instead of training a model just once with all data present before the training starts. However, in incremental learning, new samples are always streaming in whereby the model to be trained needs to continuously…
Ashish Sabharwal, Horst Samulowitz, Gerald Tesauro
We study a novel machine learning (ML) problem setting of sequentially allocating small subsets of training data amongst a large set of classifiers. The goal is to select a classifier that will give near-optimal accuracy when trained on all data, while also minimizing the cost of misallocated samples. This is motivated…
Simon Viet Johansson, Hampus Gummesson Svensson, Esben Bjerrum, Alexander Schliep + 3 more
Computer aided synthesis planning is a rapidly growing field for suggesting synthetic routes for molecules of interest. The methods used are usually dependent on access to large datasets for training, but with a finite experimental budget there are limitations on how much data can be obtained from experiments. Active…
Jianfeng Xu, Congcong Liu, Xiaoying Tan, Xiaojie Zhu + 7 more
'Huan Wan' 'Weijun Kong' 'Chun Li' 'Xu Hu' 'Kun Kuang' 'Fei Wu'] To address the growing size of AI model training data and the lack of a universal data selection methodology—factors that significantly drive up training costs—this paper presents the General Information Metrics Evaluation (GIME) method. GIME leverages…
Tianhao Wang, Si Chen, Ruoxi Jia
In this work, we initiate the study of one-round active learning, which aims to select a subset of unlabeled data points that achieve the highest model performance after being labeled with only the information from initially labeled data points. The challenge of directly applying existing data selection criteria to the…
Nicholas Menghi, Kemal Kacar, Will Penny
This paper uses constructs from the field of multitask machine learning to define pairs of learning tasks that either shared or did not share a common subspace. Human subjects then learnt these tasks using a feedback-based approach. We found, as hypothesised, that subject performance was significantly higher on the…
Gido M. van de Ven, Tinne Tuytelaars, Andreas S. Tolias
Incrementally learning new information from a non-stationary stream of data, referred to as ‘continual learning’, is a key feature of natural intelligence, but a challenging problem for deep neural networks. In recent years, numerous deep learning methods for continual learning have been proposed, but comparing their…
Changyu Deng, Xunbi Ji, Colton Rainey, Jianyu Zhang + 1 more
Title: Summary Machine learning has been heavily researched and widely used in many disciplines. However, achieving high accuracy requires a large amount of data that is sometimes difficult, expensive, or impractical to obtain. Integrating human knowledge into machine learning can significantly reduce data requirement…
Ryan Smith, Philipp Schwartenbeck, Thomas Parr, Karl J. Friston
The algorithmic and neural basis of concept learning remains poorly understood. In this paper, we articulate a novel, biologically plausible approach to concept learning based on active inference, and on the idea that a generative model can be equipped with extra (hidden state or cause) ‘slots’ that can be engaged when…
Alexander Pomberger, Antonio Pedrina McCarthy, Ahmad Khan, Simon Sung + 4 more
Multivariate chemical reaction optimization involving catalytic systems is a non-trivial task due to the high number of tuneable parameters and discrete choices. Closed-loop optimization featuring active Machine Learning (ML) represents a powerful strategy for automating reaction optimization. However, the translation…
Robert C. Wilson, Amitai Shenhav, Mark Straccia, Jonathan D. Cohen
Researchers and educators have long wrestled with the question of how best to teach their clients be they human, animal or machine. Here we focus on the role of a single variable, the difficulty of training, and examine its effect on the rate of learning. In many situations we find that there is a sweet spot in which…
Paul Francoeur, Daniel Penaherrera, David Koes
The immense size of chemical space, the relative scarcity of high quality data, and the cost of running experiments to accurately measure molecular properties makes active learning (AL) an attractive approach to efficiently explore the space and train high-quality models for molecular property prediction. While AL is…
Authors not listed
Machine olfaction—the artificial replication of the sense of smell—faces significant challenges due to the absence of large, standardized training datasets. Unlike vision, language, and audio models, which benefit from extensive corpora such as ImageNet, GLUE, and AudioSet, olfaction lacks scaled equivalents and…
Carey E. Priebe, Joshua T. Vogelstein, Florian Engert, Christopher M. White
We present modern machine learning, focusing on the state-of-the-art classification methods of decision forests and deep networks, as partition and vote schemes. This illustrative presentation allows for both a unified basic understanding of how these methods work from the perspective of classical statistical pattern…
Masanari Kimura, Howard D. Bondell
Data augmentation is known to contribute significantly to the robustness of machine learning models. In most instances, data augmentation is utilized during the training phase. Test-Time Augmentation (TTA) is a technique that instead leverages these data augmentations during the testing phase to achieve robust…
Chung‐Wei Lee, Pavlos Anastasios Apostolopulos, Igor L. Markov
For tabular data sets, we explore data and model distillation, as well as data denoising. These techniques improve both gradient-boosting models and a specialized DNN architecture. While gradient boosting is known to outperform DNNs on tabular data, we close the gap for datasets with 100K+ rows and give DNNs an…
David Buterez, Jon Paul Janet, Steven Kiddle, Pietro Liò
We investigate the potential of graph neural networks for transfer learning and improving molecular property prediction on sparse and expensive to acquire high-fidelity data by leveraging low-fidelity measurements as an inexpensive proxy for a targeted property ofinterest. This problem arises in discovery processes…
Romik Ghosh, Dana Mastrovito, Stefan Mihalas
The human brain readily learns tasks in sequence without forgetting previous ones. Artificial neural networks (ANNs), on the other hand, need to be modified to achieve similar performance. While effective, many algorithms that accomplish this are based on weight importance methods that do not correspond to biological…
Luca Saglietti, Stefano Sarao Mannelli, Andrew Saxe
In animals and humans, curriculum learning-presenting data in a curated order-is critical to rapid learning and effective pedagogy. A long history of experiments has demonstrated the impact of curricula in a variety of animals but, despite its ubiquitous presence, a theoretical understanding of the phenomenon is still…
Yannick Ureel, Maarten R. Dobbelaere, Yi Ouyang, Kevin De Ras + 3 more
By combining machine learning with design of experiments, so-called active machine learning, more efficient and cheaper research can be conducted. Machine learning algorithms are more flexible, and are better at investigating the processes spanning all length scales of chemical engineering. While the active machine…