Search · four archives

Results for Outliers and missing data

or ask it as a questionAsk the stacks →

ArchivesarXiv PubMed bioRxiv chemRxiv

WhenAny time Past month Past year Past 5 years

OrderMost relevant Most cited Newest

26 papers · ranked by Valyu relevance

01
PPubMed14 Mar 2019
Anomalous values and missing data in clinical and experimental studies
Hélio Amante Miot
During analysis of scientific research data, it is customary to encounter anomalous values or missing data. Anomalous values can be the result of errors of recording, typing, measurement by instruments, or may be true outliers. This review discusses concepts, examples and methods for identifying and dealing with such…
91relevance
02
AarXiv24 Jan 2019
MacroPCA: An All-in-One PCA Method Allowing for Missing Values as Well as Cellwise and Rowwise Outliers
Mia Hubert, Peter J. Rousseeuw, Wannes Van den Bossche
Multivariate data are typically represented by a rectangular matrix (table) in which the rows are the objects (cases) and the columns are the variables (measurements). When there are many variables one often reduces the dimension by principal component analysis (PCA), which in its basic form is not robust to outliers.…
87relevance
03
PPubMed10 Jul 2014
Outlier Detection Method in Linear Regression Based on Sum of Arithmetic Progression
K. K. L. B. Adikaram, M. A. Hussein, M. Effenberger, T. Becker
We introduce a new nonparametric outlier detection method for linear series, which requires no missing or removed data imputation. For an arithmetic progression (a series without outliers) with n elements, the ratio (R) of the sum of the minimum and the maximum elements and the sum of all elements is always 2/n …
87relevance
04
PPubMed9 Mar 2006
Detecting outliers when fitting data with nonlinear regression – a new method based on robust nonlinear regression and the false discovery rate
Harvey J Motulsky, Ronald E Brown
Background Nonlinear regression, like linear regression, assumes that the scatter of data around the ideal curve follows a Gaussian or normal distribution. This assumption leads to the familiar goal of regression: to minimize the sum of the squares of the vertical or Y-value distances between the points and the curve.…
85relevance
05
BbioRxiv2 Feb 2018
BDQC: a general-purpose analytics tool for domain-blind validation of Big Data
Eric W. Deutsch, Roger Kramer, Joseph Ames, Andrew Bauman + 21 more
Translational biomedical research is generating exponentially more data: thousands of whole-genome sequences (WGS) are now available; brain data are doubling every two years. Analyses of Big Data, including imaging, genomic, phenotypic, and clinical data, present qualitatively new challenges as well as opportunities.…
83relevance
06
AarXiv1 Nov 2021
ReLearn: A Robust Machine Learning Framework in Presence of Missing Data for Multimodal Stress Detection from Physiological Signals
Arman Iranfar, Adriana Arza, David Atienza
— Continuous and multimodal stress detection has been performed recently through wearable devices and machine learning algorithms. However, a well-known and important challenge of working on physiological signals recorded by conventional monitoring devices is missing data due to sensors insufficient contact and…
82relevance
07
AarXiv2023
Robust covariance estimation with missing values and cell-wise contamination
Karim Lounici, Grégoire Pacreau
Large datasets are often affected by cell-wise outliers in the form of missing or erroneous data. However, discarding any samples containing outliers may result in a dataset that is too small to accurately estimate the covariance matrix. Moreover, the robust procedures designed to address this problem require the…
81relevance
08
BbioRxiv28 Feb 2022
Information-Content-Informed Kendall-tau Correlation: Utilizing Missing Values
Robert M Flight, Praneeth S Bhatt, Hunter NB Moseley
Almost all correlation measures currently available are unable to directly handle missing values. Typically, missing values are either ignored completely by removing them or are imputed and used in the calculation of the correlation coefficient. In both cases, the correlation value will be impacted based on a…
80relevance
09
AarXiv28 Jul 2025
An empirical comparison of some outlier detection methods with longitudinal data
D'Orazio, Marcello
This note investigates the problem of detecting outliers in longitudinal data. It compares wellknown methods used in official statistics with proposals from the fields of data mining and machine learning that are based on the distance between observations or binary partitioning trees. This is achieved by applying the…
79relevance
10
AarXiv2012
Statistical Distortion: Consequences of Data Cleaning
Tamraparni Dasu, Ji Meng Loh
We introduce the notion of statistical distortion as an essential metric for measuring the effectiveness of data cleaning strategies. We use this metric to propose a widely applicable yet scalable experimental framework for evaluating data cleaning strategies along three dimensions: glitch improvement, statistical…
77relevance
11
BbioRxiv14 Jun 2019
Robust estimation of heritability and predictive accuracy in plant breeding: evaluation using simulation and empirical data
Vanda M Lourenço, Joseph O Ogutu, Hans-Peter Piepho
Genomic prediction (GP) is used in animal and plant breeding to help identify the best genotypes for selection. One of the most important measures of the effectiveness and reliability of GP in plant breeding is predictive accuracy. An accurate estimate of this measure is thus central to GP. Moreover, regression models…
77relevance
12
CchemRxiv25 Mar 2024
Beyond Molecular Structure: Critically Assessing Machine Learning for Designing Organic Photovoltaic Materials and Devices
Martin Seifrid, Stanley Lo, Dylan Choi, Gary Tom + 12 more
Martin Seifrid 1 , Stanley Lo 2 , Dylan G. Choi 3 , Gary Tom 2 , My Linh Le 3 , Kunyu Li 3 , Rahul Sankar 3 , Hoai-Thanh Vuong 3 , Hiba Wakidi 3 , Ahra Yi 3 , Ziyue Zhu 3 , Nora Schopp 3 , Aaron Peng 3 , Benjamin Luginbuhl 3 , Thuc-Quyen Nguyen 3 , Alán Aspuru-Guzik 2
75relevance
13
BbioRxiv12 Sep 2016
A note on detecting statistical outliers in psychophysical data
Pete R. Jones
This paper considers how best to identify statistical outliers in psychophysical datasets, where the underlying sampling distributions are unknown. Eight methods are described, and each is evaluated using Monte Carlo simulations of a typical psychophysical experiment. The best method is shown to be one based on a…
74relevance
14
BbioRxiv30 Jul 2021
Robust adaptive distance functions for approximate Bayesian inference on outlier-corrupted data
Yannik Schälte, Emad Alamoudi, Jan Hasenauer
Approximate Bayesian Computation (ABC) is a likelihood-free parameter inference method for complex stochastic models in systems biology and other research areas. While conceptually simple, its practical performance relies on the ability to efficiently compare relevant features in simulated and observed data via…
74relevance
15
BbioRxiv7 Aug 2020
Probability-based methods for outlier detection in replicated high-throughput biological data
Matthew S. Smith, Karthik Devarajan
A problem that arises frequently in high-throughput biological studies is the assessment of technical reproducibility of data obtained under homogeneous experimental conditions. This is an important problem considering the significant growth in the number of high-throughput technologies that have become available to…
74relevance
16
PPubMed5 Jul 2012
Distribution of Variables by Method of Outlier Detection
W. Holmes Finch
The presence of outliers can very problematic in data analysis, leading statisticians to develop a wide variety of methods for identifying them in both the univariate and multivariate contexts. In case of the latter, perhaps the most popular approach has been Mahalanobis distance, where large values suggest an…
72relevance
17
AarXiv23 Jun 2014
Robust estimation of multivariate location and scatter in the presence of cellwise and casewise contamination
Claudio Agostinelli, Andy Leung, Vı́ctor J. Yohai, Ruben H. Zamar
of cellwise and casewise contamination Authors: ['Claudio Agostinelli' 'Andy Leung' 'Vı́ctor J. Yohai' 'Ruben H. Zamar'] Multivariate location and scatter matrix estimation is a cornerstone in multivariate data analysis. We consider this problem when the data may contain independent cellwise and casewise outliers. Flat…
72relevance
18
PPubMed27 Sep 2022
Two novel outlier detection approaches based on unsupervised possibilistic and fuzzy clustering
Zeynel Cebeci, Cagatay Cebeci, Yalcin Tahtali, Lutfi Bayyurt + 1 more
'Yilun Shang'] Outliers are data points that significantly deviate from other data points in a data set because of different mechanisms or unusual processes. Outlier detection is one of the intensively studied research topics for identification of novelties, frauds, anomalies, deviations or exceptions in addition to…
71relevance
19
CchemRxiv16 Dec 2024
P2MAT: A machine learning (ML) driven software for Property Prediction of MATerial
Authors not listed
Accurate prediction of melting points for pure molecules remains a significant challenge in predictive chemistry, with implications across various scientific fields, including materials science, drug discovery, and separations chemistry. Traditional methods, such as group contribution (GC) techniques, have shown…
70relevance
20
AarXiv2019
Doubts on the efficacy of outliers correction methods
Marjorie Fonnesu, Nicola Kuczewski
While the utilisation of different methods of outliers correction has been shown to counteract the inferential error produced by the presence of contaminating data not belonging to the studied population; the effects produced by their utilisation when samples do not contain contaminating outliers are less clear. Here a…
69relevance
21
PPubMed16 Feb 2011
A random effects variance shift model for detecting and accommodating outliers in meta-analysis
Freedom N Gumedze, Dan Jackson
Background Meta-analysis typically involves combining the estimates from independent studies in order to estimate a parameter of interest across a population of studies. However, outliers often occur even under the random effects model. The presence of such outliers could substantially alter the conclusions in a…
69relevance
22
PPubMed22 May 2024
Outlier analysis for accelerating clinical discovery: An augmented intelligence framework and a systematic review An augmented intelligence framework for clinical discovery through outlier analysis
Ghayath Janoudi, Mara Uzun (Rada), Deshayne B. Fell, Joel G. Ray + 5 more
'Angel M. Foster' 'Randy Giffen' 'Tammy Clifford' 'Mark C. Walker' 'Raymond Francis Sarmiento'] Clinical discoveries largely depend on dedicated clinicians and scientists to identify and pursue unique and unusual clinical encounters with patients and communicate these through case reports and case series. This process…
69relevance
23
CchemRxiv7 Oct 2025
Robust Learning from Literature Data: Model Generalizability and Uncertainty for Predicting Conjugated Polymer Solution Conformation
Authors not listed
Predicting solution conformation and aggregation of conjugated polymers remains a bottleneck for translating solution processing into controlled film microstructure and for closing the loop in self-driving laboratories. We construct a cleaned, machine-readable dataset of 256 entries that links polymer size…
68relevance
24
CchemRxiv30 Dec 2025
Decoding the Unseen: Unsupervised Anomaly Detection in Metal–Organic Frameworks for Discovery Beyond the Norm
Authors not listed
The discovery of chemically novel or structurally anomalous metal-organic frameworks (MOFs) is essential for expanding reticular design space and enhancing dataset reliability. We present CHEM-AD (Chemically Unusual Metal–organic Frameworks via Autoencoder-based Detection), a label-free, CPU-efficient pipeline that…
67relevance
25
CchemRxiv5 Jun 2023
Data-driven imputation of miscibility of aqueous solutions via graph-regularized logistic matrix factorization
Diba Behnoudfar, Cory Simon, Joshua Schrier
Aqueous, two-phase systems (ATPSs) may form upon mixing two solutions of independently water-soluble compounds. Many separation, purification, and extraction processes rely on ATPSs. Predicting the miscibility of solutions can accelerate and reduce the cost of the discovery of new ATPSs for these applications. Whereas…
62relevance
26
CchemRxiv25 Mar 2025
Capturing Variability in Material Property Predictions for Plastics Recycling via Machine Learning
Authors not listed
Plastic mechanical recycling is the conventional technological step towards circularity. In such aspects, complex mixtures of polyolefin blends are often fed into mechanical recycling systems, resulting in moulded products with uncertain quality. To add to the difficulty of heterogeneous feedstocks, the testing of…
54relevance

Results for Outliers and missing data

Anomalous values and missing data in clinical and experimental studies

MacroPCA: An All-in-One PCA Method Allowing for Missing Values as Well as Cellwise and Rowwise Outliers

Outlier Detection Method in Linear Regression Based on Sum of Arithmetic Progression

Detecting outliers when fitting data with nonlinear regression – a new method based on robust nonlinear regression and the false discovery rate

BDQC: a general-purpose analytics tool for domain-blind validation of Big Data

ReLearn: A Robust Machine Learning Framework in Presence of Missing Data for Multimodal Stress Detection from Physiological Signals

Robust covariance estimation with missing values and cell-wise contamination

Information-Content-Informed Kendall-tau Correlation: Utilizing Missing Values

An empirical comparison of some outlier detection methods with longitudinal data

Statistical Distortion: Consequences of Data Cleaning

Robust estimation of heritability and predictive accuracy in plant breeding: evaluation using simulation and empirical data

Beyond Molecular Structure: Critically Assessing Machine Learning for Designing Organic Photovoltaic Materials and Devices

A note on detecting statistical outliers in psychophysical data

Robust adaptive distance functions for approximate Bayesian inference on outlier-corrupted data

Probability-based methods for outlier detection in replicated high-throughput biological data

Distribution of Variables by Method of Outlier Detection

Robust estimation of multivariate location and scatter in the presence of cellwise and casewise contamination

Two novel outlier detection approaches based on unsupervised possibilistic and fuzzy clustering

P2MAT: A machine learning (ML) driven software for Property Prediction of MATerial

Doubts on the efficacy of outliers correction methods

A random effects variance shift model for detecting and accommodating outliers in meta-analysis

Outlier analysis for accelerating clinical discovery: An augmented intelligence framework and a systematic review An augmented intelligence framework for clinical discovery through outlier analysis

Robust Learning from Literature Data: Model Generalizability and Uncertainty for Predicting Conjugated Polymer Solution Conformation

Decoding the Unseen: Unsupervised Anomaly Detection in Metal–Organic Frameworks for Discovery Beyond the Norm

Data-driven imputation of miscibility of aqueous solutions via graph-regularized logistic matrix factorization

Capturing Variability in Material Property Predictions for Plastics Recycling via Machine Learning

Anomalous values and missing data in clinical and experimental studies

MacroPCA: An All-in-One PCA Method Allowing for Missing Values as Well as Cellwise and Rowwise Outliers

Outlier Detection Method in Linear Regression Based on Sum of Arithmetic Progression

Detecting outliers when fitting data with nonlinear regression – a new method based on robust nonlinear regression and the false discovery rate

BDQC: a general-purpose analytics tool for domain-blind validation of Big Data

ReLearn: A Robust Machine Learning Framework in Presence of Missing Data for Multimodal Stress Detection from Physiological Signals

Robust covariance estimation with missing values and cell-wise contamination

Information-Content-Informed Kendall-tau Correlation: Utilizing Missing Values

An empirical comparison of some outlier detection methods with longitudinal data

Statistical Distortion: Consequences of Data Cleaning

Robust estimation of heritability and predictive accuracy in plant breeding: evaluation using simulation and empirical data

Beyond Molecular Structure: Critically Assessing Machine Learning for Designing Organic Photovoltaic Materials and Devices

A note on detecting statistical outliers in psychophysical data

Robust adaptive distance functions for approximate Bayesian inference on outlier-corrupted data

Probability-based methods for outlier detection in replicated high-throughput biological data

Distribution of Variables by Method of Outlier Detection

Robust estimation of multivariate location and scatter in the presence of cellwise and casewise contamination

Two novel outlier detection approaches based on unsupervised possibilistic and fuzzy clustering

P2MAT: A machine learning (ML) driven software for Property Prediction of MATerial

Doubts on the efficacy of outliers correction methods

A random effects variance shift model for detecting and accommodating outliers in meta-analysis

Outlier analysis for accelerating clinical discovery: An augmented intelligence framework and a systematic review An augmented intelligence framework for clinical discovery through outlier analysis

Robust Learning from Literature Data: Model Generalizability and Uncertainty for Predicting Conjugated Polymer Solution Conformation

Decoding the Unseen: Unsupervised Anomaly Detection in Metal–Organic Frameworks for Discovery Beyond the Norm

Data-driven imputation of miscibility of aqueous solutions via graph-regularized logistic matrix factorization

Capturing Variability in Material Property Predictions for Plastics Recycling via Machine Learning