26 papers · ranked by Valyu relevance
Melanie Hawkins, Gerald R. Elsworth, Richard H. Osborne
Background Data from subjective patient-reported outcome measures (PROMs) are now being used in the health sector to make or support decisions about individuals, groups and populations. Contemporary validity theorists define validity not as a statistical property of the test but as the extent to which empirical…
Mark van der Loo, Edwin de Jonge
Data validation is the activity where one decides whether or not a particular data set is fit for a given purpose. Formalizing the requirements that drive this decision process allows for unambiguous communication of the requirements, automation of the decision process, and opens up ways to maintain and investigate the…
Benjamin Kinnear, Christina St-Onge, Daniel J. Schumacher, Mélanie Marceau + 1 more
'Mélanie Marceau' 'Thirusha Naidu'] Validity has long held a venerated place in education, leading some authors to refer to it as the “sine qua non” or “cardinal virtue” of assessment. And yet, validity has not held a fixed meaning; rather it has shifted in its definition and scope over time. In this Eye Opener, the…
Jinsong Chen
Test validity lies at the core of educational and psychological testing, but there are controversies about what test validity is and how test validation should proceed. This paper develops a taxonomy to redefine test validity with hierarchical levels. On the basis of testing foundation, the hierarchy includes…
Raheleh Khorsan, Cindy Crawford
Background. Evidence rankings do not consider equally internal (IV), external (EV), and model validity (MV) for clinical studies including complementary and alternative medicine/integrative medicine (CAM/IM) research. This paper describe this model and offers an EV assessment tool (EVAT(c)) for weighing studies…
Zarrin Seema Siddiqui
A defensible assessment is the cornerstone of health professions education and requires a careful approach from all the stakeholders involved in designing and delivering the curriculum. In this article, a framework is described using four essential attributes that define an effective assessment. These attributes…
K. Larsen, R. Lukyanenko, Roland M. Mueller, V. Storey + 3 more
Researchers must ensure that the claims about the knowledge produced by their work are valid. However, validity is neither well-understood nor consistently established in design science, which involves the development and evaluation of artifacts (models, methods, instantiations, and theories) to solve problems. As a…
Lluna Maria Bru-Luna, Manuel Martí-Vilar, César Merino-Soto, José Livia-Segovia + 2 more
'José Livia-Segovia' 'Juan Garduño-Espinosa' 'Filiberto Toledano-Toledano'] Background The person-centered care (PCC) approach plays a fundamental role in ensuring quality healthcare. The Person-Centered Care Assessment Tool (P-CAT) is one of the shortest and simplest tools currently available for measuring PCC. The…
Josh Joseph Ramminger, Niklas Jacobs
Several conceptions of validity have emphasized the contingency of validity on theory. Here we revisit several contributions to the discourse on the concept of validity, which we consider particularly influential or insightful. Despite differences in metatheory, both Cronbach and Meehl’s construct validity, and…
Olawale Salaudeen, Anka Reuel, Ahmed Ahmed, Suhana Bedi + 5 more
'Zachary Robertson' 'Sudharsan Sundar' 'Ben Domingue' 'Angelina Wang' 'Sanmi Koyejo'] While the capabilities and utility of AI systems have advanced, rigorous norms for evaluating these systems have lagged. Grand claims, such as models achieving general reasoning capabilities, are supported with model performance on…
John Mingers, Craig Standing
Research in information systems includes a wide range of approaches which make a contribution in terms of knowledge, understanding, or practical developments. The measure of any research is, ultimately, its validity – are its finding true, or its recommendations correct? However, empirical studies show that discussion…
Koichi Handa, Morgan Thomas, Michiharu Kageyama, Takeshi Iijima + 1 more
While a multitude of deep generative models have recently emerged there exists no best practice for their practically relevant validation. On the one hand, novel de novo-generated molecules cannot be refuted by retrospective validation (so that this type of validation is biased); but on the other hand prospective…
Bob L. Sturm, Arthur Flexer
Validity is the truth of an inference made from evidence, such as data collected in an experiment, and is central to working scientifically. Given the maturity of the domain of music information research (MIR), validity in our opinion should be discussed and considered much more than it has been so far. Considering…
Rickey E Carter
Background In 2003, the United States Food and Drug Administration (FDA) released a guidance document on the scope of "Part 11" enforcement. In this guidance document, the FDA indicates an expectation of a risk-based approach to determining which systems should undergo validation. Since statistical programs manage and…
Victor H. R. Nogueira, Rishabh Sharma, Rafael V. C. Guido, Michael J. Keiser
As efforts to improve the robustness of molecular representations advance, so does the need for methods to test and validate them. We use a Variational Auto-Encoder (VAE), an unsupervised deep learning model, to generate anomalous samples of a well-known molecular string format called SELF-referencIng Embedded Strings…
Peter Grabitz, Yuri Lazebnik, Josh Nicholson, Sean Rife
The current crisis of veracity in biomedical research is enabled by the lack of publicly accessible information on whether the reported scientific claims are valid. One approach to solve this problem is to replicate previous studies by specialized reproducibility centers. However, this approach is costly or…
Larissa C. Hunt, Josef Fritz, Michael Herf, Lorna Herf + 1 more
Wearable light sensors are increasingly used in intervention and population-based studies investigating the consequences of environmental light exposure on human physiology. An important step in such analyses is the reliable detection of non-wear time. We observed in light data that days with less wear-time also have…
Linde Schoenmaker, Olivier Béquignon, Willem Jespers, Gerard van Westen
Generative deep learning models have emerged as a powerful approach for de novo drug design, as they aid researchers in finding new molecules with desired properties. Despite continuous improvements in the field, a subset of the outputs that sequence-based de novo generators produce cannot be progressed due to errors.…
Yanlin Zhang, Jing Zhao
We formalize scientific methodology—the end-to-end process from question formulation to evidence-grounded writing—as a phase-gated research protocol with explicit return paths and persistent constraints, and instantiate it for general-purpose language models as executable protocol specifications. The formalization…
Hanumanthrao Kannan, Alejandro Salado
This paper presents a formal theory of verification and validation (V&V) within systems engineering, grounded in the axiom that V&V are fundamentally knowledge-building activities. Using dynamic epistemic modal logic, we develop precise definitions of verification and validation, articulating their roles in confirming…
Naruki Yoshikawa, Kentaro Rikimaru, Kazuki Yamamoto
Many computer-aided drug design (CADD) methods using deep learning have recently been proposed to explore the chemical space toward novel scaffolds efficiently. However, there is a tradeoff between the ease of generating novel structures and the chemical feasibility of structural formulas. To overcome the limitations…
Veronica Dudarev, Oswald Barral, Chuxuan Zhang, Guy Davis + 1 more
Wearable sensors are quickly making their way into psychophysiological research, as they allow collecting longitudinal and ecologically valid data. The present tutorial considers fidelity of physiological measurement with wearable sensors, focusing on reliability. We elaborate why ensuring reliability for wearables is…
Granville J. Matheson
Positron emission tomography (PET), along with many other fields of clinical research, is both timeconsuming and expensive, and recruitable patients can be scarce. These constraints limit the possibility of large-sample experimental designs, and often lead to statistically underpowered studies. This problem is…
Berna Devezer, Danielle J. Navarro, Joachim Vandekerckhove, Erkan Ozge Buzbas
Current attempts at methodological reform in sciences come in response to an overall lack of rigor in methodological and scientific practices in experimental sciences. However, some of these reform attempts suffer from the same mistakes and over-generalizations they purport to address. Considering the costs of allowing…
Erkan O. Buzbas, Berna Devezer, Bert Baumgaertner
The scientific reform movement has proposed openness as a potential remedy to the putative reproducibility or replication crisis. However, the conceptual relationship between openness, replication experiments, and results reproducibility has been obscure. We analyze the logical structure of experiments, define the…
Authors not listed
Step-by-step thinking is essential in all domains of chemical sciences and engineering. While machine learning tools are broadly used, algorithms that automate reasoning are far less common. We elaborate on seven categories of human reasoning activities and connect each to applications in chemical science and…