11 papers · ranked by Valyu relevance
Caspar Schmitt, T. Kuhr
A workflow describes the entirety of processing steps in an analysis, such as employed in many fields of physics. Workflow management makes the dependencies between individual steps of a workflow and their computational requirements explicit, such that entire workflows can be executed in a stand-alone manner. Though…
Jan Janssen, Janine George, Julian Geiger, Marnik Bercx + 7 more
Numerous Workflow Management Systems (WfMS) have been developed in the field of computational materials science with different workflow formats, hindering interoperability and reproducibility of workflows in the field. To address this challenge, we introduce here the Python Workflow Definition (PWD) as a workflow…
Gregor von Laszewski, J. P. Fleischer, Geoffrey Fox
In this paper, we summarize our effort to create and utilize a simple framework to coordinate computational analytics tasks with the help of a workflow system. Our design is based on a minimalistic approach while at the same time allowing to access computational resources offered through the owner's computer, HPC…
Jan Janssen, Janine George, Julian Geiger, Marnik Bercx + 7 more
'Christina Ertural' 'Joerg Schaarschmidt' 'Alex M. Ganose' 'Giovanni Pizzi' 'Tilmann Hickel' 'Joerg Neugebauer'] Numerous Workflow Management Systems (WfMS) have been developed in the field of computational materials science with different workflow formats, hindering interoperability and reproducibility of workflows in…
Ask Hjorth Larsen, Mikael J. Kuisma, Tara M. Boland, Fredrik A. Nilsson + 1 more
'Fredrik A. Nilsson' 'Kristian S. Thygesen'] We introduce Taskblaster, a generic and lightweight Python framework for composing, executing, and managing computational workflows with automated error handling. Taskblaster supports dynamic workflows including flow control using branches and iteration, making the system…
Sebastian Pohl, Nourhan Elfaramawy, Kedi Cao, Birte Kehr + 1 more
'Matthias Weidlich'] Scientific workflows automate the analysis of large-scale scientific data, fostering the reuse of data processing operators as well as the reproducibility and traceability of analysis results. In exploratory research, however, workflows are continuously adapted, utilizing a wide range of tools and…
Nishchay Karle, Ben Clifford, Yadu Babuji, Ryan Chard + 2 more
'Daniel S. Katz' 'Kyle Chard'] Abstract—The Common Workflow Language (CWL) is a widely adopted language for defining and sharing computational workflows. It is designed to be independent of the execution engine on which workflows are executed. In this paper, we describe our experiences integrating CWL with Parsl, a…
Panchal, Deven
—Generative Agentic AI systems are emerging as a powerful paradigm for automating complex, multi-step tasks. However, many existing frameworks for building these systems introduce significant complexity, a steep learning curve, and substantial boilerplate code, hindering rapid prototyping and deployment. This paper…
Malwina Kotowicz, Magdalena Shumanska, Sven Fengler, Birgit Kurkowsky + 14 more
'Birgit Kurkowsky' 'Anja Meyer-Berhorn' 'Elisa Moretti' 'Josephine Blersch' 'Gisela Schmidt' 'Jakob Kreye' 'Scott van Hoof' 'Elisa Sánchez-Sendín' 'S. Momsen Reincke' 'Lars Krüger' 'Harald Prüß' 'Philip Denner' 'Eugenio Fava' 'Dominik Stappert' 'Bhanwar Lal Puniya'] Data management and sample tracking in complex…
Damien J. Mannion, Maria del Mar Quiroga, Jacob M. Paul, Marta I. Garrido
The processing of neuroimaging data typically involves a complicated set of operations, which often require different software packages and have intensive computational and storage demands. Although there are many options available for the neuroimaging researcher to establish their preferred set of processing…
Károly Bósa, Paul Heinzlreiter
Background Data preparation is a fundamental aspect of data engineering, a prerequisite for later tasks such as data visualization, reporting, and training machine learning models. Despite the recurring patterns in data transformation processes, the specific steps often vary depending on the project context, data…