27 papers · ranked by Valyu relevance
Dilpreet Singh, Chandan K Reddy
The primary purpose of this paper is to provide an in-depth analysis of different platforms available for performing big data analytics. This paper surveys different hardware platforms available for big data analytics and assesses the advantages and drawbacks of each of these platforms based on various metrics such as…
Renan Souza, Vitor Silva, Alexandre A. B. Lima, Daniel de Oliveira + 3 more
'Patrick Valduriez' 'Marta Mattoso' 'Daniel Katz'] Complex scientific experiments from various domains are typically modeled as workflows and executed on large-scale machines using a Parallel Workflow Management System (WMS). Since such executions usually last for hours or days, some WMSs provide user steering support…
Laszlo Gyongyosi, Sandor Imre
A scalable model for a distributed quantum computation is a challenging problem due to the complexity of the problem space provided by the diversity of possible quantum systems, from small-scale quantum devices to large-scale quantum computers. Here, we define a model of scalable distributed gate-model quantum…
Reinhardt, Steven P.
The scalable computing revolution of the late '80s through mid- '00s forged a new technical and economic model for computing that delivered massive societal impact, but its economic benefit has driven scalability to sizes that are now exhausting the energy grid's capacity. Our time demands a new revolution in scalable…
Łukasz P. Olech, Jan Kwiatkowski
In the recent years it can be observed increasing popularity of parallel processing using multi-core processors, local clusters, GPU and others. Moreover, currently one of the main requirements the IT users is the reduction of maintaining cost of the computer infrastructure. It causes that the performance evaluation of…
Tamas Foldi, Chris von Csefalvay, Nicolas A. Perez
The new barrier mode in Apache Spark allows embedding distributed deep learning training as a Spark stage to simplify the distributed training workflow. In Spark, a task in a stage doesn't depend on any other tasks in the same stage, and hence it can be scheduled independently. However, several algorithms require more…
Liang Chang, Xin Zhao, Jun Zhou, Simone Bianco + 2 more
'Jean Baptiste Thomas'] Deep neural networks have been deployed in various hardware accelerators, such as graph process units (GPUs), field-program gate arrays (FPGAs), and application specific integrated circuit (ASIC) chips. Normally, a huge amount of computation is required in the inference process, creating…
Chien-Ping Lu
At the dawn of Computing, Alan Turing proposed that instead of comprising many different specific machines, the computing machinery to make machines think should be a universal digital computer, modeled after human computers carrying out calculations with pencil on paper. Based on the belief that a digital computer…
Chaoming Wang, Xingsi Dong, Jiedong Jiang, Zilong Ji + 2 more
Whole-brain simulation stands as one of the most ambitious endeavors of our time, yet it remains constrained by significant technical challenges. A critical obstacle in this pursuit is the absence of a scalable online learning framework capable of supporting the efficient training of complex, diverse, and large-scale…
Yuanchao Zhang, Deanne M. Taylor
In single-cell RNA-seq (scRNA-seq) experiments, the number of individual cells has increased exponentially due to significant improvements on single-cell isolation and massively parallel sequencing technologies. However, computational methods have not scaled to the same order, presenting analytical challenges to the…
Yann Garniron, Thomas Applencourt, Kevin Gasperich, Anouar Benali + 15 more
Quantum Package is an open-source programming environment for quantum chemistry specially designed for wave function methods. Its main goal is the development of determinant-driven selected configuration interaction (sCI) methods and multi-reference second-order perturbation theory (PT2). The determinant-driven…
Martin Karp, Niclas Jansson, Philipp Schlatter, Stefano Markidis
As supercomputers' complexity has grown, the traditional boundaries between processor, memory, network, and accelerators have blurred, making a homogeneous computer model, in which the overall computer system is modeled as a continuous medium with homogeneously distributed computational power, memory, and data movement…
Sabuzima Nayak, Ripon Patgiri, Thoudam Doren Singh
This paper presents the overview of the current trends of Big data against the computing scenario from di fferent aspects. Some of the important aspect includes the Exascale, the computing power and the kind of applications which o ffer the Big data. This starts with the current computing hardware constraint against…
Tanveer Ahmad, Chengxin Ma, Zaid Al-Ars, H. Peter Hofstee
Current cluster scaled genomics data processing solutions rely on big data frameworks like Apache Spark, Hadoop and HDFS for data scheduling, processing and storage. These frameworks come with additional computation and memory overheads by default. It has been observed that scaling genomics dataset processing beyond 32…
Luka Skoric, Dan E. Browne, Kenton M. Barnes, Neil I. Gillespie + 1 more
'Earl T. Campbell'] Large-scale quantum computers have the potential to hold computational capabilities beyond conventional computers. However, the physical qubits are prone to noise which must be corrected in order to perform fault-tolerant quantum computations. Quantum Error Correction (QEC) provides the path for…
Seyoon Ko, Benjamin B. Chu, Daniel Peterson, Chidera Okenwa + 5 more
Admixture estimation plays a crucial role in ancestry inference and genomewide association studies (GWAS). Computer programs such as ADMIXTURE and STRUCTURE are commonly employed to estimate the admixture proportions of sample individuals. However, these programs can be overwhelmed by the computational burdens imposed…
Authors not listed
The era of exascale computing presents both exciting opportunities and unique challenges for quantum mechanical simulations. While the transition from petaflops to exascale computing has been marked by a steady increase in computational power, the shift towards heterogeneous architectures, particularly the dominant…
Tobias Weinzierl
> Summary. — Today's hardware's explosion of concurrency plus the explosion of data we build upon in both machine learning and scientific simulations have multifaceted impact on how we write our codes. They have changed our notion of performance and, hence, of what a good code is: Good code has, first of all, to be…
Parashar Dhapola, Johan Rodhe, Rasmus Olofzon, Thomas Bonald + 3 more
The increasing capacity to perform large-scale single-cell genomic experiments continues to outpace the computational requirements to efficiently handle growing datasets. Herein we present Scarf, a modularly designed Python package that seamlessly interoperates with other single-cell toolkits and allows for…
Francesco Versaci, Luca Pireddu, Gianluigi Zanetti
Modern sequencing machines produce order of a terabyte of data per day, which need subsequently to go through a complex processing pipeline. The standard workflow begins with a few independent, shared-memory tools, which communicate by means of intermediate files. Given the constant increase of the amount of data…
Authors not listed
This work establishes theoretical foundations for hierarchical quantum-classical algorithm design, where complex problems are decomposed across multiple spatial, temporal, or organizational scales with quantum and classical computation assigned to appropriate levels. We develop a mathematical framework that…
Authors not listed
We present a fast, asymptotically linear-scaling implementation of the perturbative quadruples energy correction in coupled-cluster theory using local natural orbitals. Our work follows the domain-based local pair natural orbital (DLPNO) approach previously applied to lower levels of excitations in coupled-cluster…
Justin Y. Shi
Like other engineering disciplines, software engineering should also have principles to guide the construction of sustainable computer applications. Tangible properties include a) unlimited scalability, b) maximal reproducibility, and c) optimizable energy efficiency. In practice, we expect a sustainable scientific…
Authors not listed
Machine learning models are transforming data-driven research across scientific disciplines, yet their deployment as accessible and reliable web services remains a significant challenge. We introduce the NERDD framework, a scalable, maintainable, and secure microservices platform designed to support the sustainable…
Yann Garniron, Thomas Applencourt, Kevin Gasperich, Anouar Benali + 15 more
Quantum Package is an open-source programming environment for quantum chemistry specially designed for wave function methods. Its main goal is the development of determinant-driven selected configuration interaction (sCI) methods and multi-reference second-order perturbation theory (PT2). The determinant-driven…
Martin Werner
This paper provides an abstract analysis of parallel processing strategies for spatial and spatio-temporal data. It isolates aspects such as data locality and computational locality as well as redundancy and locally sequential access as central elements of parallel algorithm design for spatial data. Furthermore, the…
Authors not listed
Self-driving laboratories (SDLs) promise accelerated scientific discovery and product development by closing the loop between robotic execution and AI/ML-driven decision making. In practice, however, SDL orchestration remains fragmented; workflows are typically encoded as laboratory-specific scripts or bespoke…