11 papers · ranked by Valyu relevance
Yueming Hao, Xu Zhao, Bin Bao, David Berard + 3 more
'Adnan Aziz' 'Xu Liu'] Deep learning (DL) has been a revolutionary technique in various domains. To facilitate the model development and deployment, many deep learning frameworks are proposed, among which Py-Torch is one of the most popular solutions. The performance of ecosystem around PyTorch is critically important…
Abhishek Ghosh, Ajay Nayak, Ashish Panwar, Arkaprava Basu
CUDA Graphs — a recent hardware feature introduced for NVIDIA GPUs — aim to reduce CPU launch overhead by capturing and launching a series of GPU tasks (kernels) as a DAG. However, deploying CUDA Graphs faces several challenges today due to the static structure of a graph. It also incurs performance overhead due to…
Alawi, Zakariya Ba
—This paper presents a comprehensive comparative survey of TensorFlow and PyTorch, the two leading deep learning frameworks, focusing on their usability, performance, and deployment trade-offs. We review each framework's programming paradigm and developer experience, contrasting TensorFlow's graph-based (now optionally…
Marten Lienen, Stephan Günnemann
We introduce an ODE solver for the PyTorch ecosystem that can solve multiple ODEs in parallel independently from each other while achieving significant performance gains. Our implementation tracks each ODE's progress separately and is carefully optimized for GPUs and compatibility with PyTorch's JIT compiler. Its…
Ho Young Jhoo, Sehoon Kim, Woosung Song, Kyuyeon Park + 2 more
'Dong-Kwon Lee' 'Kwangkeun Yi'] We present an automatic static analyzer PyTea that detects tensorshape errors in PyTorch code. The tensor-shape error is critical in the deep neural net code; much of the training cost and intermediate results are to be lost once a tensor shape mismatch occurs in the midst of the…
Nacime Bouziani, David A. Ham
Partial differential equations (PDEs) are central to describing and modelling complex physical systems that arise in many disciplines across science and engineering. However, in many realistic applications PDE modelling provides an incomplete description of the physics of interest. PDE-based machine learning techniques…
Corey Adams, Peter Harrington, Akshay Subramaniam, Mohammad Shoaib Abbas + 3 more
Scientific Machine Learning (SciML) faces unique challenges for extreme-resolution data, with mitigations that often fail to scale or degrade the accuracy of trained models. While some specialized methods have achieved remarkable results in training models or performing inference on massive spatial datasets with…
Albert Bou, Matteo Bettini, Sebastian Dittert, Vikash Kumar + 4 more
'Shagun Sodhani' 'Xiaomeng Yang' 'Gianni De Fabritiis' 'Vincent Moens'] PyTorch has ascended as a premier machine learning framework, yet it lacks a native and comprehensive library for decision and control tasks suitable for large development teams dealing with complex real-world data and environments. To address this…
Thomas Bartz–Beielstein
The goal of hyperparameter tuning (or hyperparameter optimization) is to optimize the hyperparameters to improve the performance of the machine or deep learning model. spotPython ("Sequential Parameter Optimization Toolbox in Python") is the Python version of the well-known hyperparameter tuner SPOT, which has been…
Peiyu Zang, Bosen Xie, Ruoxiang Xu, Yongqiang Cai
Physics-Informed Neural Networks (PINNs) solve PDEs by incorporating physical constraints into neural-network training, but large-scale problems are limited by automatic-differentiation memory overhead and inefficient execution of grid-based PDE operators. We present FlashPDE, a drop-in fused operator library for…
Marissa Dominijanni
This paper introduces Inferno, a software library built on top of PyTorch that is designed to meet distinctive challenges of using spiking neural networks (SNNs) for machine learning tasks. We describe the architecture of Inferno and key differentiators that make it uniquely well-suited to these tasks. We show how…