11 papers · ranked by Valyu relevance
Andrea Mari, Thomas R. Bromley, Nathan Killoran
For a large class of variational quantum circuits, we show how arbitrary-order derivatives can be analytically evaluated in terms of simple parameter-shift rules, i.e., by running the same circuit with different shifts of the parameters. As particular cases, we obtain parameter-shift rules for the Hessian of an…
Mahsa Soheil Shamaee, Sajad Fathi Hafshejani
This paper introduces a novel approach to enhance the performance of the stochastic gradient descent (SGD) algorithm by incorporating a modified decay step size based on √ 1 t . The proposed step size integrates a logarithmic term, leading to the selection of smaller values in the final iterations. Our analysis…
Michael Hintermüller, Kostas Papafitsoros, Carlos N. Rautenberg
> Abstract. We consider a mollifying operator with variable step that, in contrast to the standard mollification, is able to preserve the boundary values of functions. We prove boundedness of the operator in all basic Lebesgue, Sobolev and BV spaces as well as corresponding approximation results. The results are then…
Natalia de Castro, María A. Garrido-Vizuete, Rafael Robles, María Trinidad Villar-Liñán
'María Trinidad Villar-Liñán'] In this work we present the notion of greyscale of a graph as a colouring of its vertices that uses colours from the real interval [0,1]. Any greyscale induces another colouring by assigning to each edge the non-negative difference between the colours of its vertices. These edge colours…
Camille Castera, Jérôme Bolte, Cédric Févotte, Edouard Pauwels
In view of a direct and simple improvement of vanilla SGD, this paper presents a fine-tuning of its step-sizes in the mini-batch case. For doing so, one estimates curvature, based on a local quadratic model and using only noisy gradient approximations. One obtains a new stochastic first-order method (Step-Tuned SGD)…
Thomas Degris, Khurram Javed, Arsalan Sharifnassab, Yuxin Liu + 1 more
'Richard Sutton'] In continual learning, a learner has to keep learning from the data over its whole life time. A key issue is to decide what knowledge to keep and what knowledge to let go. In a neural network, this can be implemented by using a step-size vector to scale how much gradient samples change network…
Miikka Silfverberg, Francis M. Tyers, Garrett Nicolai, Mans Hulden
Sequence-to-sequence models have delivered impressive results in word formation tasks such as morphological inflection, often learning to model subtle morphophonological details with limited training data. Despite the performance, the opacity of neural models makes it difficult to determine whether complex…
Mário B. Amaro
This work introduces an approach to variable-step Finite Difference Method (FDM) where nonuniform meshes are generated via a weight function, which establishes a diffeomorphism between uniformly spaced computational coordinates and variably spaced physical coordinates. We then derive finite difference approximations…
Jiawei Zhang
In this paper, we aim at providing an introduction to the gradient descent based optimization algorithms for learning deep neural network models. Deep learning models involving multiple nonlinear projection layers are very challenging to train. Nowadays, most of the deep learning model training still relies on the back…
Zhaoyi Li, Hong-lin Liao
We prove that the two-step backward differentiation formula (BDF2) method is stable on arbitrary time grids; while the variable-step BDF3 scheme is stable if almost all adjacent step ratios are less than 2.553. These results relax the severe mesh restrictions in the literature and provide a new understanding of…
Simon Weißmann, Jakob Zech
stability and multilevel approximation Authors: ['Simon Weißmann' 'Jakob Zech'] In this paper we propose and analyze a novel multilevel version of Stein variational gradient descent (SVGD). SVGD is a recent particle based variational inference method. For Bayesian inverse problems with computationally expensive…