Learning-Rate-Free Momentum SGD with Reshuffling Converges in Nonsmooth Nonconvex Optimization
Sign Up to like & getrecommendations! Published in 2024 at "Journal of Scientific Computing"
DOI: 10.1007/s10915-025-02798-0
Abstract: In this paper, we propose a generalized framework for developing learning-rate-free momentum stochastic gradient descent (SGD) methods in the minimization of nonsmooth nonconvex functions, especially in training nonsmooth neural networks. Our framework adaptively generates learning… read more here.
Keywords: rate free; sgd; learning rate; free momentum ... See more keywords