Resolving learning rates adaptively by locating stochastic non-negative associated gradient projection points using line searches
Sign Up to like & getrecommendations! Published in 2021 at "Journal of Global Optimization"
DOI: 10.1007/s10898-020-00921-z
Abstract: Learning rates in stochastic neural network training are currently determined a priori to training, using expensive manual or automated iterative tuning. Attempts to resolve learning rates adaptively, using line searches, have proven computationally demanding. Reducing… read more here.
Keywords: rates adaptively; line; learning rates; line searches ... See more keywords
Learning rates and cost reduction potential of indirect coal-to-liquid technology coupled with CO2 capture
Sign Up to like & getrecommendations! Published in 2018 at "Energy"
DOI: 10.1016/j.energy.2018.09.150
Abstract: Abstract Coal-to-liquids (CTL) and CO2 capture and storage (CCS) have attracted increasing attention in energy supply systems, but few empirical studies and industrial data are available regarding the learning rates and future cost curves. In… read more here.
Keywords: coupled co2; co2 capture; technology coupled; learning rates ... See more keywords
An effect of serotonergic stimulation on learning rates for rewards apparent after long intertrial intervals
Sign Up to like & getrecommendations! Published in 2018 at "Nature Communications"
DOI: 10.1038/s41467-018-04840-2
Abstract: Serotonin has widespread, but computationally obscure, modulatory effects on learning and cognition. Here, we studied the impact of optogenetic stimulation of dorsal raphe serotonin neurons in mice performing a non-stationary, reward-driven decision-making task. Animals showed… read more here.
Keywords: trial; serotonergic stimulation; learning rates; stimulation ... See more keywords
Deep Filtering With Adaptive Learning Rates
Sign Up to like & getrecommendations! Published in 2023 at "IEEE Transactions on Automatic Control"
DOI: 10.1109/tac.2022.3183147
Abstract: This article develops a new deep learning framework for general nonlinear filtering. Our main contribution is to present a computationally feasible procedure. The proposed algorithms have the capability of dealing with challenging (infinitely dimensional) filtering… read more here.
Keywords: network; filtering adaptive; learning rates; learning ... See more keywords
Incremental PID Controller-Based Learning Rate Scheduler for Stochastic Gradient Descent.
Sign Up to like & getrecommendations! Published in 2022 at "IEEE transactions on neural networks and learning systems"
DOI: 10.1109/tnnls.2022.3213677
Abstract: As we all know, the learning rate plays a vital role in deep neural network (DNN) training. This study introduces an incremental proportional-integral-derivative (PID) controller widely used in automatic control as a learning rate scheduler… read more here.
Keywords: incremental pid; pid controller; learning rate; learning rates ... See more keywords
Learning Rates for Stochastic Gradient Descent With Nonconvex Objectives
Sign Up to like & getrecommendations! Published in 2021 at "IEEE Transactions on Pattern Analysis and Machine Intelligence"
DOI: 10.1109/tpami.2021.3068154
Abstract: Stochastic gradient descent (SGD) has become the method of choice for training highly complex and nonconvex models since it can not only recover good solutions to minimize training errors but also generalize well. Computational and… read more here.
Keywords: nonconvex; stochastic gradient; learning rates; gradient descent ... See more keywords
Learning rates for regularized least squares ranking algorithm
Sign Up to like & getrecommendations! Published in 2017 at "Analysis and Applications"
DOI: 10.1142/s0219530517500063
Abstract: The ranking problem aims at learning real-valued functions to order instances, which has attracted great interest in statistical learning theory. In this paper, we consider the regularized least squares ranking algorithm within the framework of… read more here.
Keywords: least squares; learning rates; ranking algorithm; rates regularized ... See more keywords
Learning Rates for Classification with Gaussian Kernels
Sign Up to like & getrecommendations! Published in 2017 at "Neural Computation"
DOI: 10.1162/neco_a_00968
Abstract: This letter aims at refined error analysis for binary classification using support vector machine (SVM) with gaussian kernel and convex loss. Our first result shows that for some loss functions, such as the truncated quadratic… read more here.
Keywords: classification gaussian; learning rates; rates classification; svm gaussian ... See more keywords
Aerial Scene Classification through Fine-Tuning with Adaptive Learning Rates and Label Smoothing
Sign Up to like & getrecommendations! Published in 2020 at "Applied Sciences"
DOI: 10.3390/app10175792
Abstract: Remote Sensing (RS) image classification has recently attracted great attention for its application in different tasks, including environmental monitoring, battlefield surveillance, and geospatial object detection. The best practices for these tasks often involve transfer learning… read more here.
Keywords: classification; image; fine tuning; learning rates ... See more keywords
Stochastic Control for Bayesian Neural Network Training
Sign Up to like & getrecommendations! Published in 2022 at "Entropy"
DOI: 10.3390/e24081097
Abstract: In this paper, we propose to leverage the Bayesian uncertainty information encoded in parameter distributions to inform the learning procedure for Bayesian models. We derive a first principle stochastic differential equation for the training dynamics… read more here.
Keywords: neural network; bayesian neural; learning rates; control ... See more keywords