原文传递
Statistical Inference Using Stochastic Gradient Descent: Volumes 1 and 2.
| 题名： | Statistical Inference Using Stochastic Gradient Descent: Volumes 1 and 2. |
| 作者： | Juri, N. R.; Boyles, S. D.; Zhu, T.; Perrine, K.; Chen, A.; Li, Y. |
| 关键词： | Statistical analysis, Stochastic Gradient Descent (SGD), Transportation research, Computation, Texas (state), Ornstein-Uhlenbeck processes |
| 摘要： | Volume 1: We present a novel inference framework for convex empirical risk minimization, using approximate stochastic Newton steps. The proposed algorithm is based on the notion of finite differences and allows the approximation of a Hessian-vector product from first-order information. In theory, our method efficiently computes the statistical error covariance in M-estimation, both for unregularized convex learning problems and high-dimensional LASSO regression, without using exact second order information, or resampling the entire data set. In practice, we demonstrate the effectiveness of our framework on large-scale machine learning problems, that go even beyond convexity: as a highlight, our work can be used to detect certain adversarial attacks on neural networks. Volume 2: We present a novel method for frequentest statistical inference in M-estimation problems, based on stochastic gradient descent (SGD) with a fixed step size: we demonstrate that the average of such SGD sequences can be used for statistical inference, after proper scaling. An intuitive analysis using the Ornstein-Uhlenbeck process suggests that such averages are asymptotically normal. From a practical perspective, our SGD-based inference procedure is a first order method, and is well-suited for large scale problems. To show its merits, we apply it to both synthetic and real datasets, and demonstrate that its accuracy is comparable to classical statistical methods, while requiring potentially far less computation. |
| 报告类型： | 科技报告 |
相关文献
- Probabilistic and Statistical Analysis of Complex Stochastic Networks.
- A Study of the Feasibility of Eliminating the 2 1/2 Ton Payload Truck Class
- Sampling-Based RBDO Using Stochastic Sensitivity and Dynamic Kriging for Broader Army Applications.
- Numerical Modeling of Fire Suppression Using Water Mist. 2. An Optimization Study on Jet Diffusion Flames
- VMS Strategy Development and Evaluating Driver Response
- Final Report: Studies in Structural, Stochastic and Statistical Reliability for Communication Networks and Engineered Systems.
检索历史
应用推荐
