Optimal Nonparametric Inference with Two-Scale Distributional Nearest Neighbors
Prof. Jinchi Lv
Department Chair, Data Sciences and Operations
Kenneth King Stonier Chair in Business Administration
Professor of Data Sciences and Operations
University of Southern California
The weighted nearest neighbors (WNN) estimator has been popularly used as a flexible and easy-to-implement nonparametric tool for mean regression estimation. The bagging technique is an elegant way to form WNN estimators with weights automatically generated to the nearest neighbors; we name the resulting estimator as the distributional nearest neighbors (DNN) for easy reference. Yet, there is a lack of distributional results for such estimator, limiting its application to statistical inference. Moreover, when the mean regression function has higher-order smoothness, DNN does not achieve the optimal nonparametric convergence rate, mainly because of the bias issue. In this work, we provide an in-depth technical analysis of the DNN, based on which we suggest a bias reduction approach for the DNN estimator by linearly combining two DNN estimators with different subsampling scales, resulting in the novel two-scale DNN (TDNN) estimator. The two-scale DNN estimator has an equivalent representation of WNN with weights admitting explicit forms and some being negative. We prove that, thanks to the use of negative weights, the two-scale DNN estimator enjoys the optimal nonparametric rate of convergence in estimating the regression function under the fourth-order smoothness condition. We further go beyond estimation and establish that the DNN and two-scale DNN are both asymptotically normal as the subsampling scales and sample size diverge to infinity. For the practical implementation, we also provide variance estimators and a distribution estimator using the jackknife and bootstrap techniques for the two-scale DNN. These estimators can be exploited for constructing valid confidence intervals for nonparametric inference of the regression function. The theoretical results and appealing finite-sample performance of the suggested two-scale DNN method are illustrated with several simulation examples and a real data application.
This is a joint work with Emre Demirkaya, Yingying Fan, Lan Gao, Patrick Vossler and Jingbo Wang.
Jinchi Lv is Kenneth King Stonier Chair in Business Administration, Department Chair, Professor in Data Sciences and Operations Department of the Marshall School of Business at the University of Southern California, and Professor in Department of Mathematics at USC. He received his Ph.D. in Mathematics from Princeton University in 2007. He was McAlister Associate Professor in Business Administration at USC from 2016-2019. His research interests include statistics, machine learning, data science, business applications, and artificial intelligence and blockchain.
His papers have been published in journals in statistics, economics, business, computer science, information theory, neuroscience, and biology. He is the recipient of NSF Grant (2023), NSF Emerging Frontiers (EF) Grant (2022), Fellow of American Statistical Association (2020), NSF Grant (2020), Kenneth King Stonier Chair in Business Administration (2019), Fellow of Institute of Mathematical Statistics (2019), Member of USC University Committee on Appointments, Promotions, and Tenure (UCAPT, 2019-present), USC Marshall Dean’s Award for Research Impact (2017), Adobe Data Science Research Award (2017), McAlister Associate Professor in Business Administration (2016), Simons Foundation Grant (2016), the Royal Statistical Society Guy Medal in Bronze (2015), NSF Faculty Early Career Development (CAREER) Award (2010), USC Marshall Dean’s Award for Research Excellence (2009), Journal of the Royal Statistical Society Series B Discussion Paper (2008), NSF Grant (2008), and Zumberge Individual Award from USC’s James H. Zumberge Faculty Research and Innovation Fund (2008). He has served as an associate editor of Journal of the American Statistical Association (2023-present), Journal of Business & Economic Statistics (2018-present), The Annals of Statistics (2013-2018), and Statistica Sinica (2008-2016).
