Beyond smoothness : Incorporating low-rank analysis into nonparametric density estimation

• Main Authors: VANDERMEULEN, Rob, LEDENT, Antoine
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2021
• Subjects: Density estimation; Low-rank methods; Tensor methods; Tucker decomposition; statistical guarantees; bias-variance analysis.; Theory and Algorithms
• Online Access: https://ink.library.smu.edu.sg/sis_research/7205; https://ink.library.smu.edu.sg/context/sis_research/article/8208/viewcontent/Low_rank.pdf
• Institution: Singapore Management University

The construction and theoretical analysis of the most popular universally consistent nonparametric density estimators hinge on one functional property: smoothness. In this paper we investigate the theoretical implications of incorporating a multi-view latent variable model, a type of low-rank model, into nonparametric density estimation. To do this we perform extensive analysis on histogram-style estimators that integrate a multi-view model. Our analysis culminates in showing that there exists a universally consistent histogram-style estimator that converges to any multi-view model with a finite number of Lipschitz continuous components at a rate of ˜O(1/3√n) in L1 error. In contrast, the standard histogram estimator can converge at a rate slower than 1/d√n on the same class of densities. We also introduce a new nonparametric latent variable model based on the Tucker decomposition. A rudimentary implementation of our estimators experimentally demonstrates a considerable performance improvement over the standard histogram estimator. We also provide a thorough analysis of the sample complexity of our Tucker decomposition-based model and a variety of other results. Thus, our paper provides solid theoretical foundations for extending low-rank techniques to the nonparametric setting.