Open Access
2023 Structure learning via unstructured kernel-based M-estimation
Xin He, Yeheng Ge, Xingdong Feng
Author Affiliations +
Electron. J. Statist. 17(2): 2386-2415 (2023). DOI: 10.1214/23-EJS2153


In statistical learning, identifying underlying structures of true target functions based on observed data plays a crucial role to facilitate subsequent modeling and analysis. Unlike most of those existing methods that focus on some specific settings under certain model assumptions, a general and novel framework is proposed for recovering the true structures of target functions by using unstructured M-estimation in a reproducing kernel Hilbert space (RKHS) in this paper. This framework is inspired by the fact that gradient functions can be employed as a valid tool to learn underlying structures, including sparse learning, interaction selection and model identification, and it is easy to implement by taking advantage of some nice properties of the RKHS. More importantly, it admits a wide range of loss functions, and thus includes many commonly used methods as special cases, such as mean regression, quantile regression, likelihood-based classification, and margin-based classification, which is also computationally efficient by solving convex optimization tasks. The asymptotic results of the proposed framework are established within a rich family of loss functions without any explicit model specifications. The superior performance of the proposed framework is also demonstrated by a variety of simulated examples and a real case study.

Funding Statement

Xin He’s research is supported in part by NSFC-11901375, Shanghai Pujiang Program 2019PJC051, the Fundamental Research Funds for the Central Universities, and Program for Innovative Research Team of Shanghai University of Finance and Economics. Xingdong Feng’s research is supported in part by NSFC-11971292. This research is also supported by Shanghai Research Center for Data Science and Decision Technology.


The authors thank the associate editor and two anonymous referees for their constructive suggestions, which significantly improve this paper. The authors also thank Professor Junhui Wang for helpful and valuable comments on the initial draft of this work.


Download Citation

Xin He. Yeheng Ge. Xingdong Feng. "Structure learning via unstructured kernel-based M-estimation." Electron. J. Statist. 17 (2) 2386 - 2415, 2023.


Received: 1 June 2022; Published: 2023
First available in Project Euclid: 5 October 2023

MathSciNet: MR4650899
Digital Object Identifier: 10.1214/23-EJS2153

Primary: 62G05

Keywords: consistency , Convex optimization , gradient learning , ‎reproducing kernel Hilbert ‎space , structure learning

Vol.17 • No. 2 • 2023
Back to Top