Open Access
October 1997 CART and best-ortho-basis: a connection
David L. Donoho
Ann. Statist. 25(5): 1870-1911 (October 1997). DOI: 10.1214/aos/1069362377

Abstract

We study what we call "dyadic CART" --a method of nonparametric regression which constructs a recursive partition by optimizing a complexity penalized sum of squares, where the optimization is over all recursive partitions arising from midpoint splits. We show that the method is adaptive to unknown degrees of anisotropic smoothness. Specifically, consider the anisotropic smoothness classes of Nikoľskii, consisting of bivariate functions $f(x_1, x_2)$ whose finite difference of distance h in direction i is bounded in $L^p$ norm by $Ch^{\delta_i}, i = 1, 2$. We show that dyadic CART, with an appropriate complexity penalty parameter $\lambda \sim \sigma^2 \cdot \Const \cdot \log (n)$, is within logarithmic terms of minimax over every anisotropic smoothness class $0 < C < \infty, 0 < \delta_1, \delta_2 \leq 1$.

The proof shows that dyadic CART is identical to a certain adaptive best-ortho-basis algorithm based on the library of all anisotropic Haar bases. Then it applies empirical basis selection ideas of Donoho and Johnstone. The basis empirically selected by dyadic CART is shown to be nearly as good as a basis ideally adapted to the underlying f. The risk of estimation in an ideally adapted anisotropic Haar basis is shown to be comparable to the minimax risk over anisotropic smoothness classes.

Underlying the success of this argument is harmonic analysis of anisotropic smoothness classes. We show that, for each anisotropic smoothness class, there is an anisotropic Haar basis which is a best orthogonal basis for representing that smoothness class; the basis is optimal not just within the library of anisotropic Haar bases, but among all orthogonal bases of $L^2 [0,1]^2$.

Citation

Download Citation

David L. Donoho. "CART and best-ortho-basis: a connection." Ann. Statist. 25 (5) 1870 - 1911, October 1997. https://doi.org/10.1214/aos/1069362377

Information

Published: October 1997
First available in Project Euclid: 20 November 2003

zbMATH: 0942.62044
MathSciNet: MR1474073
Digital Object Identifier: 10.1214/aos/1069362377

Subjects:
Primary: 41A30 , 62G07
Secondary: 41A25 , 62G20

Keywords: anisotropic Haar basis , anisotropic smoothness , best orthogonal basis , minimax estimation , Oracle inequalities , Spatial adaptation , Wavelets

Rights: Copyright © 1997 Institute of Mathematical Statistics

Vol.25 • No. 5 • October 1997
Back to Top