## Bernoulli

• Bernoulli
• Volume 25, Number 1 (2019), 549-583.

### Limit properties of the monotone rearrangement for density and regression function estimation

#### Abstract

The monotone rearrrangement algorithm was introduced by Hardy, Littlewood and Pólya as a sorting device for functions. Assuming that $x$ is a monotone function and that an estimate $x_{n}$ of $x$ is given, consider the monotone rearrangement $\hat{x}_{n}$ of $x_{n}$. This new estimator is shown to be uniformly consistent as soon as $x_{n}$ is. Under suitable assumptions, pointwise limit distribution results for $\hat{x}_{n}$ are obtained. The framework is general and allows for weakly dependent and long range dependent stationary data. Applications in monotone density and regression function estimation are detailed. Asymptotics for rearrangement estimators with vanishing derivatives are also obtained in these two contexts.

#### Article information

Source
Bernoulli, Volume 25, Number 1 (2019), 549-583.

Dates
Revised: October 2017
First available in Project Euclid: 12 December 2018

https://projecteuclid.org/euclid.bj/1544605256

Digital Object Identifier
doi:10.3150/17-BEJ998

Mathematical Reviews number (MathSciNet)
MR3892329

Zentralblatt MATH identifier
07007217

#### Citation

Anevski, Dragi; Fougères, Anne-Laure. Limit properties of the monotone rearrangement for density and regression function estimation. Bernoulli 25 (2019), no. 1, 549--583. doi:10.3150/17-BEJ998. https://projecteuclid.org/euclid.bj/1544605256

#### References

• [1] Anevski, D. and Fougères, A.-L. (2017). Supplement to “Limit properties of the monotone rearrangement for density and regression function estimation.” DOI:10.3150/17-BEJ998SUPP.
• [2] Anevski, D. and Hössjer, O. (2006). A general asymptotic scheme for inference under order restrictions. Ann. Statist. 34 1874–1930.
• [3] Anevski, D. and Soulier, P. (2011). Monotone spectral density estimation. Ann. Statist. 39 418–438.
• [4] Billingsley, P. (1968). Convergence of Probability Measures. New York: Wiley.
• [5] Birke, M. and Dette, H. (2007). Estimating a convex function in nonparametric regression. Scand. J. Stat. 34 384–404.
• [6] Birke, M. and Dette, H. (2008). A note on estimating a smooth monotone regression by combining kernel and density estimates. J. Nonparametr. Stat. 20 679–691.
• [7] Birke, M., Neumeyer, N. and Volgushev, S. (2013). The independence process in conditional quantile location-scale models and an application to testing for monotonicity. Submitted.
• [8] Birke, M., Neumeyer, N. and Volgushev, S. (2016). The independence process in conditional quantile location-scale models and an application to testing for monotonicity. Available at arXiv:1609.07696.
• [9] Brunk, H.D. (1958). On the estimation of parameters restricted by inequalities. Ann. Math. Stat. 29 437–454.
• [10] Chernozhukov, V., Fernandez-Val, I. and Galichon, A. (2007). Improving point and interval estimates of monotone functions by rearrangement. Biometrika 96 559–575.
• [11] Chernozhukov, V., Fernandez-Val, I. and Galichon, A. (2010). Quantile and probability curves without crossing. Econometrica 78 1093–1125.
• [12] Csőrgő, S. and Mielniczuk, J. (1995). Density estimation under long-range dependence. Ann. Statist. 23 990–999.
• [13] Dette, H., Neumeyer, N. and Pilz, K.F. (2006). A simple nonparametric estimator of a strictly monotone regression function. Bernoulli 12 469–490.
• [14] Dette, H. and Pilz, K.F. (2006). A comparative study of monotone nonparametric kernel estimates. J. Stat. Comput. Simul. 76 41–56.
• [15] Eggermont, P.P.B. and LaRiccia, V.N. (2000). Maximum likelihood estimation of smooth monotone and unimodal densities. Ann. Statist. 28 922–947.
• [16] Fougères, A.-L. (1997). Estimation de densités unimodales. Canad. J. Statist. 25 375–387.
• [17] Gasser, T. and Muller, H.-G. (1984). Estimating regression functions and their derivatives by the kernel method. Scand. J. Stat. 11 171–185.
• [18] Gijbels, I. (2004). Monotone regression. Discussion paper No. 0334. Institut de Statistique, Université Catholique de Louvain.
• [19] Grenander, U. (1956). On the theory of mortality measurements. Part II. Skand. Aktuarietidskr. 39.
• [20] Hall, P. and Huang, L.-S. (2001). Nonparametric kernel regression subject to monotonicity constraints. Ann. Statist. 29 624–647.
• [21] Hall, P. and Kang, K.-H. (2005). Unimodal kernel density estimation by data sharpening. Statist. Sinica 15 73–98.
• [22] Hardy, G.H., Littlewood, J.E. and Pólya, G. (1952). Inequalities, 2nd ed. Cambridge: Cambridge University Press.
• [23] Jankowski, H.K. and Wellner, J.A. (2009). Estimation of a discrete monotone distribution. Electron. J. Stat. 3 1567–1605.
• [24] Lieb, E.H. and Loss, M. (1996). Analysis. Graduate Studies in Mathematics 14. Providence, RI: Amer. Math. Soc.
• [25] Mammen, E. (1991). Nonparametric regression under qualitative smoothness assumptions. Ann. Statist. 19 741–759.
• [26] Mammen, E., Marron, J.S., Turlach, B.A. and Wand, M.P. (2001). A general projection framework for constrained smoothing. Statist. Sci. 16 232–248.
• [27] Meyer, M.C. and Woodroofe, M. (2004). Consistent maximum likelihood estimation of a unimodal density using shape restrictions. Canad. J. Statist. 32 85–100.
• [28] Mukerjee, H. (1988). Monotone nonparametric regression. Ann. Statist. 16 741–750.
• [29] Neumeyer, N. (2007). A note on uniform consistency of monotone function estimators. Statist. Probab. Lett. 77 693–703.
• [30] Polonik, W. (1995). Density estimation under qualitative assumptions in higher dimensions. J. Multivariate Anal. 55 61–81.
• [31] Polonik, W. (1998). The silhouette, concentration functions and ML-density estimation under order restrictions. Ann. Statist. 26 1857–1877.
• [32] Prakasa Rao, B.L.S. (1969). Estimation of a unimodal density. Sankhya, Ser. A 31 23–36.
• [33] Ramsay, J.O. (1988). Monotone regression splines in action (with comments). Statist. Sci. 3 425–461.
• [34] Schwartz, L. (1966). Théorie des Distributions. Publications de l’Institut de Mathématique de l’Université de Strasbourg, No. IX-X. Nouvelle Édition, Entiérement Corrigée, Refondue et Augmentée. Paris: Hermann.
• [35] Taqqu, M.S. (1974/75). Weak convergence to fractional Brownian motion and to the Rosenblatt process. Z. Wahrsch. Verw. Gebiete 31 287–302.
• [36] Taqqu, M.S. (1979). Convergence of integrated processes of arbitrary Hermite rank. Z. Wahrsch. Verw. Gebiete 50 53–83.
• [37] van Es, B., Jongbloed, G. and van Zuijlen, M. (1998). Isotonic inverse estimators for nonparametric deconvolution. Ann. Statist. 26 2395–2406.
• [38] van der Vaart, A.W. and van der Laan, M.J. (2003). Smooth estimation of a monotone density. Statistics 37 189–203.
• [39] Villani, C. (2003). Topics in Optimal Transportation. Graduate Studies in Mathematics 58. Providence, RI: Amer. Math. Soc.
• [40] Volguchev, S. (2013). Smoothed quantile regression processes for binary response models. Available at arXiv:1302.5644.
• [41] Volguchev, S., Birke, M., Dette, H. and Neumeyer, N. (2013). Significance testing in quantile regression. Electron. J. Stat. 7 105–145.
• [42] Volgushev, S. and Dette, H. (2013). Nonparametric quantile regression for twice censored data. Bernoulli 19 748–779.
• [43] Wand, M.P. and Jones, M.C. (1995). Kernel Smoothing. Monographs on Statistics and Applied Probability 60. London: Chapman & Hall.
• [44] Wright, F.T. (1981). The asymptotic behaviour of monotone regression estimates. Ann. Statist. 9 443–448.

#### Supplemental materials

• Supplement to “Limit properties of the monotone rearrangement for density and regression function estimation”. The supplemental article Anevski and Fougères [1] provides in a first section some technical results on maximal bounds for the rescaled version partial sum and empirical process; it gives further conditions under which Assumption 2 holds, with application to the density and regression function estimation cases, stated in Appendix B, as well as all proofs. Furthermore, Section 2 of Anevski and Fougères [1] contains a simulation study that illustrates the finite sample behaviour of our estimator, and compare it to other estimators that are considered in the paper of Birke and Dette [6].