## The Annals of Applied Statistics

### Population counts along elliptical habitat contours: Hierarchical modeling using Poisson-lognormal mixtures with nonstationary spatial structure

#### Abstract

Ecologists often interpret variation in the spatial distribution of populations in terms of responses to environmental features, but disentangling the effects of individual variables can be difficult if latent effects and spatial and temporal correlations are not accounted for properly. Here, we use hierarchical models based on a Poisson-lognormal mixture to understand the spatial variation in relative abundance (counts per standardized unit of effort) of yellow perch, Perca flavescens, the most abundant fish species in Lake Saint Pierre, Quebec, Canada. The mixture incorporates spatially varying environmental covariates that represent local habitat characteristics, and random temporal and spatial effects that capture the effects of unobserved ecological processes. The sampling design covers the margins but not the central region of the lake. We fit spatial generalized linear mixed models based on three different prior covariance structures for the local latent effects: a single Gaussian process (GP) over the lake, a GP over a circle, and independent GP for each shore. The models allow for independence, isotropy, or nonstationary spatial effects. Nonstationarity is dealt with using two different approaches, geometric anisotropy and the inclusion of covariates in the correlation structure of the latent spatial process. The proposed approaches for specification of spatial domain and choice of Gaussian process priors may prove useful in other applications that involve spatial correlation along an irregular contour or in discontinuous spatial domains.

#### Article information

Source
Ann. Appl. Stat., Volume 9, Number 3 (2015), 1372-1393.

Dates
Revised: May 2015
First available in Project Euclid: 2 November 2015

https://projecteuclid.org/euclid.aoas/1446488743

Digital Object Identifier
doi:10.1214/15-AOAS838

Mathematical Reviews number (MathSciNet)
MR3418727

Zentralblatt MATH identifier
06525990

#### Citation

Schmidt, Alexandra M.; Rodríguez, Marco A.; Capistrano, Estelina S. Population counts along elliptical habitat contours: Hierarchical modeling using Poisson-lognormal mixtures with nonstationary spatial structure. Ann. Appl. Stat. 9 (2015), no. 3, 1372--1393. doi:10.1214/15-AOAS838. https://projecteuclid.org/euclid.aoas/1446488743

#### References

• Bertolo, A., Blanchet, A. F. G., Magnan, P., Brodeur, P., Mingelbier, M. and Legendre, P. (2012). Inferring processes from spatial patterns: The role of directional and non-directional forces in shaping fish larvae distribution in a freshwater lake system. PLoS ONE 7 1–11.
• Bulmer, M. G. (1974). On fitting the Poisson lognormal distribution to species-abundance data. Biometrics 30 101–110.
• Clark, J. S. and Gelfand, A. E. (2006). Hierarchical Modelling for the Environmental Sciences: Statistical Methods and Applications. Oxford Univ. Press, Oxford, UK.
• Czado, C., Gneiting, T. and Held, L. (2009). Predictive model assessment for count data. Biometrics 65 1254–1261.
• Denison, D. G. T. and Mallick, B. K. (1998). Discussion of model-based geostatistics. Applied Statistics 47 336.
• Diggle, P. J. and Ribeiro, P. J. Jr. (2007). Model-Based Geostatistics. Springer, New York.
• Diggle, P. J., Tawn, J. A. and Moyeed, R. A. (1998). Model-based geostatistics. J. R. Stat. Soc. Ser. C. Appl. Stat. 47 299–350.
• Doornik, J. (2007). Object-Oriented Matrix Programming Using Ox, 3rd ed. Timberlake Consultants Press and Oxford, London.
• Gamerman, D. (1997). Sampling from the posterior distribution in generalized linear mixed models. Stat. Comput. 7 57–68.
• Glémet, H. and Rodríguez, M. A. (2007). Short-term growth (RNA/DNA ratio) of yellow perch (Perca flavescens) in relation to environmental influence and spatio-temporal variation in a shallow fluvial lake. Canadian Journal of Fisheries and Aquatic Sciences 64 1646–1655.
• Gneiting, T. (2013). Strictly and non-strictly positive definite functions on spheres. Bernoulli 19 1327–1349.
• Gneiting, T. and Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. J. Amer. Statist. Assoc. 102 359–378.
• Gschlößl, S. and Czado, C. (2008). Modelling count data with overdispersion and spatial effects. Statist. Papers 49 531–552.
• Guttorp, P. and Schmidt, A. M. (2013). Covariance structure of spatial and spatio-temporal processes. WIREs Computational Statistics 5 279–287.
• Hanks, E. M., Schliep, E. M., Hooten, M. B. and Hoeting, J. A. (2015). Restricted spatial regression in practice: Geostatistical models, confounding, and robustness under model misspecification. Environmetrics 26 243–254.
• Hastings, W. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57 97–109.
• Hodges, J. S. and Reich, B. J. (2010). Adding spatially-correlated errors can mess up the fixed effect you love. Amer. Statist. 64 325–334.
• Hudon, C. (1997). Impact of water level fluctuations on St. Lawrence River aquatic vegetation. Canadian Journal of Fisheries and Aquatic Sciences 54 2853–2865.
• Hughes, J. and Haran, M. (2013). Dimension reduction and alleviation of confounding for spatial generalized linear mixed models. J. R. Stat. Soc. Ser. B. Stat. Methodol. 75 139–159.
• Ingebrigtsen, R., Lindgren, F. and Steinsland, I. (2014). Spatial models with explanatory variables in the dependence structure. Spat. Stat. 8 20–38.
• Legates, D. R. (1991). The effect of domain shape on principal components analyses. International Journal of Climatology 11 135–146.
• Lindgren, F., Rue, H. and Lindström, J. (2011). An explicit link between Gaussian fields and Gaussian Markov random fields: The stochastic partial differential equation approach. J. R. Stat. Soc. Ser. B. Stat. Methodol. 73 423–498.
• Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H. and Teller, E. (1953). Equation of state calculations by fast computing machines. Journal of Chemical Physics 21 1087–1092.
• Miller, D. L. and Wood, S. N. (2014). Finite area smoothing with generalized distance splines. Environ. Ecol. Stat. 21 715–731.
• Paciorek, C. J. (2010). The importance of scale for spatial-confounding bias and precision of spatial regression estimators. Statist. Sci. 25 107–125.
• Plummer, M., Best, N., Cowles, K. and Vines, K. (2006). CODA: Convergence diagnosis and output analysis for MCMC. R News 6 7–11.
• Poppick, A. and Stein, M. L. (2014). Using covariates to model dependence in nonstationary, high-frequency meteorological processes. Environmetrics 25 293–305.
• Ramsay, T. (2002). Spline smoothing over difficult regions. J. R. Stat. Soc. Ser. B. Stat. Methodol. 64 307–319.
• Reich, B. J., Hodges, J. S. and Zadnik, V. (2006). Effects of residual smoothing on the posterior of the fixed effects in disease-mapping models. Biometrics 62 1197–1206.
• Roberts, G. O. and Rosenthal, J. S. (2009). Examples of adaptive MCMC. J. Comput. Graph. Statist. 18 349–367.
• Royle, J. A. (2004). $N$-mixture models for estimating population size from spatially replicated counts. Biometrics 60 108–115.
• Sampson, P. and Guttorp, P. (1992). Nonparametric estimation of nonstationary spatial covariance structure. J. Amer. Statist. Assoc. 87 108–119.
• Schmidt, A. M., Guttorp, P. and O’Hagan, A. (2011). Considering covariates in the covariance structure of spatial processes. Environmetrics 22 487–500.
• Schmidt, A. M. and Rodríguez, M. A. (2011a). Modelling multivariate counts varying continuously in space. In Bayesian Statistics 9 (J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith and M. West, eds.) 611–629. Oxford Univ. Press, Oxford.
• Schmidt, A. M. and Rodríguez, M. A. (2011b). Reply to the discussion of Boys, Farrow, and Germain. In Bayesian Statistics 9 (J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith and M. West, eds.) 630–638. Oxford Univ. Press, Oxford.
• Schmidt, A. M., Rodríguez, M. A. and Capistrano, E. S. (2015). Supplement to “Population counts along elliptical habitat contours: Hierarchical modeling using Poisson-lognormal mixtures with nonstationary spatial structure.” DOI:10.1214/15-AOAS838SUPP.
• Scott, W. B. and Crossman, E. J. (1973). Freshwater Fishes of Canada. Fisheries Research Board of Canada, Bulletin 184, Ottawa, Canada.
• Soubeyrand, S., Enjalbert, J. and Sache, I. (2008). Accounting for roughness of circular processes: Using Gaussian random processes to model the anisotropic spread of airborne plant disease. Theor. Popul. Biol. 73 92–103.
• Spiegelhalter, D. J., Best, N. G., Carlin, B. P. and van der Linde, A. (2002). Bayesian measures of model complexity and fit. J. R. Stat. Soc. Ser. B. Stat. Methodol. 64 583–639.
• Wikle, C. K. (2003). Hierarchical models in environmental science. Int. Stat. Rev. 71 181–199.
• Wikle, C. K. (2010). Hierarchical modeling with spatial data. In Handbook of Spatial Statistics (A. Gelfand, P. Diggle, M. Fuentes and P. Guttorp, eds.). Chapman & Hall/CRC Handb. Mod. Stat. Methods 89–106. CRC Press, Boca Raton, FL.
• Williams, C. K. I. (1998). Discussion of model-based geostatistics. Applied Statistics 47 342.
• Wood, S. N. (2006). Generalized Additive Models: An Introduction with $R$. Chapman & Hall/CRC, Boca Raton, FL.
• Wood, S. N., Bravington, M. V. and Hedley, S. L. (2008). Soap film smoothing. J. R. Stat. Soc. Ser. B. Stat. Methodol. 70 931–955.
• Yaglom, A. M. (1987). Correlation Theory of Stationary and Related Random Functions. Vol. I: Basic Results. Springer, New York.

#### Supplemental materials

• Additional results for “Population counts along elliptical habitat contours: Hierarchical modeling using Poisson-lognormal mixtures with nonstationary spatial structure”. This supplement contains four sections which provide further results on: (1) circular transformations, (2) model comparison criteria, (3) analyses of model fit and correlation of local effects, and (4) restricted spatial regression.