The Annals of Applied Statistics

Maximizing the information content of a balanced matched sample in a study of the economic performance of green buildings

Cinar Kilcioglu and José R. Zubizarreta

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


Buildings have a major impact on the environment through excessive use of resources, such as energy and water, and large carbon dioxide emissions. In this paper we revisit a previously published study about the economics of environmentally sustainable buildings and estimate the effect of green building practices on market rents. For this, we use new matching methods that take advantage of the clustered structure of the buildings data. We propose a general framework for matching in observational studies and specific matching methods within this framework that simultaneously achieve three goals: (i) maximize the information content of a matched sample (and, in some cases, also minimize the variance of a difference-in-means effect estimator); (ii) form the matches using a flexible matching structure (such as a one-to-many/many-to-one structure); and (iii) directly attain covariate balance as specified—before matching—by the investigator. To our knowledge, existing matching methods are only able to achieve, at most, two of these goals simultaneously. Also, unlike most matching methods, the proposed methods do not require estimation of the propensity score or other dimensionality reduction techniques, although with the proposed methods these can be used as additional balancing covariates in the context of (iii). Using these matching methods, we find that green buildings have 3.3% higher rental rates per square foot than otherwise similar buildings without green ratings—a moderately larger effect than the one found by the prior study.

Article information

Ann. Appl. Stat., Volume 10, Number 4 (2016), 1997-2020.

Received: June 2015
Revised: June 2016
First available in Project Euclid: 5 January 2017

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Causal inference matched sampling observational studies propensity score


Kilcioglu, Cinar; Zubizarreta, José R. Maximizing the information content of a balanced matched sample in a study of the economic performance of green buildings. Ann. Appl. Stat. 10 (2016), no. 4, 1997--2020. doi:10.1214/16-AOAS962.

Export citation


  • Aronow, P. M. and Samii, C. (2016). Does regression produce representative estimates of causal effects? Amer. J. Polit. Sci. 60 250–267.
  • Baiocchi, M. (2011). Designing robust studies using propensity score and prognostic score matching. Chapter 3 in Methodologies for Observational Studies of Health Care Policy, Dissertation, Department of Statistics, The Wharton School, Univ. Pennsylvania, Philadelphia, PA.
  • Baiocchi, M., Small, D. S., Lorch, S. and Rosenbaum, P. R. (2010). Building a stronger instrument in an observational study of perinatal care for premature infants. J. Amer. Statist. Assoc. 105 1285–1296.
  • Bertsimas, D. (2014). Statistics and Machine Learning via a Modern Optimization Lens. The 2014–2015 Philip McCord Morse Lecture.
  • Bixby, R. and Rothberg, E. (2007). Progress in computational mixed integer programming—A look back from the other side of the tipping point. Ann. Oper. Res. 149 37–41.
  • Chan, K. C. G., Yam, S. C. P. and Zhang, Z. (2016). Globally efficient nonparametric inference of average treatment effects by empirical balancing calibration weighting. J. R. Stat. Soc. Ser. B. Stat. Methodol. 78 673–700.
  • Cochran, W. G. (1965). The planning of observational studies of human populations. J. R. Stat. Soc. Ser. B. Stat. Methodol. 128 234–266.
  • Cochran, W. and Rubin, D. (1973). Controlling bias in observational studies: A review. Sankhya 35 417–446.
  • Crump, R. K., Hotz, V. J., Imbens, G. W. and Mitnik, O. A. (2009). Dealing with limited overlap in estimation of average treatment effects. Biometrika 96 187–199.
  • Diamond, A. and Sekhon, J. S. (2013). Genetic matching for estimating causal effects: A general multivariate matching method for achieving balance in observational studies. Rev. Econ. Stat. 95 932–945.
  • Eichholtz, P., Kok, N. and Quigley, J. M. (2010). Doing well by doing good? Green office buildings. Am. Econ. Rev. 100 2492–2509.
  • Fogarty, C., Mikkelsen, M., Gaieski, D. and Small, D. (2016). Discrete optimization for interpretable study populations and randomization inference in an observational study of severe sepsis mortality. J. Amer. Statist. Assoc. 111 447–458.
  • Graham, B. S., De Xavier Pinto, C. C. and Egel, D. (2012). Inverse probability tilting for moment condition model with missing data. Rev. Econ. Stud. 79 1053–1079.
  • Hainmueller, J. (2012). Entropy balancing for causal effects: A multivariate reweighting method to produce balanced samples in observational studies. Polit. Anal. 20 25–46.
  • Hansen, B. B. (2004). Full matching in an observational study of coaching for the SAT. J. Amer. Statist. Assoc. 99 609–618.
  • Hansen, B. B. (2007). Flexible, optimal matching for observational studies. R News 7 18–24.
  • Hansen, B. B. and Bowers, J. (2008). Covariate balance in simple, stratified and clustered comparative studies. Statist. Sci. 23 219–236.
  • Hansen, B. B. and Klopfer, S. O. (2006). Optimal full matching and related designs via network flows. J. Comput. Graph. Statist. 15 609–627.
  • Hansen, B. B., Rosenbaum, P. R. and Small, D. S. (2014). Clustered treatment assignments and sensitivity to unmeasured biases in observational studies. J. Amer. Statist. Assoc. 109 133–144.
  • Hartman, E., Grieve, R., Ramsahai, R. and Sekhon, J. S. (2015). From sample average treatment effect to population average treatment effect on the treated: Combining experimental with observational studies to estimate population treatment effects. J. Roy. Statist. Soc. Ser. A 178 757–778.
  • Haviland, A., Nagin, D. and Rosenbaum, P. (2007). Combining propensity score matching and group-based trajectory analysis in an observational study. Psychol. Methods 12 247.
  • Hill, J. (2008). Discussion of research using propensity-score matching: Comments on “A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003” by Peter Austin, Statistics in Medicine [MR2439882]. Stat. Med. 27 2055–2061.
  • Hsu, J. Y., Zubizarreta, J. R., Small, D. S. and Rosenbaum, P. R. (2015). Strong control of the familywise error rate in observational studies that discover effect modification by exploratory methods. Biometrika 102 767–782.
  • Iacus, S. M., King, G. K. and Porro, G. (2012). Causal inference without balance checking: Coarsened exact matching. Polit. Anal. 20 1–24.
  • Imai, K. and Ratkovic, M. (2015). Robust estimation of inverse probability weights of marginal structural models. J. Amer. Statist. Assoc. 110 1013–1023.
  • Imbens, G. W. (2015). Matching methods in practice: Three examples. J. Hum. Resour. 50 373–419.
  • Imbens, G. W. and Rubin, D. B. (2015). Causal Inference—For Statistics, Social, and Biomedical Sciences: An Introduction. Cambridge Univ. Press, New York.
  • Kalton, G. (1968). Standardization: A technique to control for extraneous variables. Appl. Statist. 17 118–136.
  • Keele, L., Titiunik, R. and Zubizarreta, J. R. (2015). Enhancing a geographic regression discontinuity design through matching to estimate the effect of ballot initiatives on voter turnout. J. Roy. Statist. Soc. Ser. A 178 223–239.
  • Kilcioglu, C. and Zubizarreta, J. R. (2016). Supplement to “Maximizing the information content of a balanced matched sample in a study of the economic performance of green buildings.” DOI:10.1214/16-AOAS962SUPP.
  • Lehmann, E. L. (2006). Nonparametrics: Statistical Methods Based on Ranks, 1st ed. Springer, New York.
  • Li, F., Morgan, K. L. and Zaslavsky, A. M. (2016). Balancing covariates via propensity score weighting. Working paper.
  • Li, Y. P., Propert, K. J. and Rosenbaum, P. R. (2001). Balanced risk set matching. J. Amer. Statist. Assoc. 96 870–882.
  • Linderoth, J. T. and Lodi, A. (2010). MILP software. In Wiley Encyclopedia of Operations Research and Management Science (J. J. Cochran, L. A. Cox, P. Keskinocak and J. P. Kharoufeh, eds.). Wiley, New York.
  • Lu, B. (2005). Propensity score matching with time-dependent covariates. Biometrics 61 721–728.
  • Nemhauser, G. L. (2013). Integer programming: Global impact. EURO INFORMS, July 2013.
  • Nikolaev, A. G., Jacobson, S. H., Cho, W. K. T., Sauppe, J. J. and Sewell, E. C. (2013). Balance optimization subset selection (BOSS): An alternative approach for causal inference with observational data. Oper. Res. 61 398–412.
  • Pimentel, S. D., Kelz, R. R., Silber, J. H. and Rosenbaum, P. R. (2015). Large, sparse optimal matching with refined covariate balance in an observational study of the health outcomes produced by new surgeons. J. Amer. Statist. Assoc. 110 515–527.
  • Rosenbaum, P. R. (1987). Model-based direct adjustment. J. Amer. Statist. Assoc. 82 387–394.
  • Rosenbaum, P. R. (1989). Optimal matching for observational studies. J. Amer. Statist. Assoc. 84 1024–1032.
  • Rosenbaum, P. (1991). Discussing hidden bias in observational studies. Arch. Intern. Med. 115 901–905.
  • Rosenbaum, P. R. (2002). Observational Studies, 2nd ed. Springer, New York.
  • Rosenbaum, P. R. (2005). Heterogeneity and causality: Unit heterogeneity and design sensitivity in observational studies. Amer. Statist. 59 147–152.
  • Rosenbaum, P. R. (2010). Design of Observational Studies. Springer, New York.
  • Rosenbaum, P. R. (2014). Weighted $M$-statistics with superior design sensitivity in matched observational studies with multiple controls. J. Amer. Statist. Assoc. 109 1145–1158.
  • Rosenbaum, P. R. (2015). How to see more in observational studies: Some new quasi-experimental devices. Annual Review of Statistics and Its Application 2 21–48.
  • Rosenbaum, P. R., Ross, R. N. and Silber, J. H. (2007). Minimum distance matched sampling with fine balance in an observational study of treatment for ovarian cancer. J. Amer. Statist. Assoc. 102 75–83.
  • Rosenbaum, P. R. and Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika 70 41–55.
  • Rosenbaum, P. R. and Rubin, D. B. (1985). Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. Amer. Statist. 39 33–38.
  • Rosenbaum, P. R. and Silber, J. (2001). Matching and thick description in an observational study of mortality after surgery. Biostatistics 2 217–232.
  • Rosenbaum, P. R. and Silber, J. H. (2009). Amplification of sensitivity analysis in matched observational studies. J. Amer. Statist. Assoc. 104 1398–1405.
  • Rubin, D. B. (1979). Using multivariate matched sampling and regression adjustment to control bias in observational studies. J. Amer. Statist. Assoc. 74 318–328.
  • Rubin, D. B. (2008). For objective causal inference, design trumps analysis. Ann. Appl. Stat. 2 808–840.
  • Silber, J. H., Rosenbaum, P. R., Kelz, R. R., Gaskin, D. J., Ludwig, J. M., Ross, R. N., Niknam, B. A., Hill, A., Wang, M., Even-Shoshan, O. and Fleisher, L. A. (2015). Examining causes of racial disparities in general surgical mortality: Hospital quality versus patient risk. Med. Care 53 619–629.
  • Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statist. Sci. 25 1–21.
  • Traskin, M. and Small, D. (2011). Defining the study population for an observational study to ensure sufficient overlap: A tree approach. Statistics in Biosciences 3 94–118.
  • Tukey, J. W. (1986). Sunset salvo. Amer. Statist. 40 72–76.
  • Weston, S. and Calaway, R. (2014). Getting Started with doParallel and foreach.
  • Yang, D., Small, D. S., Silber, J. H. and Rosenbaum, P. R. (2012). Optimal matching with minimal deviation from fine balance in a study of obesity and surgical outcomes. Biometrics 68 628–636.
  • Zubizarreta, J. R. (2012). Using mixed integer programming for matching in an observational study of kidney failure after surgery. J. Amer. Statist. Assoc. 107 1360–1371.
  • Zubizarreta, J. R. (2015). Stable weights that balance covariates for estimation with incomplete outcome data. J. Amer. Statist. Assoc. 110 910–922.
  • Zubizarreta, J. R. and Kilcioglu, C. (2016). designmatch: Construction of Optimally Matched Samples for Randomized Experiments and Observational Studies that are Balanced by Design R package version 0.2.0.
  • Zubizarreta, J. R., Paredes, R. D. and Rosenbaum, P. R. (2014). Matching for balance, pairing for heterogeneity in an observational study of the effectiveness of for-profit and not-for-profit high schools in Chile. Ann. Appl. Stat. 8 204–231.
  • Zubizarreta, J. R., Reinke, C. E., Kelz, R. R., Silber, J. H. and Rosenbaum, P. R. (2011). Matching for several sparse nominal variables in a case–control study of readmission following surgery. Amer. Statist. 65 229–238.
  • Zubizarreta, J. R., Small, D. S., Goyal, N. K., Lorch, S. and Rosenbaum, P. R. (2013). Stronger instruments via integer programming in an observational study of late preterm birth outcomes. Ann. Appl. Stat. 7 25–50.

Supplemental materials

  • Supplement to “Maximizing the information content of a balanced matched sample in a study of the economic performance of green buildings”. In this on-line supplement, we include the appendices to “Maximizing the information content of a balanced matched sample in a study of the economic performance of green buildings” by Kilcioglu and Zubizarreta (2016).