The Annals of Applied Statistics

Imputation of truncated p-values for meta-analysis methods and its genomic application

Shaowu Tang, Ying Ding, Etienne Sibille, Jeffrey S. Mogil, William R. Lariviere, and George C. Tseng

Full-text: Open access


Microarray analysis to monitor expression activities in thousands of genes simultaneously has become routine in biomedical research during the past decade. A tremendous amount of expression profiles are generated and stored in the public domain and information integration by meta-analysis to detect differentially expressed (DE) genes has become popular to obtain increased statistical power and validated findings. Methods that aggregate transformed $p$-value evidence have been widely used in genomic settings, among which Fisher’s and Stouffer’s methods are the most popular ones. In practice, raw data and $p$-values of DE evidence are often not available in genomic studies that are to be combined. Instead, only the detected DE gene lists under a certain $p$-value threshold (e.g., DE genes with $p$-value${}<0.001$) are reported in journal publications. The truncated $p$-value information makes the aforementioned meta-analysis methods inapplicable and researchers are forced to apply a less efficient vote counting method or naïvely drop the studies with incomplete information. The purpose of this paper is to develop effective meta-analysis methods for such situations with partially censored $p$-values. We developed and compared three imputation methods—mean imputation, single random imputation and multiple imputation—for a general class of evidence aggregation methods of which Fisher’s and Stouffer’s methods are special examples. The null distribution of each method was analytically derived and subsequent inference and genomic analysis frameworks were established. Simulations were performed to investigate the type I error, power and the control of false discovery rate (FDR) for (correlated) gene expression data. The proposed methods were applied to several genomic applications in colorectal cancer, pain and liquid association analysis of major depressive disorder (MDD). The results showed that imputation methods outperformed existing naïve approaches. Mean imputation and multiple imputation methods performed the best and are recommended for future applications.

Article information

Ann. Appl. Stat., Volume 8, Number 4 (2014), 2150-2174.

First available in Project Euclid: 19 December 2014

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Microarray analysis meta-analysis Fisher’s method Stouffer’s method missing value imputation


Tang, Shaowu; Ding, Ying; Sibille, Etienne; Mogil, Jeffrey S.; Lariviere, William R.; Tseng, George C. Imputation of truncated p -values for meta-analysis methods and its genomic application. Ann. Appl. Stat. 8 (2014), no. 4, 2150--2174. doi:10.1214/14-AOAS747.

Export citation


  • Begum, F., Ghosh, D., Tseng, G. C. and Feingold, E. (2012). Comprehensive literature review and statistical considerations for GWAS meta-analysis. Nucleic Acids Research 40 3777–3784.
  • Bellot, G. L., Tan, W. H., Tay, L. L., Koh, D. and Wang, X. (2012). Reliability of tumor primary cultures as a model for drug response prediction: Expression profiles comparison of tissues versus primary cultures from colorectal cancer patients. J. Cancer Res. Clin. Oncol. 138 463–482.
  • Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. R. Stat. Soc. Ser. B Stat. Methodol. 57 289–300.
  • Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Ann. Statist. 29 1165–1188.
  • Bianchini, M., Levy, E., Zucchini, C., Pinski, V., Macagno, C., Sanctis, P. D., Valvassori, L., Carinci, P. and Mordoh, J. (2006). Comparative study of gene expression by cDNA microarray in human colorectal cancer tissues and normal mucosa. Int. J. Oncol. 29 83–94.
  • Birnbaum, A. (1954). Combining independent tests of significance. J. Amer. Statist. Assoc. 49 559–574.
  • Birnbaum, A. (1955). Characterizations of complete classes of tests of some multiparametric hypotheses, with applications to likelihood ratio tests. Ann. Math. Statist. 26 21–36.
  • Borovecki, F. et al. (2005). Genome-wide expression profiling og human blood reveals biomarkers for huntingtons disease. Proc. Natl. Acad. Sci. USA 102 11023–11028.
  • Cardoso, J. et al. (2007). Expression and genomic profiling of colorectal cancer. Biochimica et Biophysica Acta-Reviews on Cancer 1775 103–137.
  • Chan, S. K., Griffith, O. L., Tai, I. T. and Jones, S. J. M. (2008). Meta-analysis of colorectal cancer gene expression profiling studies identifies consistently reported candidate biomarkers. Cancer Epidemiol Biomarkers Prev. 17 543–552.
  • Chang, L.-C., Lin, H.-M., Sibille, E. and Tseng, G. C. (2013). Meta-analysis methods for combining multiple expression profiles: Comparisons, statistical characterization and an application guideline. BMC Bioinformatics 14 368.
  • Choi, J. K. et al. (2003). Combining multiple microarray studies and modeling interstudy variation. Bioinformatics 19 84–90.
  • Choi, H. et al. (2007). A latent variable approach for meta-analysis of gene expression data from multiple microarray experiments. BMC Bioinformatics 8 364–383.
  • Duval, S. and Tweedie, R. L. (2000a). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 56 455–463.
  • Duval, S. and Tweedie, R. (2000b). A nonparametric “trim and fill” method of accounting for publication bias in meta-analysis. J. Amer. Statist. Assoc. 95 89–98.
  • Fisher, R. A. (1925). Statistical Methods for Research Workers. Oliver & Boyd, Edinburgh.
  • Gorlov, I. P., Byun, J., Gorlova, O. Y., Aparicio, A. M., Efstathiou, E. and Logothetis, C. J. (2009). Candidate pathways and genes for prostate cancer: A meta-analysis of gene expression data. BMC Med. Genomics 2 48.
  • Griffith, O. L., Jones, S. J. M. and Wiseman, S. M. (2006). Meta-analysis and meta-review of thyroid cancer gene expression profiling studies identifies important diagonstic biomarkers. J. Clin. Oncol. 24 5043–5051.
  • Hedges, L. V. and Olkin, I. (1980). Vote-counting methods in research synthesis. Psychological Bulletin 88 359.
  • Hedges, L. V. and Olkin, I. (1985). Statistical Methods for Meta-Analysis. Academic Press, Orlando, FL.
  • Ioannidis, J. P. A., Allison, D. B., Ball, C. A. et al. (2009). Repeatability of published microarray gene expression analysis. Nature Genetics 41 149–155.
  • Jiang, X., Tan, J., Li, J., Kivimäe, S., Yang, X., Zhuang, L., Lee, P. L., Chan, M. T. W., Stanton, L. W., Liu, E. T., Cheyette, B. N. R. and Yu, Q. (2008). DACT3 is an epigenetic regulator of Wnt/beta-catenin signaling in colorectal cancer and is a therapeutic target of histone modifications. Cancer Cell 13 529–541.
  • LaCroix-Fralish, M. L., Austin, J.-S., Zheng, F. Y., Levitin, D. J. and Mogil, J. S. (2011). Patterns of pain: Meta-analysis of microarray studies of pain. Pain 152 1888–1898.
  • Li, K. C. (2002). Genome-wide coexpression dynamics: Theory and application. Proc. Natl. Acad. Sci. USA 99 16875–16880.
  • Li, J. and Tseng, G. C. (2011). An adaptively weighted statistic for detecting differential gene expression when combining multiple transcriptomic studies. Ann. Appl. Stat. 5 994–1019.
  • Littell, R. C. and Folks, J. L. (1971). Asymptotic optimality of Fisher’s method of combining independent tests. J. Amer. Statist. Assoc. 66 193–194.
  • Littell, R. C. and Folks, J. L. (1973). Asymptotic optimality of Fisher’s method of combining independent tests. II. J. Amer. Statist. Assoc. 68 193–194.
  • Little, R. J. A. and Rubin, D. B. (2002). Statistical Analysis with Missing Data, 2nd ed. Wiley, Hoboken, NJ.
  • McCarley, R. W., Wible, C. G., Frumin, M., Hirayasu, Y., Levitt, J. J. and Shenton, M. E. (2001). Why vote-count reviews don’t count [letter to the editor]. Biological Psychiatry 49 161–163.
  • Moreau, Y. et al. (2003). Comparison and meta-analysis of microarray data: From the bench to the computer desk. Trends in Genetics 19 570–577.
  • Olkin, I. and Saner, H. (2001). Approximations for trimmed Fisher procedures in research synthesis. Stat. Methods Med. Res. 10 267–276.
  • Owen, A. B. (2009). Karl Pearson’s meta-analysis revisited. Ann. Statist. 37 3867–3892.
  • Pirooznia, M., Nagarajan, V. and Deng, Y. (2007). Gene venn—a web application for comparing gene lists using venn diagram. Bioinformation 1 420–422.
  • Rhodes, D. R., Barrette, T. R., Rubin, M. A., Ghosh, D. and Chinnaiyan, A. M. (2002). Meta-analysis of microarrays: Interstudy validation of gene expression profiles reveals pathway dysregulation in prostate cancer. Cancer Res. 62 4427–4433.
  • Roy, S. N. (1953). On a heuristic method of test construction and its use in multivariate analysis. Ann. Math. Statistics 24 220–238.
  • Segal, E. et al. (2004). A module map showing conditional activity of expression modules in cancer. Nature Genetics 3 1090–1098.
  • Song, C. and Tseng, G. C. (2014). Hypothesis setting and order statistic for robust Genomic meta-analysis. Ann. Appl. Stat. 8 777–800.
  • Stouffer, S. et al. (1949). The American Soldier, Volume I: Adjustment During Army Life. Princeton Univ. Press, Princeton, NJ.
  • Tang, S., Ding, Y., Sibille, E., Mogil, J. S., Lariviere, W. R. and Tseng, G. C. (2014). Supplement to “Imputation of truncated $p$-values for meta-analysis methods and its genomic application.” DOI:10.1214/14-AOAS747SUPP.
  • Tseng, G. C., Ghosh, D. and Feingold, E. (2012). Comprehensive literature review and statistical considerations for microarray meta-analysis. Nucleic Acids Res. 40 3785–3799.
  • Wang, X., Lin, Y., Song, C., Sibille, E. and Tseng, G. C. (2012). Detecting disease-associated genes with confounding variable adjustment and the impact on genomic meta-analysis: With application to major depressive disorder. BMC Bioinformatics 13 52.

Supplemental materials