*Journal of Computers*, Vol 6, No 2 (2011), 162-171, Feb 2011

### The Effects of Imputing Missing Data on Ensemble Temperature Forecasts

#### Abstract

A major issue for developing post-processing methods for NWP forecasting systems is the need to obtain complete training datasets. Without a complete dataset, it can become difficult, if not impossible, to train and verify statistical post-processing techniques, including ensemble consensus forecasting schemes. In addition, when ensemble forecast data are missing, the real-time use of the consensus forecast weighting scheme becomes difficult and the quality of uncertainty information derived from the ensemble is reduced. To ameliorate these problems, an analysis of the treatment of missing data in ensemble model temperature forecasts is performed to determine which method of replacing the missing data produces the lowest Mean Absolute Error (MAE) of consensus forecasts while preserving the ensemble calibration. This study explores several methods of replacing missing data, including ones based on persistence, a Fourier fit to capture seasonal variability, ensemble member mean substitution, three day mean deviation, and an Artificial Neural Network (ANN). The analysis is performed on 48-hour temperature forecasts for ten locations in the Pacific Northwest. The methods are evaluated according to their effect on the forecast performance of two ensemble post-processing forecasting methods, specifically an equal-weight consensus forecast and a ten day performance-weighted window. The methods are also assessed using rank histograms to determine if they preserve the calibration of the ensembles. For both postprocessing techniques all imputation methods, with the exception of the ensemble mean substitution, produce mean absolute errors not significantly different from the cases when all ensemble members are available. However, the three day mean deviation and ANN have rank histograms similar to that for the baseline of the non-imputed cases (i.e. the ensembles are appropriately calibrated) for all locations, while persistence, ensemble mean, and Fourier substitution do not consistently produce appropriately calibrated ensembles. The three day mean deviation has the advantage of being computationally efficient in a real-time forecasting environment.

#### Keywords

#### References

[1] Kidson, J.W., and K.E. Trenberth, 1988: Effects of missing data on estimates of monthly mean general circulation statistics, J. Climate, 1, 1261–1275.

doi:10.1175/1520-0442(1988)001<1261:EOMDOE>2.0.CO;2

[2] Vincent, L. A., and D. W. Gullet., 1999: Canadian historical and homogeneous temperature datasets for climate change analyses. International Journal of Climatology. 19, 1375-1388.

doi:10.1002/(SICI)1097-0088(199910)19:12<1375::AID-JOC427>3.0.CO;2-0

[3] Schneider, T., 2001: Analysis of incomplete climate data: estimation of mean values and covariance matrices and imputation of missing values. J. Climate, 14, 853–871

doi:10.1175/1520-0442(2001)014<0853:AOICDE>2.0.CO;2

[4] Richman, M. B., R, B. Trafalis, and I. Adrianto., 2009: Missing data imputation through machine learning algorithms. Artificial Intelligence Methods in the Environmental Sciences, S. E. Haupt, A. Pasini, and C. Marzban, Eds., Springer-Verlag, 153-169.

[5] Kemp, W. P., D. G. Brunell, D.O. Everson, and A. J. Thomson, 1983. Estimating missing daily maximum and minimum temperatures. J. Climate, 22, 1587-1593.

[6] Glahn, H. R., and D. A. Lowry, 1972: The use of Model Output Statistics (MOS) in objective weather forecasting. J. Appl. Meteor., 11, 1203-1211.

doi:10.1175/1520-0450(1972)011<1203:TUOMOS>2.0.CO;2

[7] Hamill, T. M., S.L. Mullen, C. Snyder, Z. Toth, and D.P. Baumhefner, 2000: Ensemble forecasting in the short to medium range: report from a workshop, Bull. Amer. Meteor. Soc., 81, 2653-2664.

doi:10.1175/1520-0477(2000)081<2653:EFITST>2.3.CO;2

[8] Woodcock, F. and C. Engel, 2005: Operational consensus forecasts. Wea. Forecasting 20, 101-111.

doi:10.1175/WAF-831.1

[9] Raftery, A. E., T. Gneiting, F. Balabdaoui, and M. Polakowski, 2005: Using Bayesian model averaging to calibrate forecast ensembles, Mon. Wea. Rev., 133, 1155- 1174.

doi:10.1175/MWR2906.1

[10] Fraley, C. A.E. Raftery, and T. Gneiting, 2010: Calibrating multimodel forecast ensembles with exchangeable and missing members using Bayesian Model Averaging, Mon. Wea. Rev., 138, 190-202.

doi:10.1175/2009MWR3046.1

[11] Greybush, S. J., S.E. Haupt., and G. S. Young., 2008: The regime dependence of optimally weighted ensemble model consensus forecasts of surface temperature. Wea. Forecasting, 23, 1146-1161.

doi:10.1175/2008WAF2007078.1

[12] Grimit, E.P. and C.F. Mass, 2002: Aspects of effective mesoscale short-range ensemble forecasting system over the Pacific Northwest. Wea. Forecasting, 17, 192-205.

doi:10.1175/1520-0434(2002)017<0192:IROAMS>2.0.CO;2

[13] Rubin, D.B. (1976) Inference and missing data. Biometrika, 63, 581-592.

doi:10.1093/biomet/63.3.581

[14] Little, R.J.A. and Rubin, D.B. (2002). Statistical Analysis with Missing Data, 2nd edition, New York: John Wiley.

[15] Witten, I. H., and E. Frank., 2005: Data mining: practical machine learning tools and techniques, 2nd Edition, Morgan Kaufmann, San Francisco, 2005.

[16] Krasnopolsky, V. M., 2009: Neural network applications to solve forward and inverse problems in atmospheric and oceanic satellite remote sensing. Artificial Intelligence Methods in the Environmental Sciences, S. E. Haupt, A. Pasini, and C. Marzban, Eds., Springer-Verlag, 191-205.

[17] Young, G. S., 2009: Implementing a neural network emulation of a satellite retrieval algorithm. Artificial Intelligence Methods in the Environmental Sciences, S. E. Haupt, A. Pasini, and C. Marzban, Eds., Springer-Verlag, 207-216.

[18] Wilks, D.S., 2005: Statistical methods in the atmospheric sciences, 2nd ed., Academic Press, 626 pp.

[19] Anderson, J. L., 1996: A method for producing and evaluating probabilistic forecasts from Ensemble Model Integrations. J. Climate, 9, 1518–1530.

doi:10.1175/1520-0442(1996)009<1518:AMFPAE>2.0.CO;2

[20] Hamill, T. M., and S. J. Colucci, 1996: Random and systematic error in NMC’s short-range Eta ensembles. Preprints, 13th Conf. on Probability and Statistics in the Atmospheric Sciences, San Francisco, CA, Amer. Meteor. Soc., 51–56.

[21] Talagrand, O., R. Vautard, and B. Strauss, 1997: Evaluation of probabilistic prediction systems. Proceedings, ECMWF Workshop on Predictability, ECMWF, 1–25.

[22] 2001: Interpretation of rank histograms for verifying ensemble forecasts. Mon. Wea. Rev., 129, 550-560.

doi:10.1175/1520-0493(2001)129<0550:IORHFV>2.0.CO;2

[23] Marzban, C., R. Wang, F. Kong, S. Leyton, 2010: On the effect of correlations on rank histograms: Reliability of temperature and wind-speed forecasts from fine-scale ensemble reforecasts, Mon. Wea. Rev., in press.

[24] Saetra, O, H. Hersbach, J-R Bidlot and D.S. Richardson, 2004: Effects of observation errors on the statistics for ensemble spread and reliability. Mon. Wea. Rev., 132, 1487-1501.

doi:10.1175/1520-0493(2004)132<1487:EOOEOT>2.0.CO;2

[25] Elmore, K.L., 2005: Alternatives to the chi-square test for evaluating rank histograms from ensemble forecasts. Wea. Forecasting, 20, 789-795.

doi:10.1175/WAF884.1

[26] Jolliffe, I. T., and C. Primo, 2008: Evaluating rank histograms using decomposition of the chi-square test statistic, Mon. Wea. Rev., 136, 2133-2139.

doi:10.1175/2007MWR2219.1

Full Text: PDF