NOAA / Space Weather Prediction Center

Forecast Verification Bibliography
NOAA National Weather Service Space Weather Prediction Center

Verification Quick Links
Verification Home, K5 Warnings, K6 Warnings, Proton Warnings,
F10.7 Solar Flux, M-class Flares, X-class Flares, 10 MeV Proton Events,
Estimated Ap, Fredericksburg A, Geomagnetic Probabilities,
Glossary, Bibliography

Barnston, Anthony G., 1992:
Correspondence among the correlation, RMSE, and Heidke forecast verification measures; refinement of the Heidke score. Weather and Forecasting, 7, 699-709. PDF version

Barnston, Anthony G. and H.M. van den Dool, 1993:
A degeneracy in cross-validated skill in regression-based forecasts. J. Climate, 6, 963-977. PDF version

Blattenberger, G., and F. Lad, 1985:
Separating the Brier score into calibration and refinement components: A graphical exposition. The American Statistician, 39, 26-32.

Brier, G.W., and R.A. Allen, 1951:
Verification of Weather Forecasts. Compendium of Meteorology, Amer. Meteor. Soc., 841-848.

Briggs, W.M., and R.A. Levine, 1997:
Wavelets and field forecast verification. Monthly Weather Review, 125, 1329-1341. PDF version

Brooks, H.E., and C.A. Doswell, 1996:
A comparison of measures-oriented and distributions-oriented approaches to forecast verification. Weather and Forecasting, 11, 288-303. PDF version

Charba, J.P., D.W. Reynolds, B.E. McDonald, and G.M. Carter, 2003:
Comparative verification of recent quantitative precipitation forecasts in the National Weather Service: A simple approach for scoring forecast accuracy. Weather and Forecasting, 18, 161-183. PDF version

Clemen, R.T., A.H. Murphy, and R,L. Winkler, 1995:
Screening probability forecasts: Contrasts between choosing and combining. International Journal of Forecasting, 11, 133-146.

Daan, Harold, 1985:
Sensitivity of verification scores to the classification of the predictand. Monthly Weather Review, 113, 1384-1392. PDF version

DeGroot, M.H., and S.E. Fienberg, 1983:
The comparison and evaluation of forecasters. The Statistician, 32, 14-22.

Doswell III, C.A., R.P. Davies-Jones, and D.L. Keller, 1990:
On Summary measures of skill in rare event forecasting based on contingency tables. Weather and Forecasting, 5, 576-585. PDF version

Doswell III, C.A. and Harold E. Brooks, 1997:
Budget cutting and the value of weather services. Forecaster's Forum, Weather and Forecasting, 13, 206-212. PDF version

Ehrendorfer, M., and A.H. Murphy, 1988:
Comparative evaluation of weather forecasting systems: Sufficiency, quality, and accuracy. Monthly Weather Review, 116, 1757-1770. PDF version

Finley, J.P., 1884:
Tornado Predictions. American Meteorological Journal, 1, 85-88.

Flueck, J.A., 1987:
A study of some measures of forecast verification. 10th Conf. Probability and Statistics in Atmospheric Sciences, Edmonton, Alberta, Amer. Meteor. Soc., 69-73.

Gandin, L.S. and A, H, Murphy, 1992:
Equitable skill scores for categorical forecasts. Monthly Weather Review, 120, 361-370. PDF version

Gilbert, G.F., 1884:
Finley's Tornado Predictions. American Meteorological Journal, 1, 166-172.

Krzysztofowicz, R., 1992:
Bayesian correlation score: A utilitarian measure of forecast skill. Monthly Weather Review, 120, 208-219. PDF version

Livezey, R.E., 1986:
Caveat emptor! The evaluation of skill in climate predictions. Toward Understanding Climate Change. The J.O. Fletcher Lectures on Problems and Prospects of Climate Analysis and Forecasting, Editor U. Radok, Westview Press, Boulder, CO, 149-178.

Livezey, R.E., 1995:
Evaluation of forecasts. Analysis of Climate Variability (H. von Storch and A. Navarra, Editors). Berlin, Germany, Springer-Verlag, pp. 177-196.

Lupo, A.R. and P.S. Market, 2002:
The application of a simple method for the verification of weather forecasts and seasonal variations in forecast accuracy. Weather and Forecasting, 17, 891-897. PDF version

Marshall, K.T., and R.M, Oliver, 1995:
Decision Making and Forecasting. New York, NY, McGraw-Hill, 407 pp. See Chapter 8 (pp. 303-341).

Mason, I., 1989:
Dependence of the critical success index on sample climate and threshold probability. Aust. Meteor. Mag., 37, 75-81.

Michaelson, J., 1987:
Cross-validation in statistical climate forecast models. J. Climate Appl. Meteor., 26, 1589-1600.

Mielke, P.W., 1991:
The application of multivariate permutation methods based on distance functions in the earth sciences. Earth Sciences Review, 31, 55-71.

Mozer, J.B. and W.M. Briggs, 2003:
Skill in real-time solar wind shock forecasts. J. Geophys. Res., 108(A6), 1262-1270. PDF version

Murphy, A.H., 1991:
Forecast verification: its complexity and dimensionality. Monthly Weather Review, 119, 1590-1601. PDF version

Murphy, A.H., 1991:
Probabilities, odds, and forecasts of rare events. Forecaster's Forum, Weather and Forecasting, 6, 302-307. PDF version

Murphy, A.H., 1993:
What is a good forecast? An essay on the nature of goodness in weather forecasting, Weather and Forecasting, 8, 281-293. PDF version

Murphy, A.H., 1995:
A coherent method of stratification within the general framework for forecast verification. Monthly Weather Review, 123, 1582-1588. PDF version

Murphy, A.H., 1996:
The Finely affair: a signal event in the history of weather forecasting. Weather and Forecasting, 11, 003-020. PDF version

Murphy, A.H., 1996:
General decompositions of MSE-based skill scores: Measures of some basic aspects of forecast quality. Monthly Weather Review, 124, 2353-2369. PDF version

Murphy, A.H., 1997:
Forecast verification. Economic Value of Weather and Climate Forecasts (R.W. Katz and A.H. Murphy, Editors). Cambridge, U.K., Cambridge University Press, 222pp.

Murphy, A.H., B.G. Brown, and Y.-S. Chen, 1989:
Diagnostic verification of temperature forecasts. Weather and Forecasting, 4, 485-501. PDF version

Murphy, A.H., and H. Daan, 1985:
Forecast evaluation. Probability, Statistics, and Decision Making in the Atmospheric Sciences (A.H. Murphy and R.W. Katz, Editors). Boulder, CO, Westview Press, pp. 379-437.

Murphy, A.H., and E.S. Epstein, 1989:
Skill scores and correlation coefficients in model verification. Monthly Weather Review, 117, 572-581. PDF version

Murphy, A.H., and R.L. Winkler, 1987:
A general framework for forecast verification. Monthly Weather Review, 115, 1330-1338. PDF version

Murphy, A.H,, and R.L. Winkler, 1992:
Diagnostic verification of probability forecasts. International Journal of Forecasting, 7, 435-455.

Potts, J.M., C.K. Folland, I.T. Joiliffe, and D. Sexton, 1996:
Revised 'LEPS' scores for assessing climate model simulations and long-range forecasts. Journal of Climate, 9, 34-53. PDF version

Radok, Uwe, 1988:
Chance behavior of skill scores, Mon. Wea. Rev., 116, 489-494. PDF version

Saha, Suranjana and H.M. Van den Dool, 1988:
A measure of the practical limit of predictability. Monthly Weather Review, 116, 2522-2526. PDF version

Shaefer, J.T., 1990:
The Critical Success Index as an Indicator of Warning Skill. Weather and Forecasting, 5, 570-575.

Schervish, M.J., 1989:
A general method for comparing probability assessors. Annals of Statistics, 17, 1856-1879.

Stanski, H.R., L.J. Wilson and W.R. Burrows, 1989:
Survey of common verification methods in meteorology. Geneva, Switzerland, World Meteorological Organization, World Weather Watch Report No. 8 (TD No. 358), 114 pp.

Thibeaux, H.J. and F.W. Zwiers, 1984:
The interpretation and estimation of effective sample size. J. Climate Appl. Meteor., 23, 800-811.

Thomson, A.W.P., 2000:
Evaluating space weather forecasts of geomagnetic activity from a user perspective.Geophys. Res. Lett., 27, 4049-4052. PDF version

Van den Dool, H.M., 1987:
A bias in skill in forecasts based on analogues and antilogues. J. Climate Appl. Meteor., 26, 1278-1281. PDF version

Ward, M.N. and C.K. Folland, 1991:
Prediction of seasonal rainfall in the north Nordeste of Brazil using eigenvectors of sea-surface temperatures. International Journal of Climatology, 11, 711-743.

Watterson, I.G., 1996:
Non-dimensional measures of climate model performance. International Journal of Climatology, 16, 379-391.

Wilks, D.S., 1995:
Statistical Methods in the Atmospheic Sciences. San Diego, CA, Academic Press, 464 pp. See Chapter 7 (pp. 233-283).

Wilks, D.S., 2000:
Diagnostic verification of the climate prediction center long-lead outlooks, 1995-98. Journal of Climate, 13, 2389-2403. PDF version

Wilson, L.J., W.R. Burrows, and A. Lanzinger, 1999:
A strategy for verification of weather element forecasts from an ensemble prediction system. Monthly Weather Review, 127, 956-970. PDF version

Winkler, R.L., 1994:
Evaluating probabilities: Asymmetric scoring rules. Management Science, 40, 1395-1405.

Winkler, R.L., 1996:
Scoring rules and the evaluation of probabilities (with discussion), Test 5 , in press.

Yates, J.F., 1994:
Subjective probability accuracy analysis. Subjective Probability (G. Wright and P. Ayton, Editors). Chichester, U.K., Wiley, pp. 381-410.

Zhang, H. and T. Casey, 2000:
Verification of categorical probability forecasts. Weather and Forecasting, 15, 80-89. PDF version


Updated: October 1, 2007