Events and Seminars : 2012 Seminars

Estimation and Decomposition of Prediction Performance

Traditional likelihood methods and machine learning tools provide many alternative strategies for building a risk prediction model based on training data. The estimation of prediction performance, however, is a challenging task in the absence of independent validation data. Two relevant questions are:What is the best prediction model based on my training data? What statistical strategy finds the best prediction model? This talk discusses popular cross-validation estimates with regard to these questions. The main tool is a decomposition of the expected Brier score into model accuracy and model uncertainty. Repeated splits of the training data can be used to estimate these terms and to compare and test alternative statistical modeling strategies. The talk is illustrated with examples from medical statistics.

Click here to view slides