2014 NCAA Men's Division I Basketball Tournament Prediction Accuracy

Nate Silver's relaunch of FiveThirtyEight.com includes predictions for the 2014 NCAA Men's Division I basketball tournament. Nate's model provides the odds of each team advancing in the tournament and is updated after each game.

The tables below use Brier scores to determine the accuracy of the probabilistic predictions in Nate's model. Brier scores range from 0 to 1, with 0 meaning the probabilities in the model perfectly match the outcomes of the games and 1 meaning no matches were made (so the closer to 0, the better calibration of the model). If a 50% probability were assigned to each team for each game, the Brier score would be 0.25.

For the 67 games played in the tournament, Nate's model has an overall Brier score of 0.1969. The 95% confidence interval for the overall score is 0.1495 - 0.2441, making this a low performing model (an overall Brier score of 0.0500 or less would make it a high performing model). It should be noted, however, that the Brier score for Nate's model is 0.2559 when calculated by tournament round.

Nate's model performed poorly in the first round of the tournament where the Brier score for the four games of 0.2744 exceeded what the score would have been by assigning each team a 50% probability of winning.

In the second round of play, Nate's model receives a Brier score of 0.1896. Most notable are wins by Harvard, Stanford, North Dakota State, Tennessee, Dayton, Stephen F. Austin, and Mercer with Nate's model showing probabilities of winning for these teams ranging from 42% to 7%.

In the third round of the tournament, the Brier score for Nate's model is 0.1899. Most notable in this round are wins by Baylor, Kentucky, Connecticut, Dayton, and Stanford with Nate's model showing probabilities of winning for these teams ranging from 46% to 21%.

The Brier score Nate's model in the fourth round is 0.2166. This is about 13% better than assigning all teams a 50% probability of winning. Kentucky was an upset based on Nate's model.

The Brier score for Nate's model for the fifth round exceeds what the Brier score would have been by assigning each team a 50% probability of winning. Wisconsin and Connecticut were upsets based on Nate's model.

In round 6, Nate's model missed both winners by a large margin.

In the final round, Nate's model picked the winning team, but only after recalculating the probabilities for the final game.

How well is Nate's model calibrated?

Nate's model performed poorly in rounds 1, 5, and 6, Nate's model had the final game virtually a toss-up, and the model did not improve as the tournament progressed. The overall Brier score of 0.1969 confirms that this is a low performing model.