2015 NFL Prediction Accuracy

Microsoft's Cortana and Nate Silver's fivethirtyeight.com Elo are predicting the outcomes of NFL games each week during the 2015 season. (See the accuracy results for the two models for the 2014 NFL season here.)

The tables below use Brier scores to determine the accuracy of the probabilistic predictions from both models.

Brier scores range from 0 to 1, with 0 meaning the probabilities in the models perfectly match the outcomes of the games and 1 meaning no matches were made (so the closer to 0.000, the better calibration of the model).

If a 50% (0.500) probability were assigned to each team for each game, the Brier score would be 0.250.

For the 2015 season, the Elo model has a Brier score of 0.230 and the Cortana model has a Brier score of 0.229. (In the 2014 NFL season, the Elo model had a Brier score of 0.208 and the Cortana model had a Brier score of 0.213.)

Both the Elo model and the Cortana model failed to correctly pick Denver in the Super Bowl. 

The 95% confidence intervals for both models for the season include 0.250, making both no better or worse than assigning each team a 50% win probability.

The Brier scores for the season reflect that the average win probabilities among winning teams are 56% for the Elo model and 55% for the Cortana model.

On a weekly basis, the Elo model has failed to perform better than assigning each team a 50% win probability seventeen out of twenty-one weeks and the Cortana model has failed to perform better than assigning each team a 50% win probability seventeen out of twenty-one weeks. The Elo model performed better than assigning each team a 50% win probability in weeks 6, 15, and Championship Sunday and the Cortana model performed better than assigning each team a 50% win probability in weeks 3, 15, and the Divisional Round.

In week 10, both models performed worse than assigning each team a 50% win probability. This is not surprising as both models failed to perform better than assigning each team a win probability of 50% in the second week of the season which invalidated both models (as was the case in the 2014 season).

The 95% confidence interval for the Cortana model includes 0.250, making this the seventeenth week the Cortana model did no better than assigning each team a 50% win probability.

The 95% confidence intervals for both models for week 17 include 0.250, making both models no better or worse than assigning each team a 50% win probability.

In week 16, the Elo model had a Brier score of 0.264 and the Cortana model had a Brier score of 0.249.

The Elo model performed worse than assigning each team a 50% win probability and the 95% confidence interval for the Cortana model includes 0.250, making it no better or worse than assigning each team a 50% win probability.

In week 15, the Elo model had a Brier score of 0.166 and the Cortana model had a Brier score of 0.182. This is the best performance for the Elo model for the season to date.

The Elo model performed about 34% better than assigning each team a 50% win probability and the Cortana performed about 27% better than assigning each team a 50% win probability.

In week 14, the Elo model had a Brier score of 0.248 and the Cortana model had a Brier score of 0.212.

The 95% confidence intervals for both models for week 13 include 0.250, making both models no better or worse than assigning each team a 50% win probability.

In week 12 of the season, the Elo model had a Brier score of 0.206 and the Cortana model had a Brier score of 0.229.

The 95% confidence intervals for both models for week 12 include 0.250, making both models no better or worse than assigning each team a 50% win probability.

In week 11 of the season, the Elo model had a Brier score of 0.194 and the Cortana model had a Brier score of 0.223.

The 95% confidence intervals for both models for week 11 include 0.250, making both models no better or worse than assigning each team a 50% win probability.

Both models performed worse than assigning each team a 50% win probability. The average probability for winning teams in week 11 for the Elo model is 52% and 49% for the Cortana model.

In week 10 of the season, the Elo model had a Brier score of 0.384 and the Cortana model had a Brier score of 0.382.

Both models performed worse than assigning each team a 50% win probability. The 95% confidence interval for the Cortana model excludes 0.025.

In week 9 of the season, the Elo model had a Brier score of 0.228 and the Cortana model had a Brier score of 0.229.

The 95% confidence intervals for both models for week 9 include 0.250, making both models no better or worse than assigning each team a 50% win probability.

In week 8 of the season, the Elo model had a Brier score of 0.180 and the Cortana model had a Brier score of 0.173.

The 95% confidence intervals for both models for week 8 include 0.250, making both models no better or worse than assigning each team a 50% win probability.

In week 7 of the season, the Elo model had a Brier score of 0.233 and the Cortana model had a Brier score of 0.196.

The 95% confidence intervals for both models for week 7 include 0.250, making both models no better or worse than assigning each team a 50% win probability.

In week 6 of the season, the Elo model had a Brier score of 0.184 and the Cortana model had a Brier score of 0.213.

The 95% confidence interval for the Elo model excludes 0.250, while the 95% confidence interval for the Cortana model includes 0.250, making the Cortana model no better or worse than assigning each team a 50% win probability.

The 95% confidence intervals for both models for week 4 include 0.250, making both models no better or worse than assigning each team a 50% win probability.

The 95% confidence interval for the Elo model includes 0.250 in week 3, making the Elo model no better or worse than assigning each team a 50% win probability, while the 95% confidence interval for the Cortana model excludes 0.250.

In week 2, the Elo model had a Brier score of 0.286, which is worse than assigning each team a 50% win probability. The Cortana model had a Brier score of 0.278, which is also worse than assigning each team a 50% win probability.

The 95% confidence intervals for the Brier scores for the Elo model and the Cortana model in week 1 include 0.250, making both models no better or worse than assigning each team a 50% win probability.