2016 NFL Prediction Accuracy

Microsoft's Cortana and Nate Silver's fivethirtyeight.com Elo are again predicting the outcomes of NFL games each week during the 2016 season. (See the accuracy results for the two models for the 2015 NFL season here.)

The tables below use Brier scores to determine the accuracy of the probabilistic predictions from both models.

Brier scores range from 0 to 1, with 0 meaning the probabilities in the models perfectly match the outcomes of the games and 1 meaning no matches were made (so the closer to 0.000, the the 95% confidence intervals for both models for Wild Card Weekend include 0.250, making both models no better or worse than assigning each team a 50% win probability.better calibration of the model).

If a 50% (0.500) probability were assigned to each team for each game, the Brier score would be 0.250.

For the 2016 season, the Elo model has a Brier score of 0.219 and the Cortana model has a Brier score of 0.218.

In the 2015 season, the Elo model had a Brier score of 0.230 and the Cortana model had a Brier score for 0.229. In the 2014 season, the Elo model had a Brier score of 0.208 and the Cortana model had a Brier score of 0.213.

Both the Elo model (win probability of 0.61) and the Cortana model (win probability of 0.52) correctly picked New England in the Super Bowl.

The 95% confidence intervals for the season totals for both models exclude 0.250, making both models slightly better than assigning each team a 50% win probability.

The average win probabilities among winning teams are 56% for the Elo model and 56% for the Cortana model.

The 95% confidence interval for the Elo model for Championship Sunday includes 0.250, making it no better or worse than assigning each team a 50% win probability. The 95% confidence interval for the Cortana model does not include 0.250.

For the Divisional Round, the Elo model had a Brier score of 0.225 and the Cortana model had a Brier score of 0.190.

The 95% confidence intervals for both models for the Divisional Round include 0.250, making both models no better or worse than assigning each team a 50% win probability.

For Wild Card Weekend, the Elo model had a Brier score of 0.141 and the Cortana model had a Brier score of 0.137.

While this was the best weekly performance for both models so far in the season, the 95% confidence intervals for both models for Wild Card Weekend include 0.250, making both models no better or worse than assigning each team a 50% win probability.

In week 17 of the season, the Elo model had a Brier score of 0.174 and the Cortana model had a Brier score of 0.187.

The 95% confidence intervals for both models for week 17 include 0.250, making both models no better or worse than assigning each team a 50% win probability.

In week 16 of the season, the Elo model had a Brier score of 0.248 and the Cortana model had a Brier score of 0.239.

The 95% confidence intervals for both models for week 16 include 0.250, making both models no better or worse than assigning each team a 50% win probability.

The 95% confidence intervals for both models for week 15 include 0.250, making both models no better or worse than assigning each team a 50% win probability.

In week 14 of the season, the Elo model had a Brier score of 0.203 and the Cortana model had a Brier score of 0.173.

The 95% confidence interval for the Elo model for week 14 includes 0.250, making it no better or worse than assigning each team a 50% win probability. The 95% confidence interval for the Cortana model does not include 0.250, for the second time this season.

The 95% confidence interval for the Elo model for week 13 does not include 0.250 for the first time this season. The 95% confidence interval for the Cortana model includes 0.250, making it no better or worse than assigning each team a 50% win probability.

The 95% confidence intervals for both models for week 12 include 0.250, making both models no better or worse than assigning each team a 50% win probability.

The 95% confidence interval for the Elo model for week 11 includes 0.250, making it no better or worse than assigning each team a 50% win probability. The 95% confidence interval for the Cortana model does not include 0.250, which is a first for this model this season.

In week 10 of the season, the Elo model had a Brier score of 0.233 and the Cortana model had a Brier score of 0.252.

The 95% confidence interval for the Elo model for week 10 includes 0.250, making it no better or worse than assigning each team a 50% win probability. The Cortana model performed worse than assigning each team a 50% win probability.

In week 9 of the season, the Elo model had a Brier score of 0.233 and the Cortana model had a Brier score of 0.216.

The 95% confidence intervals for both models for week 9 include 0.250, making both models no better or worse than assigning each team a 50% win probability.

In week 8 of the season, the Elo model had a Brier score of 0.238 and the Cortana model had a Brier score of 0.239.

*The Brier scores for the Washington/Cincinnati tie were calculated by giving one-half of the Brier scores for the win for each team (the Elo model had Cincinnati with a 52% win probability and the Cortana model had Cincinnati with a 66% win probability).

The 95% confidence intervals for both models for week 8 include 0.250, making both models no better or worse than assigning each team a 50% win probability.

*The Brier scores for the Seattle/Arizona tie were calculated by giving one-half of the Brier scores for the win for each team (the Elo model had Arizona with a 51% win probability and the Cortana model had Seattle with a 55% win probability).

The 95% confidence intervals for both models for week 7 include 0.250, making both models no better or worse than assigning each team a 50% win probability.

In week 6 of the season, the Elo model had a Brier score of 0.242 and the Cortana model had a Brier score of 0.219.

The 95% confidence intervals for both models for week 6 include 0.250, making both models no better or worse than assigning each team a 50% win probability.

The Elo model performed worse than assigning each team a 50% win probability and the 95% confidence interval for the Cortana model includes 0.250, making it no better or worse than assigning each team a 50% win probability.

In week 4 of the season, the Elo model had a Brier score of 0.264 and the Cortana model had a Brier score of 0.271.

Both models performed worse than assigning each team a 50% win probability.

In week 3 of the season, the Elo model had a Brier score of 0.213 and the Cortana model had a Brier score of 0.247.

The 95% confidence intervals for both models for week 3 include 0.250, making both models no better or worse than assigning each team a 50% win probability.

In week 2 of the season, the Elo model had a Brier score of 0.229 and the Cortana model had a Brier score of 0.247.

The 95% confidence intervals for both models for week 2 include 0.250, making both models no better or worse than assigning each team a 50% win probability.

In week 1 of the season, the Elo model had a Brier score of 0.206 and the Cortana model had a Brier score of 0.229.

The 95% confidence intervals for both models for week 1 include 0.250, making both models no better or worse than assigning each team a 50% win probability.