Doesn't it always seem like when the weather man or woman says its going to be sunny it actually rains or if its supposed to snow a foot it ends up not snowing at all. That is because it is relatively hard to determine the accuracy of an event when the event is binomial (aka the event either happens or it doesn't) but the prediction is given as a percentage.
However, that is exactly what The Wizard of Oddz does, so how do you know whether or not The Wizard is providing value or is just full of crap? Well fortunately, unlike most sites that provide predictions, The Wizard will do a recap of the year current year of picks and compare it to other more popular known predictors.
First off, in the inaugural year of The Wizard of Oddz website, The Wizard's record was 48-8 in correctly selecting the Mid-Colonial Division games. That means in games where The Wizard had a team as the favorite for that game, 48 out of 56 times that team won or 85.7% of the time.
However, that alone does not tell the complete story, as mentioned The Wizard is providing a percentage chance that an event will either occur or not (i.e. a team will win or not).
There is a specific statistic called a Brier Score that tells you how accurately someone is at predicting these yes or no (win or loss) type scenarios. A score of 0 basically says that you are perfect at predicting these events (you are really good) and a score of 1 says that you never predict events correctly (you are really bad). You would get a score of 0.25 if you weighted every event as 50/50 (you are no better than flipping a coin).
American Research Group did a review of the 2015 NFL Predictions of FiveThirtyEight and Cortana for every game of the season. For those of you who are unaware, FiveThirtyEight is a popular website run by Nate Silver (an ESPN statistician) that focuses on numbers and statistics as the foundation for analysis of sports, elections, and other events. Cortana, now Bing, a product of Microsoft uses search, social, and other relevant data to make intelligent predictions about upcoming events.
According to American Research Group the Brier Score for FiveThirtyEight and Cortana for 2015 NFL football season were -
FiveThirtyEight - 0.230
Cortana - 0.229
These scores indicate that the predictive ability by both of these services as a whole were really no different than assigning a 50/50 value to each game (which would have given a score of 0.250.
On the other hand The Wizard scored a Brier Score for the 2016 Mid-Penn Colonial football season of -
The Wizard - 0.129
- which is significantly better than both of the more popular predictive analytic sites and also statistically better than a 50/50 odd of predicting games correctly. Therefore while other sites may have gained large popularity, The Wizard certainly has quality fundamental statistics behind its predictions.