*
At the completion of the Italy-Costa Rica match in Group D in Recife, half of the 2014 World Cup group games (24 of 48) had been played. There have been expected victories for some nations, big upsets for others – Adios Spain! Bye-bye England! – and more goals than most fans would have expected. So who could have forecast this? Actually, a huge variety of ‘experts’, forecasters, theorists, modelers and systems have tried to predict the outcome of this tournament, from Goldman Sachs to boffin statistical organisations. In his second post for Sportingintelligence, and as part of an ongoing evaluation of rates of success (click HERE for Part 1 and background), Roger Pielke Jr sorts the best from the rest.
Follow Roger on Twitter: @RogerPielkeJR and on his blog
.
By Roger Pielke Jr.
20 June 2014
With Costa Rica’s surprising victory over Italy the group stages have reached their halfway point, with 24 of 48 matches in the books. There is still a lot to be decided and the same goes for the World Cup prediction evaluation exercise that I’m tracking. But like the World Cup itself a few pieces are starting to fall into place.
For background detail on the 11 different prognostications we are tracking, see the original piece; but their identities are clear enough in general in the updated prediction league table (below), which is starting to show some spread.
The FIFA Rankings and Andrew Yuan are sitting at the top with 16 correct outcomes from 24 games. The naïve forecast that I chose to use as a threshold of skill, based on transfer values, has slipped into last place alongside Infostrada and the FT.
Article continues below
The spread across the methods in matches picked correctly is only three games, which would suggest a chance for those behind to catch up. I was curious about this so I created a crosstab showing how many of the 48 matches that each method has in common with each of the others. It shows that catching up is possible, but the opportunities are limited.
I was surprised to see how many matches are in common across the methods. For example, across the 48 group stage matches the approach based on estimated player transfer values has between 36 and 42 matches in common with each of the 11 prediction methods that I am tracking.
Article continues below
On the one hand, this makes sense as the powers in world football are generally the same independent of methodology. If the top eight teams across the groups are each predicted to win their group stage matches, then that alone accounts for 24 of the 48 total matches. But on the other hand, skillful prediction means outperforming a simple approach, so it should be fully expected that a skilled method would be able to identify some of the opportunities for upsets.
For instance, several of the methods lost all of the matches played by Spain and England, including the transfer value baseline. Were those games inherently unpredictable? Or do they reflect a fundamental conservativism in the methods? In an alternate universe where England won their first two matches, the FIFA rankings would trail the transfer value approach, and the Elo ratings would be out in front all alone.
The FT method (which to be fair, was designed to say something about players not teams) has the greatest deviation from the other approaches, with Goldman Sachs not far behind. These approaches thus have offer a much wider spread of predictive outcomes, which could mean a big improvement on the baseline approach or a big degradation.
Prediction entails risk, and most methods have taken very little. It is well understood in studies of risk that we’d rather take a high probability of being right or wrong with the crowd, rather than stick our necks out and risk being wrong on all on our own. The World Cup predictions would appear to bear that out.
The two leaders thus far are the FIFA Rankings and Andrew Yuan. This is not surprising as they have 47 out of 48 Group Stage matches picked identically. Their ranking against each other will be decided by a single match. This raises an important question with relevance far beyond the World Cup or sports: what is the point of a complex prediction methodology that essentially replicates a simpler, readily available index?
None of the methods are able to outperform the FIFA Rankings – thus far – providing some good evidence of their value in ranking teams. There are 24 group matches left to play, and no doubt more surprises are in store. However, the results so far indicate that while there is a lot of entertainment value in the World Cup predictions, there is little evidence of value added.
The next update will come after the 36th match, Croatia vs. Mexico on Monday. Stay tuned.
..
Roger Pielke Jr. is a professor of environmental studies at the University of Colorado, where he also directs its Center for Science and technology Policy Research. He studies, teaches and writes about science, innovation, politics and sports. He has written for The New York Times, The Guardian,FiveThirtyEight, and The Wall Street Journal among many other places. He is thrilled to join Sportingintelligence as a regular contributor. Follow Roger on Twitter: @RogerPielkeJR and on his blog
.
More on this site mentioning the World Cup
Follow SPORTINGINTELLIGENCE on Twitter
