Stats For Goalkeepers Don't Tell the Whole Story - Or Even Much of One

Just Terrible

In recent days, I've come across two different blog posts using stats to compare Liverpool's Loris Karius to, respectively, his contemporaries this season, and to David De Gea in his first season at Manchester United. The metrics used were things such as average number of saves per game, total clean sheets, goals conceded per game, and the total number of saves, claimed crosses, punched crosses, and 'sweeper clearances.'

There are two significant problems with all of these metrics: These first is that almost all of them are inextricably linked with how busy a goalkeeper is, which is something out of a goalkeeper's control. One article, amusingly claiming that De Gea is 'among the 10 worst goalkeepers in the Premier League,' makes this claim based on him making only 2.4 saves per game. But, obviously, playing for a Jose Mourinho team, he's not facing as many shots per game as many others keepers. We know this is true, without even looking for the number of shots allowed, because only three goalkeepers in the league have conceded fewer goals.

The same applies when a keeper is 'graded' based on the number of crosses he claims or punches. De Gea was derided for punching 'only' one cross clear. Firstly, he prefers to catch, which is better than punching, and secondly, again, it makes no sense to judge keepers on hard numbers like these, when they are not all busy to the same degree. We don't know the total number of crosses played into his box, and more importantly, the number that would be considered a goalkeeper's ball (and this, to add to the problems, is highly subjective).

A better stat might consider the percentage of crosses a goalkeeper successfully catches or punches, but such a stat will always punish goalkeepers who are more proactive and try to deal with more difficult crosses, as Jens Lehmann regularly did when he was at Arsenal.

Mad Jens - I miss him

Save percentage is a metric I've seen gaining more usage. It has some merit - outliers in either direction are probably indicative of some degree of a goalkeeper's ability - but it is a deeply suspect stat, because every shot is different. One goalkeeper might make three straightforward saves in a game, while the other might face only one shot, arrowed into the top corner. One has a 100% save percentage day, the other 0%. Over the course of the season, every goalkeeper is going to be involved of both of those scenarios. You can say they even out, but there is no scientifically valid reason they must. The variance in difficulty of shots faces by goalkeepers is far too great for any one-size-fits-all metric like save percentage. It is a somewhat more meaningful stat in ice hockey, a sport in which every game takes place in the same conditions. But a Premier League goalkeeper will play some games in howling winds, or rain, or with a low sun in his eyes, and all these factors further impact the difficulty of any given save.

As an example, imagine a shot from eighteen yards, from an angle forty-five degrees to the goalkeeper's left. There is a considerable difference in degree of difficulty for the goalkeeper, and the likelihood of the attacker scoring, depending on whether this shot is struck with the left foot, which opens up the far post, or the right foot, which somewhat limits the attacker's chance to finish. Save percentage doesn't take these, or any, variables into account. It looks only at the number of shots and the number of goals conceded, effectively treating every shot as if it were the same.

Then there are the metrics that are dependent on team performance, such as goals allowed per game. Swansea City goalkeepers have allowed more than twice as many  goals as Southampton goalkeepers. Is this defensive record down entirely, or even mostly, to the respective goalkeepers? Of course it isn't. Team defending - overall team play, really - plays an enormous role in goals conceded. A team's style of play will also go a large way to determining how often the goalkeeper is needed to do things like sweep up behind his back four. Teams that sit deep and play on the counter attack do not put their goalkeeper in this situation very often; so negatively judging a goalkeeper based on his total interventions like this is, again, flawed.

No doubt, someone will develop more advanced metrics which do give us a better view of a goalkeeper's performance. Until then, the eye test is still the most reliable tool. This is always subjective, but then again, so are 'scientific' metrics like 'most errors leading to a goal.' Who decides what constitutes an error?

Please don't conclude that I am anti-analytics, or anti-technology. Nothing could be less true. I just don't believe the current metrics some use to judge goalkeepers are sound. Especially not ones that come to the conclusion that David De Gea, one of the most talented and effective goalkeepers of this generation, is among the '10 Worst Keepers of the Premier League.'

Tweet