How to ruin an advanced stat

If you watch sports like mine, there is no doubt that you have recently encountered some statistics conducted by AWS. This kind of insight is everywhere – but why? My knee-jerk response to them is often, “Hmm! I don’t know about him. ” Not because I’m an advanced statistics skeptic, but because what I see while watching the broadcast rarely seems useful or entertaining, which is, and if I’m wrong here, correct me, not an insight that’s supposed to be.

AWS is Amazon Web Services, a company that offers customers products such as machine learning tools and cloud storage and servers, in partnership with various sports leagues. A stated purpose of the AWS partnership is to increase the audience’s understanding and engagement in real-time broadcasting. (AWS does a lot more on the rear end as well.) The company partnered with Formula One in 2018 and since then, has provided resources for vehicle development and, at the fan-facing end, television graphics that range from enhancing driver performance to car fantasy. An abstract percentage of performance that analyzes obligatory angles using driver’s telemetry data to explain why a driver was faster around a turn and why. AWS has since partnered with several different sports leagues, including the Bundesliga, the NFL, the PGA Tour and the NHL, much to the same end.

Most recently on the NHL front, a face-off probability status called NHL Edge IQ Face-Off Probability has been introduced by AWS. In a SportsNet broadcast, the graphic looks like this:

Screenshot: Sportsnet

My initial reaction was, I will admit, not the scary kind. On the other hand, I have no current feelings. Where did the numbers come from? Are they right? Why do percentages have three significant figures – .1 percent really important? What does it actually do to influence how I view the game? Thanks for telling me that face-off is about 50-50?

A little research, and the answer to the first question — or some of the answers — ends:

Priya Ponnapalli, senior manager at Amazon Machine Learning Solutions Lab, said Face-Off Probability uses more than 70 different data points from historical and in-game statistics as well as relevant data. Ponnapalli said artificial intelligence takes time for 10 years of faceoff results – more than 200,000 draws for all players in the league – and uses data that includes a player’s success rate based on faceoff position, home game vs. away games and history against specific opponents. There are. . It also causes personal data such as hand, height and weight.


It helps a bit. A smiling face in a foggy window pane, if you will. But machine learning can be obscured by nature ব Ben Clemens’ very good article in Fangrafe discusses issues similar to those shown on Apple TV baseball broadcasts. In short, what machine learning does is a set of sample data, or “training data”, which has various parameters that you think affect the outcome (in terms of face-off probability, location, home vs. distance, etc.), and, your bug -Learn how to predict future results with different values ​​of the same parameters, different from standard analysis which only serves to draw a conclusion about the dataset.

It is easier for people to make predictions based on just one topic, such as a head-to-head match-up, but add more variables and it becomes more complicated to implement and evaluate. You can see the quote above and say that the weight of the player seems to be an insignificant factor in the face-off percentage, especially since the players consider themselves, but may not be an imperfect algorithm, or vice versa. Unfortunately, the only way to verify a machine learning algorithm without having it in hand is to see how well its predictions match the actual results.

There are many statistics available publicly nowadays; This event is not here. All that remains is to manually collect a sample of the predictions where we can find them. Thanks for the branding nature, A search for “NHL Edge IQ” on NHL Twitter Somewhat misleadingly tweeted face-off predictions are available, but only 10. So I decided to manually tweak the Oilers and Avalanche series via SportSnet broadcasts so that the play-by-play of each game is open in another tab – look-off and look for a face-off potential graphic. The process is time consuming and my computer starts to hiss towards me in about the 10th way, but I thought that due to the lot of face-offs in the game of hockey I would now find quite a few graphics that I was looking for. For them.

I was stupid.

Game 1 had two graphics, Game 2 had two graphics, Game 3 had none, and Game 4 had two (at least that’s what I calculated, although the graphics are hard to miss because the follow-up slide “NHL Edge IQ powered by AWS” possibility It lasts an average of 1.5 per game, which scores at home, which is not a very good rate for a self-respecting blogger who likes to do things like going out, so I canceled my plan to create an organization that Can’t get a sample, even a small child complains of size. (Everything I’ve recorded is here.) Alas, I have to wait another day to look at the possibilities.

One thing I’ve finally gotten out of my predicament, though, is this: SportSnet Broadcast has shown face-off probability standards for two almost identical Leon Drysitel-Darren Helm games 1 and 2. Both were in the Edmonton power play late in the second season, in the same place, on the ice with the same players. In Game 1, DrySittle had a 53.1 percent chance of winning the draw. According to the official play-by-play, he lost. In Game 2, Drysettle had a 54.9 percent chance of winning the draw and losing (probably again, depending on your perspective on events).

The gap could be explained by other factors covered by the NHL Edge IQ, such as a minute and break change or in-game face-off performance, but I had a broad takeaway that I, well, didn’t pay attention to. Maybe your mileage will be too high, but I was shown the accuracy in the broadcast at 0.1 precent for a possibility that said Dresitol A little The chances of a face-to-face win are high but mostly it was a toss-up, and I did nothing for it. Don’t worry about the accuracy of the numbers, this is the biggest flaw of NHL Edge IQ graphic: seeing the numbers ফেস the 1.8 percent gap between the face-off and unrefined probability of both broadcasts – does nothing.

The statistics don’t have to be important to show or discuss (see the various nonsense I’ve pointed out so far on this site), and the face-off prediction statistics, if accurate, are not useless (coaching for this, or perhaps gambling). But the NHL rarely uses Twitter and broadcast Face-off predictions, partly because the nature of the stat makes it difficult to do. This is a damn shame because statistics, when performed well, can do what the NHL Edge IQ promises: creating excitement, understanding or entertainment by feeding on stories that unfold on screen. Like Mookie Bates insists on May statistics whenever he comes to bat until you realize that something is going to happen, or more surprisingly, nothing will happen; And knowing the xG value of the shot, how fluffy a target was, or how impressive the save was, or How incredible Jake Ottinger was during the seven-game series; And just watching a joke about the shortest shorts in MLB history.

There are great storytellers who use statistics, whether broadcast or not; The NHL is not currently one of them. If I don’t know what a face-off possibility adds, the NHL and hockey broadcasters certainly don’t. The message that rallied around the possibility of a face-off appeared to be “ignore it” or “face-off matter”, with each video shown on NHL Twitter ending in a goal. A commonly misunderstood NHL network segment that misleadingly concludes semantically by discussing face-off potential. But the face-off does not show the possibility that the face-off is important. One of the reasons you should be careful about presenting the status on the broadcast is face-off mating, and the repeated emphasis is that the NHL is trying to reverse-engineer a justification for buying a fancy new tool.

This seems to be the purpose of the statistics, rather than to enhance our outlook on the game. In the case of Apple TV statistics, no matter what the flaws, I can buy that they are at least trying to report. The way you saw baseball because they cropped every at-bat. Face-off probability: Used appropriately, used sparingly and ineffectively, and, if I find it particularly remarkable, is nothing more than an AWS ad rather than a crop-up at a similar point in each game. Percentage has a flow in and out of your mind leaving just as much behind as a wave. This is the logo on their side to stick to. Of course, any organization would partner with the Sports League because of advertising. We see it all the time: Fifth Innings, sponsored by your local car dealership, Jim Nantz is listening to CBS’s upcoming primetime schedule. It is bad to see advanced figures flattened into the same structure where they turn into invalid numbers, a thin excuse for a corporate logo to claim a few more seconds of wind time.

Advanced statistics are in a golden age right now, both in terms of access and the sheer amount of new data available. AWS even actually gives it some power. But the face-off probability gives a hint of the risk of this particular type of advanced-statistical diffusion: the celebration of a tool that creates accepted numbers at the expense of finding any real value in those numbers. It is a tragedy.

Related Posts

Leave a Reply

Your email address will not be published.