Twitter Moneypuck

[Martta Borromeo]

Variouspeople have made models to try to predict the outcomes of hockey games, including me.I think the problem is intrinsically interesting, and I find it helpful as a point around whichto organize much of my work in hockey. A good model has many aspects, chiefly:Sensible methods which give insight into the processes being modelled;Interpretability of inputs and outputs; andSuitably accurate and precise results.This article is about the last item.

The models under consideration here are:My model, Magnus, published on this site,Peter Tanner's model, published at Moneypuck.com,Dominik Luszcyzszyn's model, published on his twitter account@domluszczyszyn, (and at The Athletic)Matt Chernos's model, published on his twitter account@mchernos,The model published on the account of twitter user @hockeystatisti1,For models that output probabilities, there is no sensible way to say that a model is "right" or"wrong", such binary judgments apply to binary choices which these models do not make. Instead,I use using "logarithmic loss", that is, assigning ascore of -ln p to an event which occurs and which was predicted to occur with probability p. This loss function is called "logarithmic" because it is defined with thenatural logarithm function (ln) and is is commonly abbreviate to "log-loss".I like log-loss best because it is the only scoring rule which, if the people estimating the probabilitiesknow that it will used ahead of time to judge them, incentivizes them to quote probabilitieswhich exactly match their certainty of the events in question occurring.

Each point displayed is an individual game of the 2020-2021 season, with its log-loss-as-test-scorefor each model. Most obviously, most models are slightly better than the guessing benchmarkof 50 points, and the differences between models in overall performance are quite small. Moreinterestingly, despite the overall similarities in results, each model has a considerable spreadin results. Some models, like mine especially, are by nature conservative, assigning probabilitiesclose to 50%. Others, like Dominik Luszczyszyn's, are more extravagant,assigning probabilities far from 50% more often.

For a more difficult task than simply out-peforming a person guessing, I've also includedprobabilities corresponding to (approximately de-vigorished) Pinnacle closing lines, helpfully supplied for almost all games by Matt Chernos.

In addition to game-by-game accuracy, we would like our predictive models to be well-calibrated;that is, for any sufficiently-large set of games we would like the predicted percentage ofwinners to closely match the observed percentage of winners. I have two methods of forming such setstoday: binning the games by home-team win probability and binning them by time.Deciles of RiskOne common calibration measure is to form so-called "deciles of risk",that is, sorting all of the winning team probabilities (not the home team probabilities) for each model from lowest to highest, binning theminto ten bins, and comparing the predicted probabilities for each bin to the proportion of home teamwins in each bin.

For models without structural defects, these differences should be randomlydistributed around zero, as several of the models considered here are. However, it is impossibleto miss that my model specifically (the red line) is calibrated poorly; It's not sureenough when backing winners and not sure enough when fading losers.

Alternatively, we can sort the games by their playing date, and then compare the calibration ofthe models over time. Here the striking feature is how similar to one another many of the modelsare. Here again my model is the one that stands out for being unusual; dipping where all the othersrise near the end of the season.Probability vs Log-lossTo see just where individual models went wrong, it's helpful to consider a set of teamsand compare the probabilities given to the log-losses for those teams. First of all, we canconsider all home-teams:

For some very successful teams, like Colorado, Vegas, or Tampa, an easy way to score highly was to back themvery strongly. For other, very unsuccessful teams, like Buffalo,the easy way to score highly wasto strongly back their opponents. Most other teams don't show a clear "underrate"/"overrate"pattern.

We\u2019ll leave it to you to decide whether Bo Horvat\u2019s eight-year, $68 million contract extension is, in the words of New York Islanders GM Lou Lamoriello, \u201Ctoo long and too much money.\u201D But if it is either or both of those things, that\u2019s because the Islanders put themselves in a position to have to overpay in the first place. And they were prepared to do that because Lamoriello is pushing all of his chips to the middle of the table in an effort to win a Stanley Cup in the short-term.

Time will tell whether or not that notion is folly. On one hand, the Islanders are only a season-and-a-half removed from being a top-four team in the NHL in back-to-back years. On the other, they\u2019re in very serious danger of missing the playoffs for the second straight season. The website moneypuck.com has the Islanders\u2019 chances of making the playoffs in 2022-23 pegged at just 13.2 percent. But thanks to Horvat\u2019s extension, short-term doesn\u2019t necessarily mean just this season.

Going into Thursday, the three have played more minutes together than anyone on the team -- 162 minutes, 16 seconds according to naturalstattrick.com. Bonino was leading the team with 12 goals. Grimaldi already has reached a career high with 14 points.

Underlying analytics suggest the three have been one of the most productive lines in the league -- on both ends of the ice. Going into Thursday's game, they were fourth in the league in goals for per 60 minutes at 3.5 and 14th in goals against per 60 at 1.57, per moneypuck.com.

Numbers also suggest that Smith is due for a breakthrough. He's shooting just 2.9%, almost 6% lower than his career average. He averaged 2.7 shots per game last season, and is down to 2.34 this season.

"I'm angry when I go home at night," Smith said. "It doesn't work out sometimes. But those guys are playing great. ... There's really nothing to complain about. Yeah, it sucks personally. I want to chip in more. Those guys are the hot hand right now and we just have to ride it out."

Smith, who is in the final season of a five-year, $21.25 million contract, said he "would love to play better," that he isn't playing any differently that he did during his first eight years in the league.

3a8082e126