Good players or good results?

CJ Turtoro combs through the New Jersey Devils' roster and attempts to answer just that in his latest.

Be sure to join the Discord channel to talk hockey with our writers and subscribers.

---

By CJ Turtoro (@CJTDevil)

The New Jersey Devils just passed the quarter mark on the season. Normally, this is a good opportunity to do a quick audit on the assets as we have accumulated enough data that a few metrics we rely on to judge a player’s impact have begun to stabilize.

Around this time, I like to try and answer a couple of basic questions about players and their performance so far in the season.

The first question is “are the results good/bad?” and the second question is “are the results good/bad enough to change our assessment of the player?”

To do this, we need what is called a “value” metric and a “talent” metric. For my proxies, I’ll be using the Evolving-Hockey Goals Above Replacement and HockeyViz’s Synthetic Goals.

For the uninitiated let me just briefly (I’ll try, I promise) explain the difference between the two metrics. If you don’t care, skip down to where you see the graph with the pretty colors.

Evolving-Hockey contains two value models named GAR and xGAR. GAR is a complicated suite of inputs but it’s most helpful to think of it as the statistical impact a player has on goals scored (GF) minus the statistical impact a player has on expected goals against (xGA).

If you’re familiar with RelT metrics, they get you 95% of the way there. If you’re not, just think of it as the difference between what you’d project their GF/xGA to be based on their teammates/opponents/usage and what actually happened.

For xGAR, it’s similar but gives far more credit to an individual shooter and far less to their teammates a la War-on-Ice’s model. They each also value penalty differential adjusted for circumstances. In this piece, I average the two for stability.

Since we don’t want to overfit the small sample of current data, I find it prudent to compare these results to another model that is weighted to each player’s prior performance.

In this piece, I’ll use Hockeyviz’s Synthetic Goals metric. Synthetic Goals is a unique statistic that measures player impact on xGs and penalties similar to how we described above, but then also measures their ability to overachieve those offensively through passing/shooting.

Every year, the player starts with every skill where they left the previous season and they get better/worse depending on their performance. For units, it measures impact in terms of 1200 total minutes (1000 even-strength, 100 powerplay, 100, penalty kill). 

For the purposes of apples-to-apples comparisons insofar as they’re possible, I’ve used the GAA (goals above average) metrics rather than GAR from Evolving-Wild, and converted them into per-1200 minute metrics.

If we graph synthetic goals vs the Goals Above Average on an x-y axis and plot the Devils players, we get the graph shown below. Here’s the pretty colors!

Time to discuss! First, let’s get everything unsurprising out of the way.

Jack, Nico, and Dougie are in the “Good Player, Good Results” quadrant. Bratt would be there too, but, as I said on X, I think that’s a combination of bad shooting luck, some weird early penalty results, and what I believe to be a bit of a misattribution of xG blame from Jack’s slow start. His numbers will come around and have already started to.

Also not surprising guys that aren’t on the team (Casey, Nemec) or were forced here by injury (Dowling, MacDermid) are in the “Bad Player, Bad Results” quadrant. Nemec and Casey are perhaps disappointing to some, but they’ll have plenty of time to disprove those numbers later in their careers.

And it’s not surprising that Bastian, Mercer, Palat, Lazar, Luke, Pesce, and Haula are all “meh”. Mostly middle-sx guys doing middle-six stuff. 

Let me talk briefly about everyone whose name I didn’t just mention. And I think it helps to juxtapose in each case with two players in each. In each case I’ll give my take, followed by which stat I’m cheating towards.

Kovacevic vs Siegenthaler