The enigma of the Arizona Diamondbacks bullpen

When the story of the 2018 season is eventually written, the bullpen will likely play a role larger than the innings pitched. But will they be the heroes or villains? How they can be seen as having performed, depends enormously on what you look at. By ERA, they are among the best in the league, with their 3.42 figure worse only than the Cubs, and well below the NL average (4.06). On the other hand, their record of 21-31 gives them the lowest win percentage in the National League. Maybe they’re both good AND bad, for they rank above average both in the number of shutdowns and meltdowns - outings which changed WP by more than 6%.

Let’s dig a little but further and see if we can find the “truth” about Arizona’s 2018 relief corps.

ERA vs. W-L record

We all know (or should!) the issues with pitching wins for individuals. They’re dependent on a number of other factors beyond that player’s performance. But at a team level, they are still useful, particularly in that they can show us when a game was decided. There is quite a strong correlation between ERA and record. This is as you’d expect: the lower an ERA, the less likely you are to lose, and the more likely you are to win. The chart below shows this, plotting bullpen ERA against relief win percentage for all thirty teams in the majors this season (the statistical correlation between the two figures was -0.64, which is quite significant),

The trend line shows the expected performance for a given ERA. If a team is above that, this means their bullpen has a better record than expected; below it means they have been worse, and the further away from the line, the greater the gap between the expected and actual performances. I’ve highlighted the dot which represents the D-backs bullpen, and you can see it’s further beneath the line than just about any other team. We can use the formula for the line [W% = (ERA - 6.05) / -3.85] to work out what the D-backs relief win percentage “should” be, given their ERA of 3.42. That’s a W% of .683, which over the 52 bullpen decisions is 35.5 wins. They have actually managed only 21.

We can do this at the individual level, and see which Arizona relievers have records most out of whack with their ERAs. Only one is significantly better. Fernando Salas, whose 4.50 ERA projects to a .403 W%, while his actual record was 4-4. At the other end of the spectrum, we find Andrew Chafin, whose 2.51 ERA is completely out of whack with his 1-5 record. Since the Diamondbacks entered the league in 1998, only three pure relievers have had a lower ERA and W% [min 40 IP and five decisions]. T.J. McFarland is another whose ERA (2.00) should have resulted in a significantly better record than it has (2-2).

Turning the calendar page

If we look at the D-backs bullpen over the course of the season, we see a not dissimilar pattern, of unrewarded performance. Over the first three months of the season, when they were lights out, they were not rewarded with the wins you’d expect based on ERA. Up until the break, they were only 16-16, despite a 2.85 ERA, the best in the National League by almost one-quarter of a run. From then on, the ERA has been a lot worse (4.87), and results have been closer to expectations: the ERA predicts a .306 W%, with the actual for Arizona at .250 (5-15). But there has not been a single calendar month to this point where the D-backs’ bullpen have had a better than expected W-L record.

In particular, how the heck do you throw almost a hundred innings in April with a 1.92 ERA, and go only 6-5? For comparison, the #2-4 teams in the majors that month combined for an ERA half a run worse (2.42), yet had a record of 21-8. Regardless of the reason, if the Diamondbacks had taken better advantage of that first-half performance, the additional wins would sure have been very welcome at this point.

The win (probability)’s the thing

If Arizona’s bullpen ranks highly for ERA and poorly for win percentage, when it comes to win probability, they are exactly in the middle of the National League. Our relievers have been worth a total of +263%. However, that is entirely due to Yoshihisa Hirano, who is at +305%. Everyone else else combined? -42%. The worst culprit there is Jake Diekman’s -94%, startling considering he has thrown only ten innings for us. If we look at the ratio of shutdown to meltdowns for our relievers, we perhaps get a clue as to Chafin’s problem. [MLB average ratio is about 1.75].

1. Yoshihisa Hirano - 4.63
2. Archie Bradley - 3.00
3. Brad Ziegler - 3.00
4. Brad Boxberger - 2.50
5. T.J. McFarland - 2.00
6. Fernando Salas - 1.50
7. Andrew Chafin - 1.07
8. Silvino Bracho - 1.00
9. Jorge De La Rosa - 0.63
10. Jake Diekman - 0.17
11. Matt Andriese - 0.00

Interesting to note both Bradley and Boxberger ranking highly here: I’m pretty sure few D-backs fans have much confidence in either right now. That may be a result of what I’d call their “super meltdowns”. If we look at games with a WP of plus or minus 20% (rather than 6%, as for the regular SD/MD stat), the MLB ratio for super shutdowns/super meltdown by relievers is about 1:2.2. It’s a lot easier to lose a game in relief than win it - putting up a zero, the best you can do, won’t move the needle much. But both Bradley and Boxberger have ratios well in excess of that, at 1:7 and 3:10 respectively.

But what we do see is our relievers ranks dead last in Fangraphs’ “Clutch” metric, in part because they have the highest pLI of any team in the league. Basically, the average at-bat for the Arizona bullpen has been more high-intensity than anyone else. Given this, their Win Probability should be a good deal higher than it is, because those are the times with the biggest potential to get a shutdown. However, this doesn’t necessarily imply much about their future performance: clutch pitching is about as elusive as a repeatable skill as clutch hitting [Daniel Descalso notwithstanding!]

Conclusions

I’m not really sure what the above proves. It seems fairly certain that the D-backs bullpen’s overall performance this year is not reflected in the games it has decided. But it’s not so much the recent games which show the biggest gap: of late, they have had their butts handed to them, and deservedly so. It’s the .500 record over the first half of the season, when they had the lowest relief ERA in the majors, which may well prove to be more significant in the final outcome.

Beyond that, the relievers seems to have struggled in the clutch, though we see a wide range of results across the individual players. It does seem that making Hirano the de facto closer is probably a good thing, though I am forced to wonder how many more games might have been saved if that decision had been made by Torey Lovullo a month ago. And if Jake Diekman doesn’t see any important innings for the rest of the season, I think I’ll be quite fine with that.