Measuring the 2015 Diamondbacks defense: An introduction

The problem with defensive metrics is, there's no single figure that is universally agreed upon as reliable. With pitching, you can look at ERA+ to get a good feel for a pitcher's performance; some may prefer the likes of FIP, but the two are generally going to be pretty close to each other. Same with OPS+ and wRC+ - if a batter does well in one, they will probably be looked upon kindly by the other. But when it comes to an objective measure of defense, there is much less consensus. As we'll see, stats can supposedly be looking at the same plays by the same players, and come up with radically different outcomes.

This is an area which may improve in coming years. If you've been watching the post-season, you'll probably have seen the Statcast technology, providing information on how far an outfielder had to run to catch the ball. Eventually, the data captured by the system will become available for us to crunch, just as Pitch f/X data has done for pitchers, and will open the door to a whole new world of objective measurements. That day is not yet here, and what we have instead are a number of different systems that measure things in different ways, and so naturally come up with different opinions as a result.

Before we get into looking at the numbers for the Diamondbacks, it's probably best to have a quick refresher on the various systems, how they are figured out and their strengths and weaknesses. You probably want to combine most, if not all, of them into a final assessment, but the weighting you give to each will depend on how you feel about its reliability, and probably also how well it matches the "eye test". If a metric consistently conflicts with how you perceive the ability in the field of a player you see often, you're likely not going to give a lot of weight when it comes to assessing a player you do not know so well.

Errors and fielding percentage

Full-on baseball scorekeeping is much like accountancy: all the numbers have to add up, with every batter and out rigorously accounted for. If a hitter reaches base or advances one or more extra bases, not through his own efforts than because of a defensive miscue, an error is charged to the fielder responsible. Over the course of a season, the count of errors can provide a simple measure of a player's defensive ability. Divide the errors by the number of plays in which he was involved [put-outs + assists + errors] and you get a fielding percentage, which also takes into account playing time. While certainly simple, this has a number of major weaknesses.

Errors are subjective, being awarded by the official scorer. While some are obvious, others are not, the definition relying on vague terms like "ordinary effort". An error cannot be split; if a throw home can't be handled by the catcher, the scorer must decide whether he or the fielder get 100% of the error. Players with better range will likely also make more errors, because they simply get to balls other fielders wouldn't reach - the general rule is, if you don't touch the ball, it's a hit, not an error. And external factors certainly influence scorers, such as this, which despite a number of misplays, was called a triple [Regular readers will know what it is without having to click on the link!]

Another problem is that fielding percentage is absolutely dependent on position. A fielding percentage of .978 means something radically different at first-base and shortstop, because the former get credit for all those throws from the rest of the infield. In fact, .978 would be above average at short, but would make you, far and away, the worst regular player at first in the major leagues [your name would also be Pedro Alvarez, but we'll get to that in a future installment]. So, while F% can tell you how good a player is compared to other first basemen, it can't say how valuable he was with the glove compared to others on his team.

Total Zone Fielding Runs Above Average

Originally designed by Sean Smith of BaseballProjection.com, Total Zone FRAA was an attempt to refine assessing fielding ability, but also be workable from play-by-play information. This means it can give us figures for players going back as far as we have PBP records. Obviously, this is a lot further than we have, say, complete video footage or other data needed for the more complete systems, so will give us numbers by which we can compare historical figures to those in the present day.. The system provides a range of variations, depending on how specific the PBP information available is, and also includes factors for outfielder arm strength and infielder double-play rate.

However, in terms of assessing present day players, it is still quite simplistic. FRAA only looks at the most basic of information - was there an out or not - and does not take into account factors such as the difficulty of the chance. A shortstop could end up receive the same credit for a diving play deep in the hole, followed by a throw from his knees across to nail the runner at first, as for a routine ground-ball that they bobble, short-hop the throw over and are rescued by a good pick out of the dirt by the first-baseman. While the ability to go back deep into the historical archives is great, it largely ignore the fact we do now have better data available.

Baseball Info Solutions: Defensive Runs Saved + Ultimate Zone Rating

The company BIS is responsible for putting together the most informative, widely-available database of defensive information, which is used to figure out the metrics used by the two major baseball stats sties. Defensive Runs Saves (DRS) shows up on Baseball-Reference.com, while Ultimate Zone Rating (UZR) appears on Fangraphs. Since they are based on the same basic information, they will tend to head toward the same results, but they aren't quite the same, so you should be aware of the differences between the two systems, even if they work in similar ways.

The core idea in both is figuring out both how often a specific play gets made, and what the impact of making it is, compared to if it wasn't made. For example, based on a batted ball's hit speed and location, the left-fielder converts it to an out 20% of the time. If David Peralta gets to the ball, hooray! He gets 0.80 points. If he doesn't he is assessed -0.20 points. Now, for DRS, that is then converted into runs, based on the situation of the bases. Peralta's catch is worth a lot more runs "saved" with the bases loaded and two outs, than if they are empty, so we can figure out exactly how valuable the catch was. Total that up over the season, for all David's play, and you get his DRS.

Well, kinda: for there are also other factors that go into the final tally, depending on position. For example, outfielders are also assessed on saving extra bases; pitchers and catchers on stolen bases; infielders on their ability to turn double-plays, etc. Additionally, BIS scouts assess good defensive work and misplays, in what's effectively a more sophisticated version of "errors", that works in both directions, to reward the best and penalize gaffes. If you want more specific information, BIS employee Dan Merqury wrote an extended piece over on Athletics Nation that goes into additional detail. I stole this paragraph from it:

Simply, what is the difference between DRS and UZR?

While both get their data from BIS and use very similar methods (especially in the Plus/Minus calculation), DRS has batted ball timer data, while UZR does not. The GFP/DME [Good Fielding Plays/Defensive Misplays] system is also exclusive to DRS, while UZR attempts to measure something similar by adding up errors.

Similarly, if you want more info on UZR, there's plenty available. The bottom line, however, is that both come up with a number showing the number of runs above (or below) average for that player or team, in that league and year. As such, there can be variations: if a league is blessed with particularly good shortstops one season, the value of plays there will go down as a result, because a higher percentage of them will be made. So this does make it harder to compare performances across seasons, since the quality of a player's colleagues affect the baseline to which he is compared. [Another difference is that to mitigate this, UZR uses multiple seasons for that baseline]

Summary

Much as you don't need to know much about internal combustion in order to drive a car, you don't really need to know the specifics about how UZR, etc. are calculated in order to look at them and draw conclusions. However, knowing their differences and limitations will help you appreciate what they tell you and what they can't. As with anything, knowing what you don't know, is likely every bit as important as knowing what you do know.

You should be aware, for example, that all defensive stats are liable to significant fluctuations, based entirely on random chance - we've all seen games where a player doesn't have a single ball hit their way. Just as a hitter or pitcher can have a good year because BABIP falls their way, so it is with defensive metrics. It doesn't make that season any less valuable, but when you have a series of data pointing one direction, and one going the other way, you should tend to believe the former with regard to future performance. If a previous butcher suddenly has a glowing UZR, it's more likely he has been lucky than he has bought a magic glove from a second-hand store!

And with that, we'll move on, and will be starting with first-base, where Paul Goldschmidt seeks to reclaim his crown as the Gold Glove winner in the National League...