Fielding Metrics, Part 3: To Infinity, and Beyoncé!

by Jim McLennan on Jan 14, 2009 1:22 AM EST Comment 25 comments

Previously, we had Fielding Metrics Made Easy, and Fielding Metrics Made Somewhat Complex. Now, we've basically reached the level of numbers which I can only make a vague stab at clarifying. I have read the explanations, and despite my high tolerance for figures [two years of college statistics and a mis-spent youth playing D&D], my brain still got glazed over faster than a batch of hot Krispy Kremes. I suspect that these are numbers which only God and that bloke off Fringe can truly claim to understand.

An example would be Clay Davenport's work for Baseball Prospectus, which results in a measure called FRAA, for Fielding Runs Above Average. The BP site seems quite coy about the precise scheme behind it,, which appears to be semi-proprietary. But second-hand reports suggest "It takes a fielder's range factor and makes adjustments for innings played, team defense, estimated ground ball/fly ball data, strike outs and some other things. It then compares the range to league average and estimates the number of runs saved above or below average." Pixie-dust may, or may not, be among the "other things" which are taken into account. There are a good example of baseball statistics as a religion; you just have to have faith, and accept the dogma sent down from on high.

Going even beyond FRAA, there is FRAA2, which "incorporates adjustments for league difficulty and normalizes defensive statistics over time." That sounds shiny and impressive, so I'm going with that. I'm not quite sure how the system tracks players like Jackson, who played multiple positions; I assume it probably sums the values obtained at 1B and LF. Leading the way for Arizona here is Orlando Hudson, with a rating of +13. Chris Young had the same FRAA total, but played a lot more innings at his position. Unlike some of the defensive metrics, it does measure catchers. and Chris Snyder performed well, at +11, ahead of - and this might be a surprise, Chris Burke, with +9. Conor Jackson rounds out the top five, scoring +8.

At the other end of the spectrum... FRAA2 gives a resounding raspberry to Mark Reynolds, with only one player in the majors getting a worse rating than the -18 awarded to Reynolds [that being Rockies' right-fielder Brad Hawpe, at -23 - though see the comments at the end of Part 2, about the dangers of directly comparing stats across positions], Special K's figure was twice as much as that of the next worst Arizona player. That dubious honor went to Justin Upton (-9), though Stephen Drew wasn't all the far behind at -7. Backup outfielders Alex Romero (-5) and Jeff Salazar (-4) were also significantly negative.

Finally, Fangraphs.com recently added another fielding metric to the stats on their site. This one is UZR - Ultimate Zone Ratings. If you want more info on how it's worked out, please see here, but there are two factors involved, Range Runs and Error Runs. The former is the runs +/- average, determined by how the fielder is at getting to balls hit in his vicinity. The latter is the same, but based on the number of errors compared to an average fielder. Both stats depend on the distribution of balls in play hit to him, and their sum gives you the overall UZR. So it combines, to a certain extent, the subjective measure of errors, and the objective one of the difference in balls reached, compared to an average player. The guy who originally came up with UZR, Mitchel Lichtman, now works for the St. Louis Cardinals.

The following chart has all the Diamondbacks who played 100 innings or more at a single position. Also shown are the Range Runs and the Error Runs, with the sum of these being the UZR. The final column scales the UZR to take into account the different number of innings played.

Name	Pos	Inn	RngR	ErrR	UZR	UZR/150
Adam Dunn	1B	128	-1.7	-1.1	-2.8	-26.1
Tony Clark	1B	133	0.5	0.5	1.0	6.9
Chad Tracy	1B	523	-3.6	0.8	-2.8	-6.5
Conor Jackson	1B	571.2	1.9	0.1	2.0	4.4
David Eckstein	2B	152	-1.6	1.0	-0.7	-4.3
Augie Ojeda	2B	286	1.9	1.2	3.1	15.3
Orlando Hudson	2B	904.2	-5.5	-0.6	-6.1	-9.1
Augie Ojeda	SS	126.2	3.8	-0.9	2.9	39.4
Stephen Drew	SS	1294.1	-11.7	-1.0	-12.7	-14.5
Augie Ojeda	3B	110.1	-0.1	1.2	1.1	9.6
Mark Reynolds	3B	1288.1	6.4	-8.3	-1.9	-2.2
Chris Burke	LF	116.1	4.2	0.2	4.5	33.5
Jeff Salazar	LF	133.1	0.0	0.1	0.1	2.0
Eric Byrnes	LF	419.2	-3.6	0.3	-3.4	-12.9
Conor Jackson	LF	656	4.2	0.4	4.6	9.8
Chris Young	CF	1390	1.0	0.0	1.0	1.0
Adam Dunn	RF	182.2	-10.0	0.0	-10.0	-71.2
Alexander Romero	RF	221	-1.1	-0.1	-1.2	-9.2
Justin Upton	RF	860.1	-0.6	-2.2	-2.8	-4.8

Looking at the results here, the phrase that springs immediately to mind is: "God, Adam Dunn sucked, didn't he?" We could probably have put a zombie in right - and one of the slow, Romero ones at that - and got better range than Dunn showed. He was somewhat better at first, albeit in part because we got to see less of him there. In contrast, notice the great performance of Augie Ojeda all over the diamond: if you add up his UZR at those three positions, he comes out at 7.1, the best number of anyone on the team. Given the questionable numbers for most of our infield [and in case you're wondering, Felipe Lopez at 2B was -3.5/-0.6/-4.1/-4.6 - which are actually better numbers than O-Dawg last season], expect Ojeda to get most of his starts with Brandon Webb on the mound.

It's also interesting to contrast the results produced on the left-hand side of the infield, by Reynolds and Drew. The split of Special K's numbers is particularly stark: his Range Runs were actually the best on the team, but were wiped out by the negative impact of all the Error Runs he accumulated. On the other hand, Drew had very steady hands, but his poor range suggests an alternative career for him would involve Stephen's use to block off lanes during road-works on the Loop 101. These two provide a contrast in how the old math [errors and fielding percentage] may not paint the entire picture of defensive skills.

Moving to the outfield, our 'best' glove, taking playing time into account, was Chris Burke? That may be the first - and only - time in 2008, that the words "best" and "Chris Burke" have been used in the same sentence. UZR concurs with most of the other metrics as far as Jackson goes, finding him a good bit better in RF than at 1B [the shift in position there isn't too drastic, as far as difficulty goes] Chris Young doesn't come off particularly well here: he's positive, but only just so. The right-field position appears to have been a Bermuda Triangle of Defensive Suckitude, pretty much regardless who was there: as noted, Dunn was the worst, but neither Romero nor Upton proved a net positive there.

So, what have we learned? Probably that there is still no one metric, as yet, which we can point to and say, "That is an accurate measurement of defensive skill." For some players (such as Stephen Drew), there does appear to be a general consensus of the various systems, which can cause us to label a player as better than most, average. or deficient with the glove. However, there are others for whom there is no such agreement. Orlando Hudson would be one: FRAA2 thinks he's the best defender on the team, while UZR laughs sarcastically at that, and reckons O-Dawg is well below average. Research will, no doubt, continue. But if I don't see another fielding metric before Opening Day, I can't say I'll mind too much.

0 recs | Comment 25 comments |

Story-email Email Printer Print

Comments

::head explodes::

"Of course, it’s downright frightening to imagine how two Adam Dunns would turn the double play." - Joe Posnanski

by DbacksSkins on Jan 14, 2009 2:18 AM EST reply actions 0 recs

I have to say, though,

I think we’ve finally found some metrics that even foulpole can get behind — with FRAA2 listing O-Dawg as the best and Upton as one of the worst. If foulpole ever shows up again, we should tell him we’ve found a fielding metric he’ll like. (Despite the universally poor numbers for Stephen “Wolf” Drew)

"Of course, it’s downright frightening to imagine how two Adam Dunns would turn the double play." - Joe Posnanski

by DbacksSkins on Jan 14, 2009 10:49 AM EST up reply actions 0 recs

But Wolf

is TEH BOMB!

If you make every game a life-and-death thing, you're going to have problems. You'll be dead a lot.

by unnamedDBacksfan on Jan 14, 2009 11:24 AM EST up reply actions 0 recs

He IS

solid. Sweet!

"Of course, it’s downright frightening to imagine how two Adam Dunns would turn the double play." - Joe Posnanski

by DbacksSkins on Jan 14, 2009 11:28 AM EST up reply actions 0 recs

(Quite) a few years ago Dave Cameron put up a pretty good analysis of the various defensive metrics available on the USS Mariner site. I lost the bookmark, but you can probably still find it in their archives and it covers most of what has been discussed in these articles and does a really good job of explaining them in plain english. I can offer a bit of info as it relates to the history and efficacy of these systems, but I’m really just cliff noting Dave’s article:

For the most part there are 2 kinds of defensive systems, those based on pitch by pitch data, and everything else (with the possible exception of zone rating, which looks like a proto PBP system). Pinto’s PMR, Dewan’s +/-, and UZR (which was invented by the sabermetrician Michael Lichtman, who goes by the handle MGL on the internet) are all PBP systems. They are probably the best (or most accurate), but they all work a bit differently from each other, so it’s easiest to compare all of them as a guide. You can figure where they all agree (or point in the right direction), you’ve got a good handle on what’s going on.

Of the non-PBP systems, BP’s stats are designed so that you can use their databases to compare players across eras, and that means that everything has to be found in a box score. Their fielding stats are no different. In the first Fielding Bible, Bill James introduced a modified version of his own Range Factor stat, which ended up looking like he reverse engineered BP’s FRAA. FWIW most sabermetricians (or people who follow baseball stats) don’t take BP’s fielding metrics very seriously. You may notice that even BP writers are writing about defense in terms of UZR or +/- now…

Of the PBP stats, Pinto’s is the one I know the least of. He’s buying the data himself and putting it into a system, but I’ve always found it kind of hard to read.

UZR has been around the longest, and it’s gone through a lot of incarnations. For quite awhile it wasn’t even available as MGL was hired by the St Louis Cardinals as a consultant, so he wasn’t able to publish his work. Lately he’s trying to understand why the numbers he got from his system changed so much when he switched data providers (from Stats to BIS). The important number to look at is UZR/150 which normalizes a player’s performance across an entire season so you can compare the defense of 2 players who played a very different number of innings. That number is expressed as runs saved (or cost, if negative) over 150 innings compared to the average at that position.

+/- in the Feilding Bible was developed by Dewan and Bill James, and came along just after MGL had to stop publishing UZR. I think they’re both good, but the thing to keep in mind about +/- is that the number is expressed in plays or enchanced plays (which really amounts to the number of singles) above/ below average at that position. To compare it to UZR you need to multiply the number of plays by the value of a single (which is something like .82 runs).

The last thing to keep in mind is that a few teams have developed their own defensive metrics similar to these. I know that both the Red Sox and A’s have their own, and they’re probably not alone (in addition to the teams that buy +/- data)…

"I'd like to f*ck Sandra Bullock." - Pedro Martinez, explaining his secret ambition to Sports Illustrated for Kids.

by OCD SS on Jan 14, 2009 10:02 PM EST reply actions 0 recs

Welcome

to the ’Pit!

"Of course, it’s downright frightening to imagine how two Adam Dunns would turn the double play." - Joe Posnanski

by DbacksSkins on Jan 14, 2009 10:30 PM EST up reply actions 0 recs

And

thanks for the summary of various defensive metrics.

"Of course, it’s downright frightening to imagine how two Adam Dunns would turn the double play." - Joe Posnanski

by DbacksSkins on Jan 14, 2009 10:30 PM EST up reply actions 0 recs

Thanks. I’m actually a Red Sox fan stopping into look at the opposition’s take on a possible Montero trade, but defensive metrics are somthing I’ve followed with interest for awhile…

The other thing I should add (and I apologize if I missed it being discussed in one of the 3 posts) is that most sabermetricians think of fielding as being something that is fairly variable from year to year, like batting average. They usually want 3 years of data to try and gauge “true talent level.” Sometimes guys just have crazy good years, or sometimes they just get really unlucky. looking at one year will tell you how a guy did, looking at his trends over several is more likely to tell you what he likely will do.

"I'd like to f*ck Sandra Bullock." - Pedro Martinez, explaining his secret ambition to Sports Illustrated for Kids.

by OCD SS on Jan 15, 2009 8:16 AM EST reply actions 0 recs

Speaking of which....

what’s your take on the possible Montero trade?

"Of course, it’s downright frightening to imagine how two Adam Dunns would turn the double play." - Joe Posnanski

by DbacksSkins on Jan 15, 2009 11:17 AM EST up reply actions 0 recs

If you go by reports of what Theo will offer, I think the Sox probably prefer one Texas’s young catchers. However I think the problem with either of them is that they do carry some risk (will Salty stick behind the dish or will Tegarden get injured/ regress at the plate because of his K rate), and the Sox are trying to hedge their risk by not giving up too much. Of course it helps that Daniels demands are either Buchholz or 2 of Masterson/ Bowden/ Hagadone…

The fit with AZ seems to stem from that they want less, but I guess the risk they see in him is that he never progresses enough with his defense to be considered anything more than a back up. I haven’t followed him enough to know, but his CS numbers and scouting reports on his arm aren’t encouraging.

I’m on the fence about a straight up deal of Monty for Bowden, but I think Theo is going to use AZ to drive down the price for a Texas C, and if it doesn’t happen he’ll bite the bullet and trade for Monty. However if Monty becomes available for Bard + he might jump on that….

My lack of knowledge about Monty is a problem though. I think the Sox would like to get someone who would be a good defender behind the dish if they can…

"I'd like to f*ck Sandra Bullock." - Pedro Martinez, explaining his secret ambition to Sports Illustrated for Kids.

by OCD SS on Jan 15, 2009 1:45 PM EST reply actions 0 recs

J.Byrnes seems unlikely to go much lower,

so I don’t think we’ll help you much with Texas. As Jim pointed out in another piece, we don’t NEED to trade Miggy; he’s a luxury, and we don’t have any particularly glaring holes (especially with the expected weakness of the division taken into account. He’s shown steady offensive improvement since he’s entered the majors. The other thing to note is that Miggy never played catcher until he reached the Dbacks’ system, so his defense would only be expected to improve over time. Having essentially been Randy Johnson’s personal catcher this past season shouldn’t have hurt his development, either — talk about a chance to learn.

"Of course, it’s downright frightening to imagine how two Adam Dunns would turn the double play." - Joe Posnanski

by DbacksSkins on Jan 15, 2009 3:50 PM EST up reply actions 0 recs

The only reason I can think of that Byrnes might go lower is if it helps the D-backs make their budget, but Monty isn’t expensive, so it would only matter if AZ was getting a subsidized ML part back (or if the Sox took on some salary). Neither seem that likely.

I’m not so sure that I wouldn’t trade Bowden for Monty (I flip flop on this quite a bit). Here’s a poll of what some Sox fans think about that deal. It’s closer than Buchholz for Salty…

"I'd like to f*ck Sandra Bullock." - Pedro Martinez, explaining his secret ambition to Sports Illustrated for Kids.

by OCD SS on Jan 16, 2009 10:10 AM EST up reply actions 0 recs

Hmmm.... link's not working.

However, I was at Sons of Sam Horn yesterday.

I think these fan polls, as well as the comments on the trade from both sides, tend to support the assertion that every team’s fans overestimate the value of their own prospects.

"Of course, it’s downright frightening to imagine how two Adam Dunns would turn the double play." - Joe Posnanski

by DbacksSkins on Jan 16, 2009 11:03 AM EST up reply actions 0 recs

The comments I read around 1pm was that people weren’t in favor of it becuase they don’t think Monty stands to be a long term solution.

"I'd like to f*ck Sandra Bullock." - Pedro Martinez, explaining his secret ambition to Sports Illustrated for Kids.

by OCD SS on Jan 16, 2009 1:52 PM EST up reply actions 0 recs

Hudson vs. Lopez

I was shocked to hear Lopez was better than O-Dawg last season, so I dug a little deeper into Fangraph’s UZR. First, a very minor correction. Yiou picked up Felipe’s career numbers at 2nd by mistake. They’re not much different; he had had very little PT there prior to 2008. Still, he did manage to earn a positive UZR in both 2005 and 2007. 2008 by itself is -3.8/-1.4/-5.2/-7.9. Still not as bad as Hudson, but the difference is down to just one run.

Which raises the question: Why do we all think of Orlando as a good fielder if he really isn’t? I think I may have an answer. In his last 3 years with Toronto he had a positive UZR every year (3.4, 16.4, 6.3). Siince coming here he’s been negative every year (-4.2, -2.4, -6.1). We acquired him in part because of his reputation as a good fielder. Reputations take longer to change than actual performance does.

BTW, this is a good example of why you look at more than one year of fielding data. He had one great year in Canada and a couple good ones. But since then consistent mediocrity.

Getting back to Lopez, he spent almost all his time at SS prior to 2008, so he figures to improve some at 2B. But why not leave him at SS? His career UZR/150 there is -9.2. That’s bad, but still a lot better than Drew’s -13.5. And Drew is not improving. Why not move him to second where his lack of range won’t be as big a problem?

by fjm235 on Jan 15, 2009 4:51 PM EST reply actions 0 recs

One Good Defensive Year = Defensive Reputation = Gold Glove + Hype = Defensive Reputation+1 = Gold Glove*2+Hype*2 = Defen….. /loop&

Ah, well, you get it. Once a player has one exceptional defensive year, (or just a decent defensive year coupled with an excellent offensive year) more often than not, unless they’re truly terrible, they’re a shoe-in for next season’s award, thus allowing Mark Grace to say, “[two]-time Gold Glover Orlando Hudson….” and adding to O-Dawg’s reputation, and the chain reaction continues. Either one is coproductive with the other.

Anyone else remember the year that Raffy Palmeiro won the Gold Glove for playing DH?

/rant

& — Sorry. I’ve been programming all day. /explanation

"Of course, it’s downright frightening to imagine how two Adam Dunns would turn the double play." - Joe Posnanski

by DbacksSkins on Jan 15, 2009 5:16 PM EST up reply actions 0 recs

Lopez SS for a year?

I’d consider moving Drew over to 2nd IF we were talking about a long term solution at SS. Lopez is here for a year and is one of those “damned if you do, damned if you don’t” situtaitons. If he’s the second half of ‘07 player, we can’t afford him next year. But if he’s the first half version not even the D-backs will want him. One year of that is just not enough to justify moving Drew and disrupting his learning curve at short.

How foolish am I to think forward a year at the possibility of Brian Roberts coming over to play 2nd next year? I could get on board with that.

by Counsellmember on Jan 15, 2009 5:21 PM EST up reply actions 0 recs

Bye bye, spammer.

"Of course, it’s downright frightening to imagine how two Adam Dunns would turn the double play." - Joe Posnanski

by DbacksSkins on Jan 15, 2009 7:44 PM EST reply actions 0 recs

Great comments, everyone

OCD SS in particular [is that Martinez quote in your SIG for real?]. Interesting to read that a change on data provider had such a marked effect on the results. Salutory to remember that there is still a significant human element in play here – was that a liner or a fly-ball? I’m leaning towards liking UZR, but I think fielding covers a broader spectrum of actions than, say, hitting. There, you have much more black or white, success or failure, rather than the ‘would an average player have got to that ball?’ concept necessary to evaluate fielding. Distilling it all to a single number may be a pipe-dream.

As with hitting, players can have good or bad years. I was inclined to think numbers might be more variable for the outfield because of the smaller number of chances, but the difference isn’t as much as I suspected there [Drew = 568 PO+A, Young = 398, Reynolds = 322, for our three most regular players]. Be interesting to see if there has been some kind of study done, correlating these metrics over years to see what kind of predictive value they have.

Nice catch on it being Lopez’s career numbers: will tweak the paragraph somehow. I think Hudson’s rep exceeds his actual play thanks to the vacuuming of pop-ups and his ability to make spectacular plays, which another second baseman could, perhaps make routine. Certainly, I was never as enamored of his defense as many [you know who you are!] and was not shocked to find Augie a better option with the glove, in almost all the systems.

Yeah, I don’t think moving Lopez to SS is any real solution. It’d just replace one set of issues with another one, and we still need to find [be that trade or prospect] someone to fill a middle-position slot in 2010. It almost certainly won’t be Lopez, for the reasons Counsellmember points out.

by Jim McLennan on Jan 15, 2009 9:08 PM EST reply actions 0 recs