clock menu more-arrow no yes

Filed under:

How Top Pitching Prospects Began Their Careers

New, 10 comments

D-backs fans are excited about Trevor Bauer and Tyler Skaggs, and I certainly understand it. After all, Arizona has never had two prospect arms of the caliber of Bauer and Skaggs, and the trio of Bauer, Skaggs, and younger flamethrower Archie Bradley represents the best trio of pitching prospects in team history by a mile, distancing itself from the 2008 trio of Jarrod Parker, Brett Anderson, and Max Scherzer. While Bradley is further off, Bauer and Skaggs excite Arizona fans not just with their immense talent, but with the fact that they're already so close to seeing time on the major league stage, where prospect hopes can turn into something tangible for all fans to love and appreciate. As the respective #9 and #13 prospects on Baseball America's latest Top-100 list, Bauer and Skaggs look to have enormous long-term value for Arizona for years.

I certainly don't wish to rain on anyone's parade, but I think it's important to remember that although things have gone swimmingly for these young arms over the last year, it wasn't too long ago - just two years ago, in fact - that Brian Matusz was the #5-ranked prospect in the game by Baseball America. His 2010 rookie season was certainly a delight, but with his 2011 collapse, the once sure-fire hopes of the lefty have turned into a stream of unanswerable questions with one general premise: how can anybody explain a 10.69 ERA over 49.2 innings? In the end, these are pitchers, and things can go terribly wrong in a terribly short time with pitchers. With this in mind, I'd like to take a look not at what ultimate projections say these talented young arms are capable of, but at what history tells us they're likely to do early in their careers as a way of giving us a solid baseline for where we should set our expectations for Mssrs. Bauer and Skaggs.

First, let's go over the process.

With Bauer and Skaggs both slotting in in the top-15 of BA's most recent list, but both landing outside of the top-5 - the realm of Stephen Strasburgs and Matt Moores - we'll look exclusively at pitchers (of course) in that 6-15 region of BA's handy all-time top-100 rankings archive (which stretches back to 1990), to try to single out the "comparable" names. Certainly, different styles of pitcher will have different expectations early in their careers versus the long-run, so this is far from a rigorous method of defining comparables, but it should help leave out the subjectivity of hand-selecting pitchers that I think are comparable.

Hoping to capture the same sense of high short-term expectations that D-backs fans have for Bauer and Skaggs, we'll also look exclusively at pitchers who have at least reached the Double-A ranks at the time of their BA ranking. Pitchers who appear in the 6-15 range multiple times after reaching the Double-A level will be counted multiple times. The reasoning for this is that it's no guarantee that Bauer and/or Skaggs will use up their prospect status in 2012, and downright unlikely in my opinion that both will. If we're looking at short-term expectations horizons, excluding other pitchers who were thought of as highly from the sample until they used up their prospect status (and remained in the same 6-15 ranking tier) seems contradictory.

I'll be using Baseball-Reference's rWAR metric for this, because I want to capture actual results and expectations of performance for these pitchers, not expected FIP. FIP is a wonderful metric, but we're trying to see how valuable these pitchers will be, not how shiny their peripherals will be. Particularly considering that it is a reasonable hypothesis to say that young pitchers tend to have inferior command when compared to older pitchers' command and thus allow more solid contact, suffer higher hit rates, and, consequently, higher deviations between their ERA and FIP, I wouldn't feel comfortable relying on FIP in this instance. However, acknowledging the random variation component of ERA and rWAR, I've decided to look at the first two seasons of rWAR value for each prospect after their ranking, hoping that by growing the sample, we'll help to stabilize the rWAR and prevent massive luck-based fluctuations.

I'll have a master table for the all of the data, and create graphs of average value per prospect in the sample, sorting the data by year, by prospect rank (grouping them 6-10 & 11-15) and by handedness (since Bauer is a RHP and Skaggs is a LHP). From there, we'll hopefully have enough information at hand to make some sound conclusions, and try to fit the short-term horizons of Bauer and Skaggs into the context of Arizona's hopes for contention in 2012.


With the process established, let's get to the data. I wound up with 51 names, with four names appearing on the list more than once, those being Neftali Feliz (pre-2009, pre-2010), Matt Cain (pre-2005, pre-2006), Willie Banks (pre-1990, pre-1991), and Ryan Anderson (pre-2000, pre-2001, pre-2002). These repeat names are italicized in the master table. Here's the basic master data table we'll be using:

(Note: BA's list is published after New Year's day each off-season, meaning that a player ranked on the 2010 list will have just completed his 2009 season. The "Yr_Prior" column, therefore, is not congruent with the year of BA's list, it is the year of the most recent baseball season prior to their ranking. This was done to make it easier for me to check if the player had appeared in Double-A in the season prior to their ranking, and to prevent mistakes in this process.)

First Last Rank Yr_Prior Age Hand Yr_1 rWAR Yr_2 rWAR
Neftali Feliz 9 2009 21 R 2.3 1.6
Madison Bumgarner 14 2009 19 L 2.3 2.8
Brett Anderson 7 2008 20 L 2.4 2.2
Neftali Feliz 10 2008 20 R 1.1 2.3
Trevor Cahill 11 2008 20 R 1.9 4.0
Clayton Kershaw 7 2007 19 L 1.2 4.2
Franklin Morales 8 2007 21 L -0.1 0.1
Homer Bailey 9 2007 21 R -1.2 0.3
Jake McGee 15 2007 20 L 0.0 0.0
Francisco Liriano 6 2005 21 L 4.0 0.0
Chad Billingsley 7 2005 20 R 1.6 3.7
Justin Verlander 8 2005 22 R 3.7 4.2
Matt Cain 10 2005 20 R 2.4 3.9
Scott Kazmir 7 2004 20 L 3.7 4.4
Matt Cain 13 2004 19 R 1.6 2.4
Greg Miller 8 2003 18 L 0.0 0.0
Zack Greinke 14 2003 19 R 3.8 0.6
Francisco Rodriguez 10 2002 20 R 1.4 3.5
Ryan Anderson 14 2001 21 L 0.0 0.0
C.C. Sabathia 7 2000 19 L 2.7 2.8
Ryan Anderson 8 2000 20 L 0.0 0.0
Roy Oswalt 13 2000 22 R 3.9 6.2
Ryan Anderson 9 1999 19 L 0.0 0.0
John Patterson 10 1999 21 R 0.0 0.0
Mark Mulder 12 1999 21 L 0.2 5.2
Kip Wells 14 1999 22 R -0.3 0.6
Matt Riley 15 1999 19 L 0.0 0.0
Matt Clement 10 1998 23 R 0.0 -0.5
Roy Halladay 12 1998 21 R 2.5 -3.2
Kris Benson 7 1997 22 R 0.0 2.5
Carl Pavano 9 1997 21 R 0.7 0.4
Bartolo Colon 14 1996 23 R -0.2 4.1
Jason Schmidt 11 1995 22 R -1.3 0.7
Billy Wagner 14 1995 23 L 1.4 1.4
Armando Benitez 11 1994 21 R -0.4 0.4
Bill Pulsipher 12 1994 20 L 1.9 0.0
Alan Benes 14 1994 22 R -0.6 -1.2
Antonio Osuna 15 1994 21 R 0.0 1.3
James Baldwin 8 1993 21 R 0.0 -1.0
Steve Karsay 12 1993 21 R 0.9 0.0
Todd Van Poppel 7 1992 20 R 0.2 -0.7
Jason Bere 8 1992 21 R 2.2 2.8
Allen Watson 9 1992 21 L -0.5 -0.9
Kurt Miller 11 1992 19 R 0.0 -0.7
Pedro Martinez 10 1991 19 R 0.2 3.2
Mark Wohlers 13 1991 21 R 0.6 -0.3
Arthur Rhodes 6 1990 20 L -1.7 1.7
Willie Banks 15 1990 21 R -0.5 -0.7
Darryl Kile 11 1989 20 R 0.0 -0.7
Willie Banks 13 1989 20 R 0.0 -0.5
Mike Harkey 14 1989 22 R 4.4 0.0

As you can see, there's a lot of variation present. Single-season rWAR totals ranging as high as Roy Oswalt's 6.2 and as low as Roy Halladay's -3.2 - no, that is not a typo, perhaps the greatest pitcher in baseball had perhaps the worst single-season pitching line of the Expansion Era - highlight the lottery-ticket nature of playing the pitching prospect game. With such underwhelming mean and median totals over the entire sample, though, it's clear that we need to look into the data deeper, as teams have shown a distinct - and completely reasonable - tendency to value prospects of this caliber as if they were more valuable than this.

However, this is a very raw sample. What I mean by this is that we haven't even identified any context within this 20-year period, not to mention begun to consider it in how the sample should be used. Thus, let's now dig a little deeper to see if we can't come up with some more concrete conclusions.


First, let's look to see if prospects from smaller time-defined samples of this 21-year sample have provided different values over the two seasons following their ranking. Here's what happens if you split the data into two chunks: prospects from 1989-1999, and prospects from 2000-2009:

Yr_Prior Data Yr_1 rWAR Yr_2 rWAR
2000-2009 Mean 1.8 2.2
Median 1.8 2.4
1989-1999 Mean 0.3 0.5
Median 0.0 0.0

Here's what happens when you break up the period of 1989-2009 as into the following four groups: 1989-1994, 1995-1999, 2000-2004, 2005-2009.

Yr_Prior Data Yr_1 rWAR Yr_2 rWAR
2005-2009 Mean 1.7 2.3
Median 1.9 2.3
2000-2004 Mean 1.9 2.2
Median 1.6 2.4
1995-1999 Mean 0.3 0.9
Median 0.0 0.6
1989-1994 Mean 0.4 0.2
Median 0.0 -0.3

As you can see, there is a remarkable shift in the short-term value of prospects in this sample that were ranked during the first half of BA's history to those ranked during the second half of BA's history (though breaking BA's prospect-ranking history into quarters has a minimal additional effect). Know that this isn't necessarily to say that the BA staff of the 90's was doing a poor job - I personally believe that the dramatic change in value is more a product of advancements in medical technology, instructional technology, and internet access to scouting reports and video than some dramatic shift in BA philosophy or analyst quality.

One piece of evidence I have for this belief is tracking the trends of the change in rWAR from year one to year two for each pitching prospect. The following graph has this difference graphed across all 51 careers, listed chronologically.


The most important thing to notice in this graph, as I see it, is the large number of below-zero points prior to the year 2000, and the relatively small number of below-zero points in the year 2000 and beyond. This strikes me as more of a difference in teams being able to keep their pitchers healthy than any fault of BA.

Nonetheless, this data does tell us that the overall sample means and medians from the master table above are of little value when looking at Bauer and Skaggs. Instead, we ought to rely more on the sample that ranges from 2000-2009, throwing the rest of the data out the window. One could gripe about adding back another year - the 2000 list was a solid one for BA, with solid inclusions Mark Mulder and Kip Wells within our initial sample - but I don't see it being particularly material to the study either way.

This leaves us with a greatly diminished sample of 22 names, including repeats of Neftali Feliz (twice), Matt Cain (twice), and Ryan Anderson (twice). As noted above, the mean rWAR of the sample is 1.8 in year one and 2.2 in year two, with a year one median of 1.8 rWAR and a year two median of 2.4 rWAR - all of which are solid figures to be put up by a first-year player. These seem like reasonable measures of averages to me, and pass the basic sniff test, validating this reduced field as a sample we can truly work with for the rest of the post. With this established, let's now look at coming up with some conclusions.


Those averages, if taken at face value, suggest that Trevor Bauer and Tyler Skaggs have the potential, right now, to be quality starting pitchers that the team can roll out every fifth day and receive value relatively similar to what Joe Saunders provided Arizona in 2011 - 2.4 rWAR. As such, we would have to wonder why a $6MM expenditure was made on Joe for what appears to be less than a win of gain even if Joe keeps up his 2012 form.

However, despite the averages making sense in our previous sample, what those averages completely ignore is perhaps the greatest question when it comes to pitching prospects: volatility. If we include a basic standard deviation calculation to our new sample, it looks a little something like this:

Yr_Prior Data Yr_1 rWAR Yr_2 rWAR
2000-2009 Mean 1.8 2.2
St Dev 1.5 1.9

Those are some enormous volatilities, and while I have no empirical data to back this up, I trust that Joe Saunders can provide much smaller deviations, particularly with a good defense behind him. If you assume - and this is going to be a big assumption - that these patterns would operate on normal distributions in the long-run, this means that about 32% of all posted rWAR totals in the first year following the ranking would either be below 0.3 or above 3.3, and about 32% of all posted rWAR totals in the second year following the ranking would either be below 0.3 or above 4.1.

For a team that has aspirations of playoff contention, I'm not sure if I would be comfortable relying on the downside of that distribution, and that's before factoring in some of the skewing effects (which admittedly make the initial normal distribution assumption all the sketchier) of the data - for instance, the fact that a player who would likely contribute significant negative value over a full season would get demoted to the minors before he had the opportunity to do so, thus preventing negative value from entering the picture.

Further, there's some heavy selection bias present in that the players who were good enough to receive big-league time did, whereas those who regressed may have been kept in the minors (i.e. recorded a 0.0 rWAR year) and would have performed less-admirably in the majors had they been forced into action. All in all, relying on prospects is a risky proposition, even those considered to be among the best major-league-ready arms in all of baseball. As such, for a team like Arizona who has hopes to repeat as NL West Champs, side-stepping that heavy standard deviation in favor of some relative stability - innings stability, at the very least - seems prudent.


While we've analyzed the merits of Bauer and Skaggs relative to Arizona's desires for playoff contention, we have yet to compare them to each other. We can obviously compare them on the basis of their respective rankings, with Bauer slotted four spots ahead of Skaggs, but can we juxtapose them in any other objective - i.e. not through scouting reports - way? The first thing to come to my mind was their handedness. Keeping in mind that Bauer is a right-hander and Skaggs a lefty, is there any evidence that suggests one group is more valuable relative to the other? Let's take a look.

Hand (Count) Data Rank Yr_1 rWAR Yr_2 rWAR
Left (10) Mean 9.3 1.6 1.7
Median 7.5 1.8 1.2
St Dev 1.6 1.8
Right (12) Mean 10.2 1.9 2.7
Median 10 1.8 3.0
St Dev 1.5 1.9

This was quite surprising to me. The samples are far from conclusive, but this data does at least suggest that left-handed pitchers in this range of BA's rankings tend to be less valuable than their right-handed counterparts, particularly after year one. With the scarcity of left-handed pitching in affiliated pro baseball and prevalence of left-handed power hitters, it wouldn't be surprising to me whatsoever if teams overestimated the benefits of being left-handed and subsequently slightly overrated left-handed pitching prospects.

Would I ever base a personnel decision on this data? No, and I cannot emphasize that enough. With a total sample of 22 pitchers, this is, at most, a cute and interesting short-term trend to observe. Particularly considering the fact that the left-handed pitchers of this sample were rated higher, I simply didn't expect to see such a significant gap in year two between the left-handers and right-handers. However, I would want to see this trend carried out over significantly larger samples of prospects before considering it to have enormous meaning. For now, I consider it to be a neat hypothesis, and perhaps something we can turn back to five years from now when looking at how these two pitchers - and those joining them in this sample over the next few years - fare in the major leagues.

(I initially had also hoped to group the sample by age in the year prior to their ranking, but, after cutting down the data set, the samples would have been too small - groups of nine, five, five, and two - to make, in my opinion, any sound conclusions. I already felt like I was stretching things with samples of 12 and 10.)


We've come to the end of the post, so here's a quick recap of what we've uncovered:

- There was a dramatic shift in how highly-rated prospects subsequently performed sometime around 1999-2000, making prospect performance data from before that era incredibly suspect.

- Ignoring selection bias and skew in the data, which would pull these figures down, the expected values for prospects ranked in the vicinity of Trevor Bauer and Tyler Skaggs is a bit south of 1.8 rWAR, a drop from, for instance, Joe Saunders' 2.4 rWAR campaign in 2012.

- Ignoring selection bias and skew in the data, which would likely pull these figures down as well, the standard deviations for rWAR of pitching prospects, even those ranked as highly as Bauer and Skaggs, is significant, and just a few runs off of the total expected value of these pitchers.

- Over the last ten years, highly-rated upper-level right-handed pitching prospects have performed better in the two years following their ranking than their left-handed counterparts. Whether there is any statistical significance to this trend is another question, given the samples used.

What does it all mean? Well, for me, it means all the more reason to be excited about the acquisition of Trevor Cahill, the return of Joe Saunders, and the unexpected emergence of Josh Collmenter. Much as we love to dream on the immense talents of highly-rated young arms, for an Arizona team that is hoping to make its mark on the NL West in 2012, relying on volatile young pitchers to carry the load is a dangerous proposition, even with how talented Bauer and Skaggs are. Simply put, if either Bauer and Skaggs are a team's fourth or fifth rotation options going into the season, that team is skating, intentionally or otherwise, on thin ice; if Bauer and Skaggs are a team's sixth and seventh rotation options going into the season, that team is stacked in the rotation.

Stacked in the rotation is a very good thing to be.