The Other Fifteen

Just so everyone's clear on this, I think forecasts for relievers really stretch the limits of the ability of forecasting systems, and so I've chosen not to include them in this exercise. I might come back later and do a reliever forecast chart if it seems to be something people want, but personally I don't see a lot of value in it.

Same color code as before: green is good, red is bad, yellow is right about average. In this case, I used dirt simple ERA as my "one-stat" metric. Why ERA? Well, because RA wasn't available for some forecasting systems. It's generally a minor enough difference between the two that I won't lose too much sleep over it. Most of the other problems with ERA are that it's a poor predictor of future performance. Well, we're using forecasting systems, so that's not relevant here.

As for Ryan Dempster - everybody but ZiPS only projected him as a reliever, so I cheated and multiplied that ERA by 1.25. If Dempster is better suited to moving back to the rotation than the average pitcher, well, then these numbers do not do him justice.

First, sorted by average ERA:

Then, sorted by difference between "best" and "worst" forecast:

There is probably a better way of measuring agreement than what I'm using here, but I'm still teaching myself a lot about Excel and what it's capable of doing as an analytical (and graphical) tool. To be honest, the liberal arts/communications major part of me is more interested in playing around with Excel as a presentation tool than as an analytical tool, so I get distracted.

So... what do these numbers mean? Well, they mean less than the hitting forecasts, that's for sure. Forecasting pitchers is much more difficult than forecasting hitters - and that's not an exact science. So feel free to take these with a grain of salt.

And it's important to understand exactly what these projections are, and what they're useful for. PECOTA does not dislike you; ZiPS could give a crap less what your favorite team. I have a real hard time figuring out whether or not this piece is tongue-in-cheek or not, but it serves as an example of the attitude I'm talking about:

So we were completely blindsided with the release of their 2008 PECOTA projection for Ted Lilly. We're not to fond of lawsuits (especially with our history) so we aren't going to distribute their subscription data via TLFC... but if we did, and we're not, it might look something like this: 10 -10.

Excuse us? We understand that all this math is based on complex algorithms and represents only a weighted mean projection, but where's the human fail-safe to ensure a bogus projection like this doesn't get published. This is the most egregious over confidence in computing since the W.O.P.R and we all know how that worked out... Matthew Broderick's career.

It goes on for a bit in that vein. [Certain people might enjoy the snark about Magic: The Gathering, also published by Wizards of the Coast, purveyor of Dungeons and Dragons. In case anyone was wondering - I stopped playing Magic regularly some time after the Ice Age expansion set. I specialized in a sweet burn deck, and also had a nice zombie-based deck. I did a demo play of the Tenth Edition set and it was still fun, but I don't really have the time and money to sink into it.]

Now, are there times I disagree with a forecast? Sure. Do I take it personally? No. These are just mathematical models; especially with pitching forecasts, they more often start a conversation rather than end it. So feel free to debate (the capstone of my series on projections is going to tackle exactly how to argue with them and win - with some caveats). Just don't take it personally. The computer certainly doesn't.

Labels: Baseball, Chicago Cubs, Projections