The Stats We Use: DRS And UZR

Last night, this tweet appeared on my timeline:

Jedd Gyorko does indeed rank second in baseball in Defensive Runs Saved, at +13, behind only Nolan Arenado, and five whole runs ahead of Evan Longoria in third place. Last year, in a full season, only three third baseman (Arenado, Adrian Beltre, and Kyle Seager) contributed more positive value by DRS than 2017 Gyorko has racked up in half the playing time, and Beltre and Seager nipped him by just two runs a piece.

Either Gyorko, once a so-so second baseman in San Diego, has transformed himself into one of the best third baseman in the league in St. Louis, or something fishy is going on here. Let’s take a step back.

Today’s two most frequently cited fielding stats, DRS and UZR, both use batted-ball data and fielding zones to determine how well defenders are performing. Where offensive stats are somewhat concrete—a home run is a home run and a double is a double, for the most part—fielding stats are essentially estimates based on various assumptions, like how hard a ball was hit and where a fielder was initially standing. In an era where the exact data on how hard a ball was hit and where a fielder was initially standing exists, somewhere, thanks to Statcast, today’s advanced fielding metrics feel a tad archaic.

That’s not a criticism of those stats, necessarily. They’re the best we have, and they’re carefully crafted by intelligent people and plenty of hard work, and they’re probably pretty good. I use them all the time (hence the title) as sort of a decent estimate of how how a player is doing, and over large samples (we’re talking multiple seasons) they’re generally viewed as fairly accurate. However, given the nature of how they’re constructed, we have to tread carefully with them, especially in small samples.

Consider an average struck ground ball sent directly at the area where a third baseman usually stands. This batted-ball is turned into an out, on average, 93 percent of the time, let’s say. Gyorko is standing right there, easily makes the play, and is given a small amount of credit for it. Manny Machado, on the other hand, is positioned 10 feet to Gyorko’s left, ranges to get to the ball and fires off balance to first, missing the runner by a half-step. He’s dinged big time for not converting the out even though he clearly made a better play than Gyorko.

You can imagine the opposite scenario, too, where Machado gets a bunch of extra credit for fielding a ball hit right at him only because he’s positioned well outside the “normal” range. There are numerous similar scenarios that could be used here, and while they should even out over time, maybe they don’t. Some teams shift and position players far differently than other teams, and while positioning is a part of good defense, it’s not easy to figure out how we want to incorporate that into how we evaluate individual players.

In an era of extreme shifting and less notable shading, these are the kind of issues that the creators of these stats are dealing with, and in many ways there’s nothing they can do about them without more information. For that reason we have to regress fielding stats toward average significantly more than we would with hitting or pitching stats, since, again, we never really know how well a player is truly fielding (there are other reasons, too, but let’s not get stuck in the rabbit hole here).

Going back to Gyorko, and the interesting thing about his fielding numbers this year: he’s +13 runs by DRS and just +0.7 runs by UZR. Two stats that use the same data source (Baseball Info Solutions) and are constructed in generally the same way, and one of them views Gyorko as an all-world defender while the other barely has him above average. Just for kicks, I grabbed all qualified third baseman from this season and looked at the difference between each player’s DRS and UZR.

Here are the players with the biggest difference between the two figures:

Player DRS UZR Diff
Jedd Gyorko +13 +0.7 12.3
Nolan Arenado +16 +6.5 9.5
Chase Headley -7 -0.8 6.2
Maikel Franco -8 -2.2 5.8
Miguel Sano -7 -1.4 5.6
Yunel Escobar -6 -0.5 5.5
Evan Longoria +8 +2.8 5.2

That’s in only half a season, remember.

UZR sees Gyorko and Chase Headley (hello, old friends) as equal defensively, but DRS has a 20-run gap between them; that’s like two wins in saber parlance, or a boatload of money on the free agent market in economic parlance. Of the 21 qualified third baseman, one-third of them had a difference of at least five runs. If you project that out over a full season, you’re talking about a range of 10 to 25 runs here. Without getting too deep into potential methodological differences I’m not fully up to date on, there’s clearly something going on here. This would be like if one offensive metric rated a guy as average while another rated him as one of the league’s best hitters.

For what it’s worth, it’s appears that UZR is more tightly clustered. The gap between the best third baseman and the worst so far this year is just under 15 runs; with DRS, that number is 24 runs. It’s possible that there’s a difference with how positioning/shifts are being handled, or that it’s something to do with how regression is applied (or both).

The best guess is that Gyorko is probably somewhere in between right now, and probably closer to average going forward. Beyond Gyorko, though, it’s important to remember the inherent limitations of modern fielding stats. For as much progress as we’ve made over something like fielding percentage, a myriad of potential pitfalls and data limitations have stunted our understanding of how to measure fielding somewhere between “pretty well” and “shrug face.” More accurately, perhaps, is that we* know how to measure it, we just don’t have all the tools to properly do so. None of this is groundbreaking, of course, for anyone who’s followed these matters over the years.

*And by we I mean the smart people who do this stuff.

The rule of thumb is to use multiple years—three, preferably—of fielding data to get an idea of where a player’s at. The problem with that is that by the time you have enough information on the player, his talent level has probably already changed. You’re left in an endless game of rough estimation in a field that’s built, in some ways, on decimal point accuracy.

Continue to use them—we still will, of course—but do so with the knowledge that there are limitations aplenty, ones that probably won’t be fully solved for years.

***

For those interested in similar topics, check out Chris’s “Getting Dirty With Stats” series from a few years back. There are, of course, plenty of interesting, in-depth articles on various sabermetrics issues at places like Baseball Prospectus, FanGraphs, and The Hardball Times. I may turn this into something of an occasional series, riffing on some of the stats I use (or don’t use) in my articles without necessarily providing a complete and thorough overview. 

You are encouraged to comment using an exisitng Twitter, Facebook, or Google account. Upvote comments you find helpful, and only downvote comments that do not belong. The downvote is not a 'disagree' button.