Estimation SWAG

Last summer I learned a new acronym, courtesy of Chinmay. He asked for a developer to put some estimates on the outstanding technical tasks needed to make a demo production ready. So I fabricated a few numbers, slapped them on the wiki page, and dropped a note in the project chat room saying that I'd put up some grossly inaccurate estimates. Chinmay replied by thanking me for the SWAG - Sophisticated, Wild-Ass Guess.

The reason I like this acronym so much is that it pretty much sums up all the software estimates I've ever seen. I mean, it's not like there's no methodology to it - you look at the task, break it down into pieces, and apply some heuristic to attach a number of hours to each piece. But really, at the end of the day, it's all just guess work. The "heuristic" usually just comes down to a gut feeling of how long something ought to take based on your knowledge of the system. In other words, a guess.

And it's not like there aren't more organized ways of doing this. We could use COCOMO models to come up with our estimates, for example. The problem is, approaches like that require you to have actual data to use as a basis for comparison, and most teams don't have that. Or they do have it, but the data is useless - corrupt, fabricated, poorly defined, etc.

The reason it's hard to get good data on developer productivity is obvious to anyone who has ever worked in a pathological development shop - because it can be dangerous. In a good organization, such data would be used to estimate costs and schedule and to help make better decisions about what projects to do and where to allocate resources. In a pathological organization...not so much.

In a pathological shop, measurement is used against you. If the measurement is number of hours needed to complete a task, then you'll be judged to be incompetent or slacking off if a "simple" task takes you "too long" to complete - where what counts as "simple" and "too long" are determined by a non-technical manager who really has no idea what you actually do. Or perhaps you'll be required to account for all eight hours of your day, down to five-minute increments. Because heaven forbid you should "cheat" the company by taking a bathroom break.

When I think of pathological examples of measurement, I always think about my last job with an online advertising startup. At one staff meeting, our VP of Engineering announced that he new expectation was that engineering would deliver on its estimates with 90% accuracy based on 80% definition from the product manager. So we were required to be 90% sure how long a task was going to take, even though we were only 80% sure exactly what it was we were building. And based on previous experience with the product manager, calling the specs he gave us "80% definition" was really generous - I'd say it was more like 50%.

In that case, the productivity measurement was being used as a tool to force the VP of Engineering out of the company (and it worked - he put in his notice the next week), but the principle applies to anyone - if a measurement can be used against you - even if that use is unfair and out of context - assume it will be used against you. That's a very sad reaction to have, but it's natural if your experience is primarily with unhealthy shops. And while I don't endorese such a paranoid attitude, having been in such a shop more than once in the past I can certainly understand it.

One of these day, I'd really like to work in a shop where performace metrics were used as a real tool in project planning. Even in healthy shops, this sort of thing seems to be uncommon. That seems to be something of a comment on the state of "software engineering" as a discipline. The differentiator between quality and pathological shops seems to be the absence of bad practices as opposed to the precense of good ones. Even in good shops, building software seems to be less engineering and more art or black magic.

Personally, I'm not convinced that that is a necessary or desireable state of affairs. I'd just like some first-hand evidence to support that suspicion.

You can reply to this entry by leaving a comment below. This entry accepts Pingbacks from other blogs. You can follow comments on this entry by subscribing to the RSS feed.

Related entries

Add your comments #

A comment body is required. No HTML code allowed. URLs starting with http:// or ftp:// will be automatically converted to hyperlinks.