Extremely thought provoking piece thought provoking piece in this month’s IEEE, by Tom DeMarco::
“You can’t control what you can’t measure.” … Implicit in the quote … is that control is an important aspect, maybe the most important, of any software project. But it isn’t. Many projects have proceeded without much control but managed to produce wonderful products such as GoogleEarth or Wikipedia.
To understand control’s real role, you need to distinguish between two drastically different kinds of projects:
Project A will eventually cost about a million dollars and produce value of around $1.1 million.
Project B will eventually cost about a million dollars and produce value of more than $50 million.
What’s immediately apparent is that control is really important for Project A but almost not at all important for Project B. This leads us to the odd conclusion that strict control is something that matters a lot on relatively useless projects and much less on useful projects. It suggests that the more you focus on control, the more likely you’re working on a project that’s striving to deliver something of relatively minor value. (emphasis mine)
As a staunch proponent of metrics for software projects, this passage was quite upsetting. I am committed to measurement. I take things apart, I tune them, and I make them better. The author suggests that my obsession with performance monitoring, continuous integration, code coverage, bug counts, and product backlogs is not only an expensive indulgence, but a warning sign that the projects themselves are of little value.
This stopped me in my tracks. I thought about it for a few days, and I’ve decided that for a startup I fundamentally disagree.
In an enormous company, he’s got a valid point. A million dollar project which eeks out a meager 10% return is a waste of resources and no amount of control is going to turn a 10% return into a 10x return.
But things are vastly different for entrepreneurs. We own our little companies, with shallow pockets. Everyday we work to mitigate risk. We are trying to effectively manage short runways. And there is a constant battle for time, our most precious resource.
If you work for a Fortune 500 company, your runway is essentially endless. In this environment, I’d agree that you must peel away layers of process and structure to allow the planes to get off the ground more quickly.
Startups are different. Startups are already implicitly pursuing the $50 million opportunity, but their resources are fiercely constrained. There’s never enough time, never enough money, and never enough patience from your investors. An entrepreneur’s time is spent hustling to avoid crashing into the field at the end of a potentially ever shorter runway. To extend DeMarco’s argument into this environment is to advocate flying blind, with no instruments.
If you’ve only got eight months of cash in the bank, you better have a darn good idea of what features you can build in that amount of time. And it will be far easier to raise money and increase the length of your runway once you can quantify the elusive path to profitability to your investors.
4 Comments

4 responses so far ↓
During my entrepreneurial stint I wanted to focus automated measurements on the business goals (ie: new user signups, user actions per week, dollars per user, etc).
It was the things that I couldn't control that I wanted to measure the most. Without those types of measurements you can't see where you're going or whether what you're doing has business value or not.
–Robert
(p.s. – fix your openid thing to work with me.yahoo.com … openid 2.0 ftw, please)
Robert,
What kind of things did you want to measure that you couldn’t? Using tools such as Google Analytics and Google Website Optimizer, you can directly link website behavior to your business goals (such as new account signups, purchases, etc).
Robert, good point on the OpenID / IntenseDebate issue. it's a known issue with the IntenseDebate plugin, and hopefully they'll fix it soon (although it's been 6 months ago since it was logged as "working on it")
Robert: what was it that you wanted to measure?
I've been pleasantly surprised by what I can accomplish with metrics using Google Analytics and Google Website Analyzer. These tools allow me to define my own goals (economic or user activity) and analyze how site activity influences these goals. Things have come a long way since you were an entrepreneur!