Show Me The Numbers

This is a topic I rant on at work, and find myself having to repeat everywhere I go:

Show Me The Numbers

Tell me what are your goals, what metrics by which you measure those goals, and how you go about gathering that data. I don't care what your gut is telling you, I want repeated observations of actual code running on real world systems with production load. I don't care you want it faster, I want a number which is "fast enough". Is 4 seconds fast? Is 0.4 seconds fast? Is 15 minutes fast enough? What is the per-unit cost per transaction at each point in your stack? Are we fast enough but spending too much to get there? Are we too slow because we are spending all our time optimizing the wrong solution? If we deliver this how will you measure the business impact? If you can't measure the business impact should we even bother doing this?

Part of the reason so many engineers get frustrated with the suits in their organization is because they give them a free pass. They have a mistaken notion that if they do their jobs the business will magically see the benefits:

Tech Work + Magic Business Pixiedust -> Profit

As if the things that the business side of this fictional fence does are immune to the laws of nature. Everything a business does involves the spending of or acquisition of a numerical quantity. There are people whose sole job is to cook those numbers into books which provide a full accounting of the business's business. They have to legally keep track of all those numbers and face criminal charges should they violate the sanctity of the business contract. You can bet your ass every pointyhaired boss is more concerned about those numbers than they are anything you are working on. Your work itself is a commodity after all, they buy it for pennies on the dollar, and resell it for far more than you'll ever see. They can only do that because they have numbers, goals, and a solid metric by which to judge success.

This is also the same basic concept behind all software and systems engineering. If you are engineering a solution, you MUST measure and revise accordingly. If you aren't measuring the impact of each design change, and contemplating the cost associated with each design decision, you aren't engineering a solution. You may be writing tons of use cases, acceptance tests, unit tests, integration tests, and all kinds of adhoc manual tests, but if you aren't measuring and tracking the impact of this activity, you aren't engineering your processes either. Metrics allow you to see patterns of behavior and judge relative impact on a per-constraint basis. You can make design decisions involving trade-offs when you can account for all of the associated costs. Developer time, market opportunity, systems footprint, operational overhead, support costs, etc.

But if you don't have numbers, you're just faking it, and no amount of work will make up for that failure.