What I’m going to relate here is a true story. I have two kids, aged seven and four, that are veritable fountains of ideas.
For some time now they have been earning a small allowance based on keeping their rooms clean, helping in the kitchen, leaving out the Friday candy, and so forth. Sometimes the “wage negotiations” can be a little tiring, but overall it appears to work out quite nicely. Greedy little worms as they are, they are always making plans about how to spend their well-earned money… even before they’ve earned it!
I’ve tried to introduce the backlog concept (one thing at a time, in order of preference) but found that they can really only think about one thing at a time, and that thing needs to be BIG… preferably a large Lego set with a three-digit price tag. They get a lot of enjoyment out of scanning the newest Lego catalogues, anticipating the next possible purchase and changing their plans on a daily if not hourly basis. I’m somewhat nervously looking forward to the day when they’ve actually scraped together enough money and want to go to the toy store: when we walk out of there, they can’t change their minds anymore.
One morning at the breakfast table the older boy sat silent for a minute, then stated that at the current rate of saving money, they would have enough money to buy one 100 € Lego set each in December. He had figured this out by himself, and I was impressed and proud. So I decided to push the boundaries a bit and conduct a breakfast table experiment.
What if, I put to the kids, instead of saving separately and buying two sets in December, they would put their money together and buy one set already in July? Since their Legos are stored together in one box (or to be precise, two big boxes and a small box), they could build and play together. And they would still be able to buy the other set on schedule in December, perhaps one of the new sets coming later this year? I’d have to drive to the toy store twice — extra fun for the kids, and no big deal for me.
Let’s take a time-out here and rephrase the breakfast table discussion in terms of agile and lean. My son used the established velocity to forecast a date of delivery for a package consisting of two components. I counter-proposed that by splitting the batch in two, the first component could be delivered early, without any delay in the delivery of the second component, and with only a slight increase in overhead. In effect, rather than getting the full delivery on schedule they would get the first part after only half the time, and the other part still on schedule. Yay! The choice should be easy! What’s there not to like?
This thinking is not limited to theoretical discussions about toys for children; it’s also very much applicable to normal business life. Many companies however want to maximize the batch size: common wisdom (which is wrong, as we will see in a moment) says that large batches bring lower overhead, which means larger savings and lower costs.
For example, Naive Inc., a supplier of bespoke software has found that delivering software to the customer is painful and has a large overhead. There is a lot of scheduling and negotiating to be done to determine the contents of the delivery package. Then someone must collect, package and deliver the components to the offshore system testing team. If it passes, the package can be delivered to the customer who needs help deploying the software, and when they finally get it up and running, the bastards start reporting bugs and change requests that feed the next scheduling and negotiating cycle. And so, logically, the managers decide to avoid the pain and make as few releases as possible. They decide to collect all the software features into one big batch and deliver that near the end of the project.
This is a prime example of one-sided legacy thinking. Firstly, it ignores the new rules and axioms brought by the information society. Secondly, it accounts only for cost and ignores value and risk. In a fast-moving world, fresh feedback is extremely valuable. Digital information can be handled in new ways that are not immediately obvious:
- Transaction costs are virtually zero. If the components are identical in structure, the benefits of scale are tremendous. Computers are extremely good at remembering large amounts of data and repeating the same operations extremely rapidly.
- Complexity increases exponentially with size. However, if the components are inherently different, a large batch is exponentially more complex.
- Digital data is mutable. You can easily mix exact information with estimates, stubs and placeholders, then observe and refine the system over time by replacing components. In comparison, a house needs to be built from the bottom up.
- Digital data attracts change. If change is necessary to a system with both software and hardware components, people tend to concentrate the changes on the software side.
- Software is not an asset, it’s a liability. Software is subject to relentless technical change, and will rot away unless it is continuously maintained. (I once had to rewrite an application because, after two years and two toolchain upgrades, we could no longer make it compile.)
What this all means is that the holding costs of software are immense, while the transaction costs can be extremely cheap. In this scenario, big batches are not economical. While collecting components for the batch you pay high holding costs, and because the transaction costs are so low anyway the economic benefit from large batches is very small. Lose-lose! Instead you should reap the benefit from low transaction costs and deliver small batches often, avoiding the holding costs in one fell swoop. Win-win!
For example, Smart Inc., another supplier of bespoke software has found that delivering software to the customer is painful and has a large overhead. Rather than avoid the pain, the managers decide to remove the root causes and invest in automating the build-and-delivery process. They start with a continuous integration system that opportunistically gets the source code and builds the product. Next, they implement a test automation system that exercises the product from the ground up to the user interface, running unit and component tests as well as scripted acceptance tests. Both systems give instant feedback to the developers.
The managers can now on a daily or weekly basis select a package for sending to the customer. This is automated using a deployment script that in minutes installs the new version and sends a brief summary of the changes to the customer. The customer feedback (change requests and bugs together) is added to the list of things to do. Because the product is known to be stable and increments are small, negotiating what should be done next becomes quick and painless.
(Both Smart Inc. and Naive Inc. are real companies, but names have been changed to protect the… uh… innocent.)
Comparing Naive Inc. to Smart Inc., it’s clear that the latter has a very short turnaround time. The developers can change from one task to another in a matter of days, and projects can be ended or canceled in a matter of weeks without losing any of the invested money. This allows Smart Inc. to react to risks and make use of opportunities. Further, Smart Inc. and their customers are very aware of the current quality level and state of the products — they know pretty well what they have right now and controlling the project is comparatively simple. Customer communications are frequent and constructive, focusing on what can be implemented next (value).
In comparison Naive Inc. has a long turnaround time, on the order of months if not years. The company does not measure what they have implemented frequently enough, and must resort to indirect metrics (bugs found, or hours spent). Controlling a project without direct information is fiendishly difficult: the risk of failure is very high, and the risk of losing invested money is also high. Customer communications focus on bugs and missing features i.e. what has not been done (cost).
Despite this, many software companies routinely choose large batches over small. There is some kind of built-in blindness that causes people to avoid looking at batch size as a project variable that can be changed.
When it comes to running a business, people are well aware of batch size and inventory costs and the importance of e.g. not carrying large risks or committing to long-term investments in a rapidly changing world. Thinking like this is not weird or unique — executives do it all the time. People accept that the business world is unpredictable, and that you need to maintain your options, hedge your bets, buy information or get insurance.
Since projects operate within the same business environment as the company itself, they are subject to the very same uncertainties, and in addition often encumbered with inherent technical complexities. The process of developing an innovative product is in fact a balancing act between trial-and-error and progress. In hindsight the product may be blindingly obvious, but never in foresight — you cannot predict what will happen, and you must let the product emerge in collaboration between engineers and end users. It’s called Research and Development for a reason.
Yet in most cases projects are not allowed to react to risks except perhaps within very narrow bounds, and not allowed to exploit opportunities at all. Pure madness. People draw up multi-year project plans fully aware that they will be hopelessly outdated after six months. People make designs and plans at the beginning of the project, at the point in time when they know the least about the product than they will ever know, then go forward blindly in the hope that reality will somehow reform itself around their plans.
So how did our breakfast discussion end? Having thought about my proposal for a moment, the kids declared that getting just one package would be unfair, and that they’ll go with the original plan and buy two sets in December. Duh.
Funnily, I can see this thinking reflected in too many companies. “We are implementing an ERP system with ten different user roles, and it would be unfair if one role got served first.” Or: “Our product must be able to capture and edit material, it would be useless without both functions.”
Business is not about fairness. It’s not about unfairness either, but there is truth in the old saying that “all is fair in love and war”. Business is about grabbing opportunities and avoiding risks. If this means being “unfair” to some potential customer or user segment, so be it. You can’t cater to everyone all the time.
Still, I hope to give my kids a proper understanding of how the digital world works. Because, as it turns out, the established institutions are not doing that yet. I owe them that.
(photo by Alan Chia, source Wikimedia Commons)