Agile Reading Glasses: Process Control
To develop a product or deliver a service you can use three different ways to control the flow of work. These are: defined process control, statistical process control and empirical process control.
Defined process control is better suited for situations in which we know already how to do things because we already did them at least one time. We are in an environment with relatively low volatility that can be easily predicted.
Statistical Process control allows to improve forecasts based on emergent recurring pattern out of data. It is suited for moderate variability and uncertainty. It requires some effort to analyze the data and recognize recurring patterns that can help stabilizing the flow, making statistical analysis more meaningful
When uncertainty is higher, we need to use empirical control because the only way we have to know if we are done, is to define what done means, and check the produced artifacts against this definition of done. This approach is quite expensive as it requires to have full visibility on the whole system (often called transparency) and it requires rapid adaptations to improve the output over time (inspect and adapt).
Video produced by Benjamin Felis - Graphic Recording & Explanation Movies
When we don’t know how to solve a problem because we never did it before, we have to start empirical. Knowing this is the most expensive way to manage risk and control the flow, we will try to recognize repeatable patterns over time, that would allow us to start using a more convenient Statistical Control. If after a while we come to the level of awareness for which the recurring pattern become a repeatable process, then we can move towards defined control and possibly automate the process.
The goal is to strive to continuously improve and learn to move from a fully empirically controlled process to a more statistically controlled one, or even to some automated parts within a defined controlled process. This is what most agile team do by automating tests cases, deployment and builds.
In order to achieve a cost effective and risk effective control approach, you need to start from empirical and move towards defined, with small incremental steps, to avoid losing control.