How to Bake an Agile Cake?

Imagine you want to make a cake – You find a recipe for a delicious chocolate cake.

The recipe describes a process, which is a linear system to predict and therefore control the output.

A process or a linear system requires the environment (like an oven) to be deterministic (read: non-changeable), for the process to remain predictable.

If your oven warms 10°C more, than the oven the recipe was made with, then your cake will get burned unless you compensate for the 10°C.

The environment in IT is constantly changing with large amounts of simultaneous changes, and therefore the environment can’t be deterministic –  No wonder processes are hard to maintain.

The complexity grows

There are almost an infinite ways a cake can be done, but only a few are viable, and even fewer deliver a delicious one. Even with only five ingredients, where each has an option of “being added” or “not being added” will give 2^5 = 32 possible cake-outcomes, with only a few being eatable.

Now imagine each ingredient having five options, like water being 15°C, 25°C, 35°C, 45°C, or 55°C hot, which will impact how yeast will react with it. It will give 5^5 = 3.125 possible cake-outcomes:

Add five different gastronomy methods, with each having five options: i.e. making the cake rise for a ½ hour, 1 hour, 2 hours, 4 hours or 8 hours and it will give 10^5 = 100.000 possible cake-outcomes.

It is impossible to theorize all the scenarios, and some assumptions might be wrong or even deadly. So how do we deal with this?

Stepwise Refinement

Edsger Wybe Dijkstra suggested “Stepwise Refinement” for dealing with this complexity in programming: A program is split into multiple steps, where each step has as few decisions as possible.

But how do you decide the temperature of the water, if you don’t even know what kind of yeast you will use or the timeframe for the dough to rise? The whole thing is so interconnected that you can’t even decide on the type of yeast or a timeframe, without deciding on water temperature.

Now imagine writing a requirement specification, where 200 requirements can interact with each other. It is the reason, why writing specifications are so complicated and the reason why the waterfall model fails.

Agile development

Agile development is about prototyping multiple designs of a feature. Each prototype needs to be tested, to find the best version and build further upon it. Easy to read, but complex to apply, as shown on the Agile Landscape V3:

Source: https://www.slideshare.net/ChrisWebb6/agile-placemat-v9, by Chris Webb

Stepwise Improvement

Michael E. Caspersen developed “Stepwise Improvement” to enhance Stepwise Refinement by making it agile without being too complex and give the possibility to visualize the development.

It should not replace the different agile methods but provides a reflection on agile development.

Stepwise Improvement has 3 type of steps:

Refinement: Implementation does not meet specification -> Improve implementation.

Extend: Specification does not comply with requirements (wishes) -> Improve specification.

Restructure: Implementation is too complex -> Simplify by reimplementation.

An example of a Stepwise Improvement development:

All these steps can be visualized in a 3D graph

1. Start with a basic specification that contains the most basic and vital requirements.
2. (Refinement) Create a basic cake based on the basic specification.
3. (Extend)Test the cake to see what is missing and improve specification, like giving the cake some lemon taste.
4. (Refinement) Add ingredient 4 (lemon juice) to comply the specification.
5. (Extend) Test the cake and discover the cake is too sour. Improve the specification to make the cake less sour.
6. (Refinement) Add ingredient 3 (sugar) to lessen the sour taste.
7. (Extend) Test the cake and discover it tastes great, but it could be better with some chocolate.
8. (Restructure) Remake the cake with minimum ingredient 4 (lemon juice) and minimum ingredient 3 (sugar), but make it still taste as great.
9. (Refinement) Add ingredient 5 (chocolate pieces) to comply the specification.

Repeat extend, refinement and restructure until cake is as desired and the recipe completely developed.
So even if you are lucky enough to get the best cake, but you will not know it*. It will only require a few cake-remakes, to get a great cake and recipe. But to know if you got the best possible outcome, would require you to test all possible cake-outcomes.

* You would need a quantum computer or that someone solves the problem in computer science of p=np, but that is highly unlikely in the near future.