I have been thinking about batch sizes lately and will sum up my thoughts in this post.
In traditional waterfall methodology all functionality is delivered in one big bang batch. The shortcomings of this method is well known and includes complex and expensive change management, an extensive integration phase, poor customer involvement, etc.
In iterative methods, such as spiral models, the functionality is delivered in several batches. The projects using RUP that I have experienced often have quite large iterations fixed both in time and in scope, which everyone except managers knows is impossible. The result is that the iteration is prolonged, which in practice means fixed scope, variable time. Due to the large process overhead in RUP, projects tend to choose fairly large iterations, which means the problems from the waterfall method is still in play.
Agile methods like Scrum are also iterative, but Scrum firmly suggests to have short time boxed iterations, which in practice means variable scope, fixed time. Because of the short iterations the planning horizon is also short and predictable, not making it a big problem with variable scope. I think the agile methods are a natural step to finally get away from the drawbacks of traditional development. They fulfill a need.
According to Lean principles the ideal batch size is 1 (feature), which gives us a pipeline. If we manage the pipeline and have batches (features) of similar size we get a very efficient system low in waste. This makes timeboxing not applicable, since you deliver one feature at a time (fixed scope). Actually timeboxing is waste because you work up a stock of non-delivered value, which is waste. Optimally you should deliver as soon as it is ready. This is actually confirmed by the fact that many teams are moving from the original 4 week sprints to 2 week sprints.
The agile methods of today, with shorter iterations and smaller batches, is a natural and necessary step to get away from the heavy traditional development methods, but I’m convinced that future agile processes will fully implement a pipline with a minimum batch size. If no one does I will do it myself. I do have some ideas for implementing the pipeline with kanban buffers, but that’s a different story (and maybe a future post).