I got recently my second thesis work completed, during which I managed to read quite a lot management literature from the 80’s and even the magnificent Henry Ford’s ”My life and work” from 1922 describing most of the Total Quality Management principles almost 90 years ago. Yet we are still not using the practices that produced one of the greatest industrial enterprises to the date.
Another fundamental and nicely written book is Eliyahu Goldratt’s The Goal describing the management process of Theory of Constraints. According to Goldratt, the mission of a company is to maximize the throughput (sales to the customers), while simultaneously minimizing the operational expenses and the inventory. Every system has a single bottleneck that constraints the throughput. By locating the bottleneck one can improve the throughput, but improving any other area will not improve the performance at all. This principle is the same as the Extreme Programming’s Leave Optimization till last. Otherwise you end up in tuning parts of your code that will not improve the performance at all, but make your code harder to understand.
After reading these and a few more books the agile software engineering principles suddenly appeared much clearer and theoretically more worthwhile. I began to understand the fundamental principles behind the methods like Scrum, XP and Lean, and learned a few new tricks that haven’t been used much before in SW engineering. Unfortunately Kent Beck hasn’t bothered to reference to the prior work and many of the Agile and Lean principles have been challenged and questioned, while their origins have remained unrecognized. For example the principle of quick iterations can be derived directly to the Goldratt’s production formula: the faster you can release the less Work-In-Progress or inventory you have. The less inventory not currently producing revenues, the lower are your capital costs in holding the inventory and the higher profit you have. Additionally you get also the feedback faster, which is a crucial meta-capability in the era of tightening competition between the Lean organizations (according to Robin Cooper: When Lean Enterprises Collide). Actually, when you think about it, the optimum iteration length or batch size is one or release cycle time close to zero, or in other words you could even use iterationless-process or a Drum-Buffer-Rope/JIT -pull-system as described by Goldratt.
A related building block of a JIT -production system are the Kanban -signal cards, that are closely analogous to the XP User Stories and the Scrum index cards. The idea is to pull the production from the customer/demand side by signaling the previous phases of the permit to produce new items, instead of producing documentation/code that might not be needed by anyone. The idealized model is the Just-in-Time -system, where a customer gets what he orders when he needs it, but nothing is produced in advance on stock. The magic in the system is the process capability, or the variance and the tolerance you need to have spare stock so that the sudden demand spikes won’t cause bottlenecks (or the increase in the cycle time lowering the critical throughput). According to the Deming’s Statistical Process Control -school, a process capability means how low variance the process has, or how low emergency stock you can keep without causing a bottleneck. A SW Engineering example of this is for example how many XP User Stories you have to produce before the beginning of an iteration, or can you also produce them Just-in-Time. At least on my XP team this reveals directly one bottleneck: we do have a queue of roughly half-foot of User Stories and the engineering team seems to be much slower to implement the stories than the customer is able to generate them. Also Goldratt notices that the easiest way to identify a bottleneck is by looking for the queue sizes, the longest queue is usually the bottleneck, especially if the queue seems to be always increasing.
From the process measurement point of view XP seems to draw the principles almost directly from the text book when introducing the Project Velocity -measurement – the only required measurement for a software team is its throughput, or how many user stories or story points they can release per iteration. Unfortunately Beck doesn’t explain very well why this measurement is the most important one, but you get it now: the most severe bottleneck in SW engineering is the coding, not the design – so you should first focus on getting your tput up. There is so much to do in improving the velocity that I don’t know yet where the next bottleck will be. I think it will take quite and much work before anyone can say that they have more designers than coders and are able to locate the next major software engineering bottleneck :).
When you know the TQM -fundamentals you will also know the remedies for the performance and quality problems. The mission of the TQM -school (as first stated by Henry Ford) is to eliminate all waste. Somebody asked from me the what is the difference between the German quality and the Japanese quality? The difference is in the principle of eliminating the waste. When a car comes out from a German car factory, it goes first to a repair shop, and then to the customer. A Japanese car can be shipped directly to the customer. The difference is in how much scrap and rework you produce. As anyone in the software industry can easily recognize, the complete rewrites and bug fixes are the activities that most of the developers spend most of their days on producing also most of the cost. The Six Sigma -school goes even further and describes the Hidden Factories meaning all scraping and rework activities that are performed without appearing on the management reports. A good example is a bug that is detected and fixed immediately without entering it on the bug management system. This happens every day. One could think that a hidden rework factory is also a coding typo that requires use of backspace. According to the Japanese concept of Poka-Yoke, or error-proofing, the remedy is to use tools and practices that make it impossible to produce an error. For example you can use Eclipse that automatically displays the syntax errors, highlights the code, suggests for the possible (and correct) methods available, and autofills the brackets so that you can’t produce the elementary coding mistakes. Software engineering is full of social practices such as code reviews and pair programming that tries to further improve the quality by detecting the errors faster. However, practicing the Poka-Yoke (or the original Baka-Yoke = idiot proofing) will get you farther more quickly when you dismiss the possibility of producing errors in the first place! Yet more examples of the same principles include the Japanse concept of Jidoka– or a self-correcting machine that halts it’s operations and alerts a maintainer, when an unrecoverable error happens, and the Designed for Six Sigma, where product is designed by choosing components that are known to have low number of errors instead of coding all code from the scratch by self.
After introducing yourself on the key Lean/TQM concepts you should hopefully pretty quickly start to recognize from which more fundamental principle all the practices originate from, and in which situations they should and should not be used. I hope that with these few advices you can start to recognize where your bottlenecks currently are located in, and also get ideas how to improve your throughput and quality :). I’ll be also writing more on the topic 🙂