Apply today for a FREE subscription to CIO Magazine!
Tue, Nov 20, 2007 20:21 EST

|
Posted by: Eugene Nizker in Best Practices Topic: Applications
Current Rating: |
Thoughts on applicability of different development methodologies
May be it’s not a waterfall that failed, but our understanding of it as Dr. Kruchten pointed out (see http://www.ibm.com/developerworks/rational/library/4626.html). However, waterfall clearly has its limitations. I tend to think that we tried to use it beyond its boundaries of applicability and now blame the process while we really need to look in the mirror. Let’s consider the topic a bit closer.
There are a great many tasks that are based on rigid and stable algorithms. It is possible – and natural – to come up with “correct” upfront requirements and design for these tasks. This means that it is possible to apply the serialized software development process that we used to call “waterfall”. The last statement has at least two corollaries. Firstly, linear / serialized development methods are not dead. And what’s more, they should be used for the said subset of tasks. Let’s call them deterministic tasks. Secondly, if we cannot “do it right” from the first attempt for the “right” type of tasks, this says something about our abilities to do our job and nothing else.
Most business applications belong to a different subset of tasks, however. Although each program is based on an algorithm anyway, these business tasks cannot be reduced to stable, once-and-forever described algorithms. We can call these tasks partially-deterministic.
I have been involved in developing business applications since the mid-70s. Never ever during my entire career has a client (a user, customer, stakeholder, etc.) known what he wants. Even less likely was that could he distinguish between “wants” and “needs”, let alone to envision what he will need tomorrow.
Why is this? Are they all stupid? Or may be they are just evil and like to torture us, poor software developers? I tend to say that the massive scale of this phenomenon calls for a quest for less extravagant reasons. We may even suggest that we are dealing with the nature of a beast. Here are two “less extravagant” reasons that I like the most. The first one is more or less obvious: they (clients) deal with a very volatile reality, which changes on them every day. Indeed, market conditions, human behaviours, acts of competitors and governments, social processes cannot be reduced to stable algorithms.
I think the second reason is not so obvious. Here it is: not only does the volatile reality influence the requirements and hence our systems, but other way around is true too. Like in O. Henry’s famous novel, the systems we develop influence the reality! This is a very important point that we do not always remember. The extent of our influence on reality is yet to be calculated. Future scientists may come up with an indicator similar to Planck’s constant to define a minimal system change beyond which we influence the reality that we are trying to automate.
So, certain tasks call for a certain process. If the above two thoughts are correct, then it’s a nail in the coffin of the linear process’ applicability to non-algorithmic tasks. Similarly, however, using an iterative process for a highly algorithmic and well-defined task may not be the best approach either.
I like the terms used here of 'deterministic' and 'partially deterministic' and concur that not only do we see the 'variable reality' as perceived by the end-user of computerized systems we develop, but that those systems can actually alter that reality...
I agree with you that theoretically there may be some "highly algorithmic and well defined task" where a H2O-Fall may work as well as an agile approach. I don't know if I have ever actually participated in such a project, but I suppose it is possible. Nevertheless, I can't see the downside to using an agile, iterative approach even on these types of projects. What is the down side for delivering the highest priority portions of that system earliest? What is the downside to getting customer eyes on the product earlier to make sure your "highly algorithmic and well-defined task" was not misinterpreted by the coders. Give me a good agile iterative approach any day. Small deliverables early and often, inspection and adaptation, etc
Very good treatise that on System influences reality. I have been ranting same here on system altering the user behavior. I wish this concept gets mainstream and gets tackled better.
Now back to Waterfall model, I do not believe there is any problem in trying to determine as much as you can before embarking on design and development. The problem is trying to implement everything in one go. This takes away our ability to learn from mistakes and apply corrective actions.
I am in agreement. You have to first understand the problem or business need you are trying to address. Once that is determined it makes perfect sense to implement the solution in an agile or iterative matter.
“…I can't see the downside to using an agile…”, one of my readers said. This is one of the reasons why I wrote the article in the first place.
As we know, Agile is a mindset. It cannot be reduced to a number of techniques. However, we often hear: “Use X, and it’ll solve all of your problems”. No, it won’t! “You cannot be agile unless you do Y and Z”. Yes, you can! When an approach that is based on common sense is reduced to a set of religious beliefs, it loses common sense. And then everyone loses – us and our clients.
When we get excited by something – a new book, a new friend, or a new software development methodology – it’s not easy to step back and look at our new “acquisition” with a critical eye. Yet, everything has its limitations, its boundaries. Remember, Newton’s physics didn’t die with Einstein’s breakthrough discovery. But it’s boundaries of applicability were redefined. We still happily use it every day without thinking about
relativistic effects.
What are the boundaries of applicability for a development process, be it waterfall or Agile? “Give me a good agile iterative approach any day” may be an expensive proposition. Every release requires regression testing. Auto-tests do help, but in their turn require maintenance, ever more with every new test. Add to it preparation and conduct of a promotion (often risky), potential disruption of operations and user re-training. All in all, I can see how Agile approach can become more expensive than Waterfall for straightforward algorithmic tasks with stable requirements.
As I said in my article, I have not seen this kind of projects yet during my 30+ years in software development. But that doesn’t mean they do not exist. After all, I only worked in the enterprise applications field.
It would be a stretch to paint me as an adept of Waterfall. As a member of Vancouver Agile council, and a leader of a team that practiced Agile for a few years with impressive results, I am sold on Agile. One can see my previous posts in this blog: “Agile Software Development - A Cinderella Story?”, “How One CIO was Converted to Agile”, “A Client as a Teammate” and others. However, I see Agile becoming more and more of a religion, and this is a very unfortunate development.
Regards,
Eugene Nizker
PS. A friend of mine who taught me how to write meaningful code some 30 years ago, Alex Zakharyonok, commented on another thought in my article: the systems we develop influence the reality. Alex says: “For business applications, I would formulate it more aggressively: we should only develop systems that change the reality; otherwise they aren’t worth our time”. I can’t agree more.