NEWSLETTERS
 

CIO.com updates, insights and advice on technology, management and your career.

 
 
 
SUBSCRIBE TO CIO
 
Are you involved in setting the direction for your company's IT budget or strategy?

Apply today for a FREE subscription to CIO Magazine!

 


Tue, Nov 20, 2007 20:21 EST

Why the Waterfall "Failed"?

Topic: Applications

Current Rating: 4 Comments: 10

Thoughts on applicability of different development methodologies

May be it’s not a waterfall that failed, but our understanding of it as Dr. Kruchten pointed out (see http://www.ibm.com/developerworks/rational/library/4626.html). However, waterfall clearly has its limitations. I tend to think that we tried to use it beyond its boundaries of applicability and now blame the process while we really need to look in the mirror. Let’s consider the topic a bit closer.

There are a great many tasks that are based on rigid and stable algorithms. It is possible – and natural – to come up with “correct” upfront requirements and design for these tasks. This means that it is possible to apply the serialized software development process that we used to call “waterfall”. The last statement has at least two corollaries. Firstly, linear / serialized development methods are not dead. And what’s more, they should be used for the said subset of tasks. Let’s call them deterministic tasks. Secondly, if we cannot “do it right” from the first attempt for the “right” type of tasks, this says something about our abilities to do our job and nothing else.

Most business applications belong to a different subset of tasks, however. Although each program is based on an algorithm anyway, these business tasks cannot be reduced to stable, once-and-forever described algorithms. We can call these tasks partially-deterministic.

I have been involved in developing business applications since the mid-70s. Never ever during my entire career has a client (a user, customer, stakeholder, etc.) known what he wants. Even less likely was that could he distinguish between “wants” and “needs”, let alone to envision what he will need tomorrow.

Why is this? Are they all stupid? Or may be they are just evil and like to torture us, poor software developers? I tend to say that the massive scale of this phenomenon calls for a quest for less extravagant reasons. We may even suggest that we are dealing with the nature of a beast. Here are two “less extravagant” reasons that I like the most. The first one is more or less obvious: they (clients) deal with a very volatile reality, which changes on them every day. Indeed, market conditions, human behaviours, acts of competitors and governments, social processes cannot be reduced to stable algorithms.

I think the second reason is not so obvious. Here it is: not only does the volatile reality influence the requirements and hence our systems, but other way around is true too. Like in O. Henry’s famous novel, the systems we develop influence the reality! This is a very important point that we do not always remember. The extent of our influence on reality is yet to be calculated. Future scientists may come up with an indicator similar to Planck’s constant to define a minimal system change beyond which we influence the reality that we are trying to automate.

So, certain tasks call for a certain process. If the above two thoughts are correct, then it’s a nail in the coffin of the linear process’ applicability to non-algorithmic tasks. Similarly, however, using an iterative process for a highly algorithmic and well-defined task may not be the best approach either.

You do not have flash or javascript support.
Average (3 votes)
4
 
 
Wed, Nov 21, 2007 10:03 EST
Anonymous user
Posted by: AJ Burke
Rating: 70

I like the terms used here of 'deterministic' and 'partially deterministic' and concur that not only do we see the 'variable reality' as perceived by the end-user of computerized systems we develop, but that those systems can actually alter that reality...

 
Tue, Nov 27, 2007 19:08 EST
Posted by: mtruxaw
Rating: 90

I agree with you that theoretically there may be some "highly algorithmic and well defined task" where a H2O-Fall may work as well as an agile approach. I don't know if I have ever actually participated in such a project, but I suppose it is possible. Nevertheless, I can't see the downside to using an agile, iterative approach even on these types of projects. What is the down side for delivering the highest priority portions of that system earliest? What is the downside to getting customer eyes on the product earlier to make sure your "highly algorithmic and well-defined task" was not misinterpreted by the coders. Give me a good agile iterative approach any day. Small deliverables early and often, inspection and adaptation, etc

 
Wed, Nov 28, 2007 11:34 EST
Anonymous user
Posted by: Anonymous
Rating: 30

Very good treatise that on System influences reality. I have been ranting same here on system altering the user behavior. I wish this concept gets mainstream and gets tackled better.

Now back to Waterfall model, I do not believe there is any problem in trying to determine as much as you can before embarking on design and development. The problem is trying to implement everything in one go. This takes away our ability to learn from mistakes and apply corrective actions.

 
Thu, Nov 29, 2007 10:01 EST
Anonymous user
Posted by: Kupe
Rating:

I am in agreement. You have to first understand the problem or business need you are trying to address. Once that is determined it makes perfect sense to implement the solution in an agile or iterative matter.

 
Mon, Dec 3, 2007 18:46 EST
Posted by: Eugene Nizker
Rating:

“…I can't see the downside to using an agile…”, one of my readers said. This is one of the reasons why I wrote the article in the first place.

As we know, Agile is a mindset. It cannot be reduced to a number of techniques. However, we often hear: “Use X, and it’ll solve all of your problems”. No, it won’t! “You cannot be agile unless you do Y and Z”. Yes, you can! When an approach that is based on common sense is reduced to a set of religious beliefs, it loses common sense. And then everyone loses – us and our clients.

When we get excited by something – a new book, a new friend, or a new software development methodology – it’s not easy to step back and look at our new “acquisition” with a critical eye. Yet, everything has its limitations, its boundaries. Remember, Newton’s physics didn’t die with Einstein’s breakthrough discovery. But it’s boundaries of applicability were redefined. We still happily use it every day without thinking about
relativistic effects.

What are the boundaries of applicability for a development process, be it waterfall or Agile? “Give me a good agile iterative approach any day” may be an expensive proposition. Every release requires regression testing. Auto-tests do help, but in their turn require maintenance, ever more with every new test. Add to it preparation and conduct of a promotion (often risky), potential disruption of operations and user re-training. All in all, I can see how Agile approach can become more expensive than Waterfall for straightforward algorithmic tasks with stable requirements.

As I said in my article, I have not seen this kind of projects yet during my 30+ years in software development. But that doesn’t mean they do not exist. After all, I only worked in the enterprise applications field.

It would be a stretch to paint me as an adept of Waterfall. As a member of Vancouver Agile council, and a leader of a team that practiced Agile for a few years with impressive results, I am sold on Agile. One can see my previous posts in this blog: “Agile Software Development - A Cinderella Story?”, “How One CIO was Converted to Agile”, “A Client as a Teammate” and others. However, I see Agile becoming more and more of a religion, and this is a very unfortunate development.

Regards,

Eugene Nizker

PS. A friend of mine who taught me how to write meaningful code some 30 years ago, Alex Zakharyonok, commented on another thought in my article: the systems we develop influence the reality. Alex says: “For business applications, I would formulate it more aggressively: we should only develop systems that change the reality; otherwise they aren’t worth our time”. I can’t agree more.

Start a Conversation
Click to post

Got something to say? We want to hear it! Click the Post button to get started. GO»

EXPERT ADVICE
See our roster of experts.

Advice & Opinion from more than 108 of IT's most insightful thinkers.

  PARTNERS       WEBCASTS    
 

Preparing for the Next Cyber Attack

Ensure you are up-to-speed on the latest security technologies available to keep your network safe in this Executive Guide. Get a thorough assessment of the corporate security threat landscape. Protect your network with data leakage protection, NAC and other technologies explained in this report.

Sponsored by Qwest  Read this Executive Guide »

 

Cloud Building: 8 Ingredients for Internal Clouds

Cloud computing: a fundamentally new way to deploy IT services and functions cost-effectively and quickly. Learn how the VMware vCloud initiative dramatically improves how consumers access their information and experience applications as well as the 8 ingredients to get you going.

Sponsored by VMWare  Read this White Paper »

 

Investing in Business Analytics Technology

You're thinking now is the time to take the plunge into business analytics, but you still have some unanswered questions. This research summary addresses the most common questions and concerns surrounding the successful launch of a business analytics initiative. It also includes real-world examples of organizations already getting return on their investment.

Sponsored by SAS  Read this White Paper »

Resource Alerts

Get instant email notifications by topic when white papers, webcasts, and case studies are added to our library.

Resource Alerts

Get instant email notification when white papers, webcasts, and case studies are added to our library. Don't just be up-to-date—be up to the minute with our new Resource Alerts.

Improving Transparency and Accuracy in IT Cross Charging

During this Webcast you'll learn how KBC Group implemented SAP BusinessObjects Profitability and Cost Management and realized many benefits.   View Now »

 

Cost Savings and Risk Reduction with Effective Systems Management

Join us and see how Novell can help you respond to today's economic challenges by increasing productivity, reducing costs and aligning IT initiatives with overall business goals.  View Now »

 

Capitalize on Your SAP Content

Learn ways to improve your content management by viewing these Open Text webinars today.  View Now »

Resource Alerts

Get instant email notification when white papers, webcasts, and case studies are added to our library. Don't just be up-to-date—be up to the minute with our new Resource Alerts.

 
NEWSLETTER

Sign-up for the Blogs & Discussion Newsletter

 
FEATURED SPONSORS
 
 
 
SPONSORED LINKS
 

Introducing the new HP ProLiant G6 server family

Accenture: Outsourcing for Competitive Advantage. More...

Better spam protection with Postini for just $1/user/mo

Introducing the new HP ProLiant G6 server family

infoBOOM! - The Mid-Sized Company CIO's Exclusive Community

Accenture IT Consulting: Logical meets technological. More . . .

The Fraudster Economy Model: Operating a Business in the Underground

Trade in your old laser printer and get up to $1000 back!

Taking the Service Desk to the Next Level

Revolutionizing Enterprise Application Deployment

Why Data Loss is Increasing--and What You Can Do About It

Data Loss Prevention: A Better Way to Approach Security

Learn how to managing client systems in the enterprise.

Build a High-Performance Open Web Platform

Mid-Sized Company CIO Community: infoBOOM!

Enterprise PBX Comparison Guide

Getting Value from Outdated Networking Equipment

Losing Ground: 2009 TMT Global Security Survey

Stop Application Fraud at the Source with Device Reputation

Learn about the VMware vSphere (TM) & Intel (R) Xeon (R) Processor 5500 Series

Learn how a virtualized enterprise can help your company reduce costs

Why Isn't Server Virtualization Saving Us More?

Learn how to save 30% through project & portfolio management.

How Open Source is Changing the Face of Enterprise Software

8 Key Ingredients to Building an Internal Cloud

Accenture IT Consulting: Enabling high performance. More...

Top Five CIO Challenges

Insight makes it easy to spend your Microsoft subsidy check.

Five minute business analytics assessment. Immediate results.

Dangerous Collaboration Practices: 5 Ways IT Can Minimize Risk

Accenture: Outsourcing for uncertain times. Click to learn more.

The Case for Investing in Business Analytics Technology. Read white paper.

Live Webinar: Applying Business Analytics. Click here to learn more

Seven Ways ITIL Can Help You in an Economic Downturn

Developing A Dynamic, Real-Time IT Infrastructure

Maximizing the Business Value of the PC Infrastructure

Communications and Collaboration Needs at Business Organizations

Using Open Source to Deploy Web Applications

Cloud Computing: Read about VMware's compelling vision & set of products

Enterprise PBX Buyer's Guide

Secondary Market Primer: Your Network at Half Price

How Interactive Viewer Reduces the Effort to Meet Visualization Requirements

Top-line Performance that's Bottom-line Efficient

White Paper: 8 Key Ingredients to Building an Internal Cloud

Read about virtualization and consolidation effort best practices

Building the Virtualized Enterprise with VMware Infrastructure

The Global Marketplace Today: Strategies for Tough Times

Top 10 Business and IT Drivers for the Wealth Management Sector

5 Steps to Automating Accounts Payable

Bottom-Line Benefits of Virtualization