IT DRILLDOWN
 
NEWSLETTERS
 

CIO.com updates, insights and advice on technology, management and your career.

 Advice and Opinion

 CIO Consumer IT

 CIO Leader

 CIO Enterprise

 CIO Insider

 

RSS Feeds »

 
 
SUBSCRIBE TO CIO
 

Are you involved in setting the direction for your company's IT budget or strategy?

Apply today for a FREE subscription to CIO Magazine!

» Subscription Services

» Reprints

 

 


Tue, Nov 20, 2007 20:21 EST

Why the Waterfall "Failed"?

Topic: Applications

Current Rating: 4 Comments: 10

Thoughts on applicability of different development methodologies

May be it’s not a waterfall that failed, but our understanding of it as Dr. Kruchten pointed out (see http://www.ibm.com/developerworks/rational/library/4626.html). However, waterfall clearly has its limitations. I tend to think that we tried to use it beyond its boundaries of applicability and now blame the process while we really need to look in the mirror. Let’s consider the topic a bit closer.

There are a great many tasks that are based on rigid and stable algorithms. It is possible – and natural – to come up with “correct” upfront requirements and design for these tasks. This means that it is possible to apply the serialized software development process that we used to call “waterfall”. The last statement has at least two corollaries. Firstly, linear / serialized development methods are not dead. And what’s more, they should be used for the said subset of tasks. Let’s call them deterministic tasks. Secondly, if we cannot “do it right” from the first attempt for the “right” type of tasks, this says something about our abilities to do our job and nothing else.

Most business applications belong to a different subset of tasks, however. Although each program is based on an algorithm anyway, these business tasks cannot be reduced to stable, once-and-forever described algorithms. We can call these tasks partially-deterministic.

I have been involved in developing business applications since the mid-70s. Never ever during my entire career has a client (a user, customer, stakeholder, etc.) known what he wants. Even less likely was that could he distinguish between “wants” and “needs”, let alone to envision what he will need tomorrow.

Why is this? Are they all stupid? Or may be they are just evil and like to torture us, poor software developers? I tend to say that the massive scale of this phenomenon calls for a quest for less extravagant reasons. We may even suggest that we are dealing with the nature of a beast. Here are two “less extravagant” reasons that I like the most. The first one is more or less obvious: they (clients) deal with a very volatile reality, which changes on them every day. Indeed, market conditions, human behaviours, acts of competitors and governments, social processes cannot be reduced to stable algorithms.

I think the second reason is not so obvious. Here it is: not only does the volatile reality influence the requirements and hence our systems, but other way around is true too. Like in O. Henry’s famous novel, the systems we develop influence the reality! This is a very important point that we do not always remember. The extent of our influence on reality is yet to be calculated. Future scientists may come up with an indicator similar to Planck’s constant to define a minimal system change beyond which we influence the reality that we are trying to automate.

So, certain tasks call for a certain process. If the above two thoughts are correct, then it’s a nail in the coffin of the linear process’ applicability to non-algorithmic tasks. Similarly, however, using an iterative process for a highly algorithmic and well-defined task may not be the best approach either.

You do not have flash or javascript support.
Average (3 votes)
4
 
 
Wed, Nov 21, 2007 10:03 EST
Anonymous user
Posted by: AJ Burke
Rating:

I like the terms used here of 'deterministic' and 'partially deterministic' and concur that not only do we see the 'variable reality' as perceived by the end-user of computerized systems we develop, but that those systems can actually alter that reality...

 
Tue, Nov 27, 2007 19:08 EST
Posted by: mtruxaw
Rating:

I agree with you that theoretically there may be some "highly algorithmic and well defined task" where a H2O-Fall may work as well as an agile approach. I don't know if I have ever actually participated in such a project, but I suppose it is possible. Nevertheless, I can't see the downside to using an agile, iterative approach even on these types of projects. What is the down side for delivering the highest priority portions of that system earliest? What is the downside to getting customer eyes on the product earlier to make sure your "highly algorithmic and well-defined task" was not misinterpreted by the coders. Give me a good agile iterative approach any day. Small deliverables early and often, inspection and adaptation, etc

 
Wed, Nov 28, 2007 11:34 EST
Anonymous user
Posted by: Anonymous
Rating:

Very good treatise that on System influences reality. I have been ranting same here on system altering the user behavior. I wish this concept gets mainstream and gets tackled better.

Now back to Waterfall model, I do not believe there is any problem in trying to determine as much as you can before embarking on design and development. The problem is trying to implement everything in one go. This takes away our ability to learn from mistakes and apply corrective actions.

 
Thu, Nov 29, 2007 10:01 EST
Anonymous user
Posted by: Kupe
Rating:

I am in agreement. You have to first understand the problem or business need you are trying to address. Once that is determined it makes perfect sense to implement the solution in an agile or iterative matter.

 
Mon, Dec 3, 2007 18:46 EST
Posted by: Eugene Nizker
Rating:

“…I can't see the downside to using an agile…”, one of my readers said. This is one of the reasons why I wrote the article in the first place.

As we know, Agile is a mindset. It cannot be reduced to a number of techniques. However, we often hear: “Use X, and it’ll solve all of your problems”. No, it won’t! “You cannot be agile unless you do Y and Z”. Yes, you can! When an approach that is based on common sense is reduced to a set of religious beliefs, it loses common sense. And then everyone loses – us and our clients.

When we get excited by something – a new book, a new friend, or a new software development methodology – it’s not easy to step back and look at our new “acquisition” with a critical eye. Yet, everything has its limitations, its boundaries. Remember, Newton’s physics didn’t die with Einstein’s breakthrough discovery. But it’s boundaries of applicability were redefined. We still happily use it every day without thinking about
relativistic effects.

What are the boundaries of applicability for a development process, be it waterfall or Agile? “Give me a good agile iterative approach any day” may be an expensive proposition. Every release requires regression testing. Auto-tests do help, but in their turn require maintenance, ever more with every new test. Add to it preparation and conduct of a promotion (often risky), potential disruption of operations and user re-training. All in all, I can see how Agile approach can become more expensive than Waterfall for straightforward algorithmic tasks with stable requirements.

As I said in my article, I have not seen this kind of projects yet during my 30+ years in software development. But that doesn’t mean they do not exist. After all, I only worked in the enterprise applications field.

It would be a stretch to paint me as an adept of Waterfall. As a member of Vancouver Agile council, and a leader of a team that practiced Agile for a few years with impressive results, I am sold on Agile. One can see my previous posts in this blog: “Agile Software Development - A Cinderella Story?”, “How One CIO was Converted to Agile”, “A Client as a Teammate” and others. However, I see Agile becoming more and more of a religion, and this is a very unfortunate development.

Regards,

Eugene Nizker

PS. A friend of mine who taught me how to write meaningful code some 30 years ago, Alex Zakharyonok, commented on another thought in my article: the systems we develop influence the reality. Alex says: “For business applications, I would formulate it more aggressively: we should only develop systems that change the reality; otherwise they aren’t worth our time”. I can’t agree more.

Start a Conversation

Click to post

Got something to say? We want to hear it! Click the Post button to get started. GO»

EXPERT ADVICE

See our roster of experts.

Advice & Opinion from more than 77 of IT's most insightful thinkers.

advertisement

TOP USERS
UserPoints
1. laith al jazi12550
2. Akshay Upadhye7650
3. Chris Moore6750
4. abdhiraj6175
5. remi5350
UserPoints
6. asengupta3750
7. tmmackay3500
8. Sureshram3125
9. Mark Cummuta3100
10. Michael Kavis2950
  PARTNERS       PODCASTS       WEBCASTS    
 

Enterprise Content Management: From Strategy to Solution

Enterprise content management (ECM) has become an important competence and infrastructural technology, particularly for large and medium-sized organizations. Hear about industry trends for ECM and why standardizing your ECM platform is so critical to your success during this roundtable discussion.

Sponsored by IBM  View This Webcast »

 

Get Rich or Get Thin: The Secure Client Webcast

Malcolm Harkins, General Manager of Intel's Information Risk and Security group, looks at the increasing sophistication of attacks. He also talks about he pros and cons of thin and rich clients in fending off those attacks.

Sponsored by Intel
  View This Webcast »

 

The Greening of the Data Center

This report outlines a four-part strategy companies can employ to achieve an energy-efficient infrastructure.

Sponsored by Sepaton  Read this White Paper »

More Partners »

Resource Alerts

Get instant email notifications by topic when white papers, webcasts, and case studies are added to our library.

Podcast: Accenture's View on Software as a Service

Publisher Emeritus at CIO magazine, Gary Beach, talks to Accenture Chief Architect, Paul Daugherty, about where software is going, and in particular about the emerging concept of software as a service and its contribution to high performance.   Read More »

 

Business Service Management: Delivering Value Throughout the Service Lifecycle

Business and IT alignment is as much a priority among CIOs, as it is a challenge...  Read More »

 

Talking Innovation with a Hall of Famer

Bill Walsh, lauded for his innovative thinking on the football field, talks techniques for applying innovation to every job.  Read More »

More Podcasts »

Resource Alerts

Get instant email notification when white papers, webcasts, and case studies are added to our library. Don't just be up-to-date—be up to the minute with our new Resource Alerts.

Crossing The Data Divide: The Case For Complex Data Exchange Technology

Hear the results of a new study conducted by IDG examining how to maximize the value of your enterprise information assets by automating the data exchange process. Join now and learn how technology can help you Cross The Data Divide.   Read More »

 

Collaborative Resiliency: A Unified Approach to Risk Management

Climate change, oil price shock, pandemic and acts of terrorism are true risks that cannot be ignored...  Read More »

 

The Great Escape: Contact Center Applications Are Moving into the Enterprise

With resulting benefits such as tighter integration with back-office applications, administrative control, and improved communications, contact center applications are moving into the enterprise. In this webcast...  Read More »

More Webcasts »

Resource Alerts

Get instant email notification when white papers, webcasts, and case studies are added to our library. Don't just be up-to-date—be up to the minute with our new Resource Alerts.

 
NEWSLETTER

Sign-up for the Advice & Opinion Newsletter

 
FEATURED SPONSORS
 
 
 
SPONSORED LINKS
 

White Paper: Learn how to succeed with BPM

Webcast: Research insight into how organizations are using virtualization

ITIL V3 and the IT Service Catalog

The New Growth Paragidm: Multi-Enterprise SOA

3M saved $3M on printing. Learn how HP can help your business

Enterprise Service Bus: A Definition

Let's Get Virtual: A Look at Today's Server Virtualization Architectures

Increase conversions on your site with the help of EV SSL.

Get Control of Mobile Data (and More)

Data Loss Prevention Starts at the Endpoint

Building a Foundation for Pragmatic Service Management White Paper

Performance Brief: Mobile Application Acceleration

Strategies for centralizing data backup

Citrix XenServer FREE trial

Nothing Short of Revolutionary: Four Steps toward More Effective Enterprise Finance

The Best IT Strategy for a Company with Global Operations

Speed, agility, flexibility - The HP BladeSystem c-Class

The Business Value of Symantec Data Center Foundation Solutions

Webcast: Why standardizing your ECM platform is so critical to your success

The PCI Data Security Standard

White Paper: Assess Your SOA

The Universal Wireless Client: Simplify mobility and reduce the cost of supporting mobile workers

Tuning ERP and the Supply Chain for Profitable Growth

Compliance by the numbers- addressing requirements with online document management and collaboration technology

Webcast: Learn how to Simplify and Standardize Architecture

Global Crossing is the most viable alternative for voice, video and data.

The New Foundation of Storage: Xiotech's Intelligent Storage Element

BPM Done Right: 15 Ways to Succeed Where Others have Failed

3 Reasons to Invest in Integration Technology Now

A CIO's View of Server Virtualization

Survival of the Fittest: Disaster Recovery Design for the Data Center

Windows Server 2008: To Upgrade or Not to Upgrade?

How Office 2007 Exposed Bill Gates

How to simplify mobility and reduce the cost of supporting mobile workers

Helping IT Become a Service Provider White Paper

Extending PCI Compliance to the Mobile Workforce

A proven approach to WAN optimization

Wireless Vulnerability Management: What It Means for Your Enterprise

Green IT: Reducing Your Carbon Footprint with Citrix

Wide-area data services enable todays global enterprise

Discover PMI's credentials and career path tools

Symantec State of the Data Center Report

Getting the Most from your Data Protection Solution

Tripwire PCI DSS Solutions: Automated, Continuous Compliance

ITCi White Paper: Challenges and Opportunities of PCI

TDWI Research report clears confusion about automating data governance

White Paper: Unlocking the Potential of B2B

See why 93 of the Fortune Global 100 depend on Blue Coat.

Taking Document Automation to the Next Level

Video Series: IT Leaders discuss how IT is becoming part of the innovation cycle.