NEWSLETTERS
 

CIO.com updates, insights and advice on technology, management and your career.

 CIO BlackBerry News and Tips
 CIO Research and Analysis
 CIO Microsoft
 CIO Insider
 
 
 
SUBSCRIBE TO CIO
 
Are you involved in setting the direction for your company's IT budget or strategy?

Apply today for a FREE subscription to CIO Magazine!

 


Tue, Nov 20, 2007 20:21 EST

Why the Waterfall "Failed"?

Topic: Applications

Current Rating: 4 Comments: 11

Thoughts on applicability of different development methodologies

May be it’s not a waterfall that failed, but our understanding of it as Dr. Kruchten pointed out (see http://www.ibm.com/developerworks/rational/library/4626.html). However, waterfall clearly has its limitations. I tend to think that we tried to use it beyond its boundaries of applicability and now blame the process while we really need to look in the mirror. Let’s consider the topic a bit closer.

There are a great many tasks that are based on rigid and stable algorithms. It is possible – and natural – to come up with “correct” upfront requirements and design for these tasks. This means that it is possible to apply the serialized software development process that we used to call “waterfall”. The last statement has at least two corollaries. Firstly, linear / serialized development methods are not dead. And what’s more, they should be used for the said subset of tasks. Let’s call them deterministic tasks. Secondly, if we cannot “do it right” from the first attempt for the “right” type of tasks, this says something about our abilities to do our job and nothing else.

Most business applications belong to a different subset of tasks, however. Although each program is based on an algorithm anyway, these business tasks cannot be reduced to stable, once-and-forever described algorithms. We can call these tasks partially-deterministic.

I have been involved in developing business applications since the mid-70s. Never ever during my entire career has a client (a user, customer, stakeholder, etc.) known what he wants. Even less likely was that could he distinguish between “wants” and “needs”, let alone to envision what he will need tomorrow.

Why is this? Are they all stupid? Or may be they are just evil and like to torture us, poor software developers? I tend to say that the massive scale of this phenomenon calls for a quest for less extravagant reasons. We may even suggest that we are dealing with the nature of a beast. Here are two “less extravagant” reasons that I like the most. The first one is more or less obvious: they (clients) deal with a very volatile reality, which changes on them every day. Indeed, market conditions, human behaviours, acts of competitors and governments, social processes cannot be reduced to stable algorithms.

I think the second reason is not so obvious. Here it is: not only does the volatile reality influence the requirements and hence our systems, but other way around is true too. Like in O. Henry’s famous novel, the systems we develop influence the reality! This is a very important point that we do not always remember. The extent of our influence on reality is yet to be calculated. Future scientists may come up with an indicator similar to Planck’s constant to define a minimal system change beyond which we influence the reality that we are trying to automate.

So, certain tasks call for a certain process. If the above two thoughts are correct, then it’s a nail in the coffin of the linear process’ applicability to non-algorithmic tasks. Similarly, however, using an iterative process for a highly algorithmic and well-defined task may not be the best approach either.

You do not have flash or javascript support.
Average (3 votes)
4
 
 
Wed, Nov 21, 2007 10:03 EST
Anonymous user
Posted by: AJ Burke
Rating: 70

I like the terms used here of 'deterministic' and 'partially deterministic' and concur that not only do we see the 'variable reality' as perceived by the end-user of computerized systems we develop, but that those systems can actually alter that reality...

 
Tue, Nov 27, 2007 19:08 EST
Posted by: mtruxaw
Rating: 90

I agree with you that theoretically there may be some "highly algorithmic and well defined task" where a H2O-Fall may work as well as an agile approach. I don't know if I have ever actually participated in such a project, but I suppose it is possible. Nevertheless, I can't see the downside to using an agile, iterative approach even on these types of projects. What is the down side for delivering the highest priority portions of that system earliest? What is the downside to getting customer eyes on the product earlier to make sure your "highly algorithmic and well-defined task" was not misinterpreted by the coders. Give me a good agile iterative approach any day. Small deliverables early and often, inspection and adaptation, etc

 
Wed, Nov 28, 2007 11:34 EST
Anonymous user
Posted by: Anonymous
Rating: 30

Very good treatise that on System influences reality. I have been ranting same here on system altering the user behavior. I wish this concept gets mainstream and gets tackled better.

Now back to Waterfall model, I do not believe there is any problem in trying to determine as much as you can before embarking on design and development. The problem is trying to implement everything in one go. This takes away our ability to learn from mistakes and apply corrective actions.

 
Thu, Nov 29, 2007 10:01 EST
Anonymous user
Posted by: Kupe
Rating:

I am in agreement. You have to first understand the problem or business need you are trying to address. Once that is determined it makes perfect sense to implement the solution in an agile or iterative matter.

 
Mon, Dec 3, 2007 18:46 EST
Posted by: Eugene Nizker
Rating:

“…I can't see the downside to using an agile…”, one of my readers said. This is one of the reasons why I wrote the article in the first place.

As we know, Agile is a mindset. It cannot be reduced to a number of techniques. However, we often hear: “Use X, and it’ll solve all of your problems”. No, it won’t! “You cannot be agile unless you do Y and Z”. Yes, you can! When an approach that is based on common sense is reduced to a set of religious beliefs, it loses common sense. And then everyone loses – us and our clients.

When we get excited by something – a new book, a new friend, or a new software development methodology – it’s not easy to step back and look at our new “acquisition” with a critical eye. Yet, everything has its limitations, its boundaries. Remember, Newton’s physics didn’t die with Einstein’s breakthrough discovery. But it’s boundaries of applicability were redefined. We still happily use it every day without thinking about
relativistic effects.

What are the boundaries of applicability for a development process, be it waterfall or Agile? “Give me a good agile iterative approach any day” may be an expensive proposition. Every release requires regression testing. Auto-tests do help, but in their turn require maintenance, ever more with every new test. Add to it preparation and conduct of a promotion (often risky), potential disruption of operations and user re-training. All in all, I can see how Agile approach can become more expensive than Waterfall for straightforward algorithmic tasks with stable requirements.

As I said in my article, I have not seen this kind of projects yet during my 30+ years in software development. But that doesn’t mean they do not exist. After all, I only worked in the enterprise applications field.

It would be a stretch to paint me as an adept of Waterfall. As a member of Vancouver Agile council, and a leader of a team that practiced Agile for a few years with impressive results, I am sold on Agile. One can see my previous posts in this blog: “Agile Software Development - A Cinderella Story?”, “How One CIO was Converted to Agile”, “A Client as a Teammate” and others. However, I see Agile becoming more and more of a religion, and this is a very unfortunate development.

Regards,

Eugene Nizker

PS. A friend of mine who taught me how to write meaningful code some 30 years ago, Alex Zakharyonok, commented on another thought in my article: the systems we develop influence the reality. Alex says: “For business applications, I would formulate it more aggressively: we should only develop systems that change the reality; otherwise they aren’t worth our time”. I can’t agree more.

Post new comment

* Subject:
* Username:
* E-mail:
The content of this field is kept private and will not be shown publicly.
Homepage:
* Body:
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <img> <blockquote> <strike> <p> <br>
  • Lines and paragraphs break automatically.
More information about formatting options

* Denotes required field.

Hot Conversations

Ex-Microsofties Look Back in Anger

Posted by Shane ONeill in News | 4 comments

The Price of IT Outsourcing

Posted by Beth Bacheldor in Best Practices | 2 comments

Start a Conversation
Click to post

Got something to say? We want to hear it! Click the Post button to get started. GO»

EXPERT ADVICE
See our roster of experts.

Advice & Opinion from more than 115 of IT's most insightful thinkers.

  PARTNERS       WEBCASTS    
 

Windows 7 Webcast Series

There's a lot of buzz about Windows 7 out there. Each month in our webcast series, listen to analysts and customers discuss how Windows 7 and the Windows Optimized Desktop is impacting large companies around the world. Learn how they evaluated Windows 7, including the cost of deployment, deployment strategies, and tangible benefits.

Sponsored by Microsoft  Listen to on-demand Recordings »

 

A Framework for Better Application Delivery

The complexity of application delivery is driven in part by the evolving applications environment. Instead of approaching application delivery from a siloed fashion, this handbook looks at end-to-end guidance and discusses the impact of ignoring the WAN, Web apps that are chatty, data center consolidation, SaaS, Web 2.0 and virtualization.

Sponsored by Riverbed  Read this White Paper »

 

Microsoft® Exchange 2010 includes archiving - but is it enough?

Microsoft® Exchange 2010 includes basic email archiving. But many organizations will find that it does not meet their requirements. This paper describes why organizations need to archive, what capabilities Exchange 2010 includes and why 3rd party archiving solutions will be necessary for most organizations.

Sponsored by Google, Inc.   Read this White Paper »

Resource Alerts

Get instant email notifications by topic when white papers, webcasts, and case studies are added to our library.

Resource Alerts

Get instant email notification when white papers, webcasts, and case studies are added to our library. Don't just be up-to-date—be up to the minute with our new Resource Alerts.

Enterprise Capture: Your Onramp to Business Process Automation

Today more than ever companies are seeking to reduce costs and...  View Now »

 

The True Cost of Legacy Systems

How well are you maximizing existing software assets? This webcast reveals the results of a commissioned study on top migration and modernization priorities for IT leaders.   View Now »

 

How To Maximize Your Virtualization Strategy and Deployment

Join award-winning technology journalist Stan Gibson in this webcast as he discusses how to enhance your virtualization strategy with the ROI, planning, implementation and platform advice. Exploit the business benefits of virtualization and successfully expand your current deployment.   View Now »

Resource Alerts

Get instant email notification when white papers, webcasts, and case studies are added to our library. Don't just be up-to-date—be up to the minute with our new Resource Alerts.

 
NEWSLETTER

Sign-up for the Blogs & Discussion Newsletter




*Required fields

By clicking the sign-up button, you agree to the Privacy Policy.

View all newsletters »

 
FEATURED SPONSORS
 
 
 
SPONSORED LINKS
 

Manage limitless content todayread EMCs 15-minute guide to ECM.

HP Exstream. Get a Free Document Assessment for Financial Services.

Take the Netezza TwinFin TestDrive!

Webinar: Jump-start your in-house e-discovery with Ringtail QuickCull from FTI Technology

Let Progress Software help your business make progress.

Best Practices to Reduce IT Operational Costs

Real-world testing ranks Trend Micro #1 against malware. See results.

Forrester: The real-world financial impact of Windows 7

Turn your desk phone and mobile phone into one with Sprint Mobile Integration.

Maximizing efficiencies with unified communications.

Stay informed with custom newsletters from Tech Dispenser

Selecting the Right Reporting Technology

An IT Leadership Action Plan for the Economic Recovery

Consolidate data centers and lower IT service costs. Learn How.

WAN optimization techniques significantly improve application performance. Read More.

The Revolution and Evolution of Private Cloud Computing

ROI of Application Delivery Controllers

Cut Costs & Green Your IT Operations with PC Power Management

Enterprise Capture: Your Onramp to Business Process Automation

Adobe® LiveCycle®solutions for intuitive user experience

Unlocking the Mainframe: Modernizing Legacy System to SOA

State of the Data Integration Market

Enhance Customer Loyalty through Higher Responsiveness

Achieving Business Agility with Application Grid

Seven Ways ITIL Can Help You in an Economic Downturn

Does your IDS really work? Find out with a free Endace Audit

Verint Systems. Discover the Power of Intelligence in Action"

CA ARCserve r12.5 is More Than Backup! Download Trial Version Today

Enterprise search helps employees get more done. Get the facts from Google.

See why ShoreTel is named best overall VoIP provider by Nemertes Research

Trend Micro ranked #1 against real-world malware. Read more.

AT&T Application Management & Hosting. Let us help you STRETCH

Microsofts new client operating system helped Pella reduce power consumption.

Efficiency goes up. Costs come down.

Dark Fiber from Sunesys Save on Unlimited Bandwidth with Fixed Costs.

Trend Micro ranked #1 against real-world malware. Read more.

Webcast: Solve Your Data Visualization Needs with Open Source BI

Webcast: Delivering the Enterprise-Ready Cloud

Ensure cost effective application delivery. Learn More.

Cloud Computing: The Impact CIOs See

What's Next for Enterprise Resource Planning?

Gartner Magic Quadrant, Application Delivery Controllers 2009

Global Research: CIOs Weigh In On Virtualization

Adobe® LiveCycle® solutions for business process automation

What's New in SOA Suite 11g?

Unleash the Power of Java with Oracle JRockit Real Time

SOA Best Practices and Design Patterns

Application Grid: Ideal Platform for IT Consolidation

Taking the Service Desk to the Next Level