Bridge Building and Software Development

    There are vocal proponents that software development is unlike any other engineering process. This may be true, but for different reasons. I was reading a PhD Thesis from Robert Powell at the Stevens Institute of Technology. Powell's thesis is titled A Definition of High-Level Decisions in the Engineering of Systems.
    In the thesis was a brilliant set of statements. From the Standish reports (which of course have statistical confidence interval issues), approximately 85% of IT project fall short of achieving complete success. The problem with Standish of course is "success" is defined as On Budget, On Schedule. It does not take into account the other variables of a project. But that's another topic. I want to get to the bridge building processes.
    In the Standish report the failures are attributed to Human factors as opposed to Technology. In 1986 Alfred Spector, president of TransArc co-authored a paper comparing bridge building with to software development. Spector was the former Director of the Information Technology Center, Carnegie Mellon and is now a VP at Google.

    Bridges are normally built on time, on budget. Software never comes in on time or on budget. Spector found when a bridges falls down (or has technical and business difficulties) it is investigated, and a report is written on the cause of the failure. The resulting information is used to produce or refine the decision or decisions in an attempt to prevent recurrence of similar failures. This report becomes part of an architecture that is used to guide future decisions for successful bridge construction.

    However, in the computer industry failures are covered up, ignored, and or rationalized, which may be due to the absence of procedures for analyzing failures.

    Subsequently, due to he lack of a rational explicit process, the same mistakes continue to occur and negatively impact the success of software development projects. Because of human failings, critical information is lost, decision making becomes difficult, and mistakes are repeated. It is the repetition of mistakes and the occurrence of new ones, because of improper decision making, that result in canceled projects, cost overruns, time overruns, and failure to provide expected features.

A second reference is a US Air Force study stating three common reasons for software system development failure:

• Risks associated with teams - if the team of developers, acquirers, end user, and system maintainers had not worked together before and did not learn to communicate effectively, they were not likely to develop a successful system.
  
• Risk associated with technology - Teams that pursued a new technical approach (for
  example, the first foray into client-server computing) found that the lack of experience
  with a new technology, architecture, or development approach contributed to failure.
  
• Risks associated with requirements - the most often-cited reason for failure
  was poor management of requirements characterized by frequently changing
  requirements, requirements that were not well understood, and requirements proliferation

A KPMG report cites the principal reasons for failure among IT projects as poor project planning.

No project can really get off the ground without a valid plan. The project plan reflects many decisions made during the development process (e.g., compatible tasks, schedule, budgets, defined and acceptable risks, etc.) and a poor plan is simply a reflection of poor decision making.

So in the end it's not about developing the software. It's not about applying one method over another. It's about project planning, good requirements, and having teams that can work together with the technology selected for the project.

When someone says bridges are not software, they're right. Maybe software development should learn from bridge building and stop doing poor engineering processes. With the advent of COTS products as the basis of most Enterprise IT, it's time to stop making software "special."

If the software development business doesn't start behaving like they are building bridges - or other projects that increase the body of knowledge from their failures - no "magic beans" are going to improve the outcomes.