Review of "The TheNewMethodology" by MartinFowler.

Martin Fowler's essay is provocative, making some very strong assertions but not providing backing facts or figures.

He gives a very vague definition of "engineering methodologies", without identifying the specific methods he is referring to (such as he does give a list of agile methods). The closest he comes to an example is implying that the NASA space shuttle software group follows an "engineering methodology".

If this is the case, then some research on the actual process used by the NASA space shuttle software group provides counter examples, e.g. I found an IBM Systems Journal, March 1994, article on the space shuttle. [http://www.research.ibm.com/journal/sj/331/billings.pdf]

Contrasting this to Fowler, at one point he states "The goal of engineering methods is to define a process that will work well whoever happens to be using it", implying that these "space shuttle methodologies" do not take into account the people factor.

The above article, however states "In 1990 it was recognized that technical management of the software development process needed to reside with people who performed the processes. They knew better than anyone what worked and what could be improved. If the process could be optimized, the development teams had the insight to make the changes."

In other words, the space shuttle process has been directly in the hands of developers for at least 15 years.

NASA follows an incremental release strategy, and rather than having stable requirements they have (from late 70's to 1994, say 15 years) "undergone more than 3000 requirements changes. Since the late 1970s, the software required more than 382 000 source lines of code to be added, modified, or deleted. These changes were implemented via more than 900 software builds and 175 patches. IBM has provided these evolutionary software versions to NASA through 260 separate software releases."

So we are left wondering what are these bad "engineering methodologies" that Fowler is talking about, and who is using them anyway?

In the introductory section, Fowler lists three types of development - nothing, monumental, and agile. The problem is that he is lumping waterfall-based methodologies with other formal processes, and trying to contrast agile to the worst features of all the formal processes. 

In reality, waterfall methodologies have not been supported for a long time, and most modern formal methodologies follow an iterative process. Even then, following a waterfall model does provide documented benefits over "code and fix", benefits that some "agile" techniques (such as pair programming) have not yet conclusively shown.

So, we should actually probably be comparing four different types of methodologies and comparing modern formal methods with modern lightweight methods on their own merits. 

For example, RUP is commonly implemented as a formal method, although the framework also supports lightweight usage as Fowler discusses - however, in neither implementation should it be considered a waterfall-type methodology (the introductory material for RUP actually criticizes waterfall development).

When Fowler says "RUP can be used in a very traditional waterfall style or in an agile manner", a more correct statement would be "RUP could be used in a very traditional waterfall style, but this would be an incorrect usage and misunderstanding of the framework, or, it can be correctly used as either a formal method or in an agile manner". To state it can only be used two ways is an incorrect representation of RUP, which was originally quite formal but never waterfall.

Almost any type of methodology can succeed, as links such as Cockburn's article "Characterizing People as Non-Linear, First-Order Components in Software Development" show. Cockburn's article actually rates the people on the project as more important than whether you are using a particular methodology (be it agile or not).

There are many techniques that provide documented, proven improvement in the quality and schedule of delivered software. Some of these share features with agile methodologies, and some also increase predictability and add formality. 

Construx, Steve McConnell's company, has a good presentation covering many techniques which can quickly improve your software process [http://www.construx.com/docs/member/LowHangingFruit.pdf]. The recent edition of Steve's book, Code Complete 2, also provides some data on the effectiveness of various quality techniques, with prototyping and formal inspections being amongst the highest. In fact, the benefits of processes such as XP can be largely related to the collection of practices they use (pair programming being, in implementation, an informal code review).

There is also not necessarily much "new" about agile methodologies; in IEEE Computer, June 2003, "Iterative and Incremental Development:A Brief History", it is mentioned that NASA's Project Mercury used test-first development with half-day timeboxed iterations - Project Mercury was undertaken in the early 1960s!

Also quoted in that article is Gerald Weinberg, who states about a 1957 project at Motorola "the technique used was, as far as I can tell, indistinguishable from XP".

Many of the features discussed by Martin Fowler, such as expecting, and specifically catering for, changing requirements, iterative development, and an adaptive process (the higher levels of CMM are all about adaptive processes) are promoted by formal methods just as much by agile ones.

Overall, the paper has many important and valid points, but some of the more emotional statements should probably be taken with a grain of salt.

BTW. I think I have the issue of SD magazine where this was originally published. :-)

-- StephenGryphon