Author of a paper and presentation titled "Misconceptions of the Agile Zealots", presented recently at the Atlanta SPIN (http://www.atlantaspin.org or http://www.cc.gatech.edu/SPIN/ver002/Archives/061703.html), Silicon Valley SPIN (http://www.gd-ais.com/sv-spin/Events/schedule.htm), and North Jersey SPIN (http://www.njspin.org/).
See http://www.toa.com (The Object Agency) and in particular his articles page, http://www.toa.com/shnn?searticles. This specific paper can be downloaded from http://www.toa.com/pub/Misconceptions.zip.
-----
In 2000 he posted this to news:comp.object.
After you read several pages of misdirection and insinuation disguised as erudition, you will read 'I am not interested in any more "XP battles."'
So he seems to have reversed his stance, with his recent "AgileZealots" career shift.
'''Battles I Cannot Win'''
"Those who cannot remember the past are condemned to repeat it."
-- George Santayanna (1863-1952), from The Life of Reason
[1905-1906], Volume 1, Reason In Common Sense
Introduction
Over the past couple of years, I have watched as interest in Extreme
Programming (XP) gradually consumed comp.object. In addition to the
informational and question-answering posts, there has been a flurry of
zealotry.
Zealotry takes many forms. Some posters felt it necessary to go far
beyond merely advocating/prosoelytizing XP. They chose to directly
confront not only existing object-oriented technology, but the
established base of software engineering in general, human psychology,
and management practices.
The onslaught has had a very noticeable impact on the traffic in
comp.object. People attempting to have discussions on non-XP traffic,
for example, sometimes find their threads taken over by XP issues.
Others simply refuse to post, lest they get caught up in the fanaitical
vortex. (I have received several pieces of private e-mail on this
topic.)
There has always been fanaticism and zealotry in comp.object. Booch
advocates slugging it out with Rumbaugh (OMT) advocates and
Shlear-Mellor advocates, for example. Language wars were also common,
e.g., Eiffel, C++, Smalltalk, Java, and Python. Of course, there were
the nitty-gritty detailed threads on the real definitions/values of
such things as covariance, polymorphism, and single versus multiple
inheritance.
The pre-XP mix of people in comp.object included good numbers of
university professors, grad students, methodologists, and gurus of all
denominations. People doing real object-oriented projects, who had an
immediate need for information, made up a significant portion of the
posters.
To be fair, there are at least 2 other major factors that have colored
what we see in comp.object:
1. The Internet itself. Until the early 1990s, the Internet was
primarily the domain of people who understood technical matters,
and who also had the resources to exploit it. These people
included universities, people working on government contracts,
large telecommunication companies, and technical consultants
for hire, for example.
Nowadays, in the "dot.com world," the Internet is a model of
cultural diversity. This diversity is wildly amplified by all the
new avenues (and cross avenues) for communication. One no longer
has to understand the differences between newsgroups, mailing
lists, bulletin boards, and chat rooms. You can tap into just
about any on-going conversation, from just about anywhere.
Our enriched discussions make it very clear that we "Internet
old-timers" can no longer make the same assumptions about our
fellow participants as we once did.
2. A branch of sociology called "diffusion research," investigates
how a new idea (an innovation) moves through a society. Diffusion
researchers study how new ideas are communicated, how they change
the society, and how the diffusion (communication) changes the
new idea itself.
Diffusion researchers identify the phases of an idea's acceptance,
and the types of people involved in those phases. For example,
the first on the scene are referred to as "innovators," while the
last on the scene (if they ever make it) are referred to as
"laggards."
You can get some idea of the phases of an innovation's acceptance
by looking at the labels given to the people involved, e.g.,
innovators, early adopters, early majority, late majority,
late adopters, and laggards.
The innovators for object-oriented technology were involved
prior to 1970. The early adopters showed up in the first half
of the 1980s. The early majority started showing up in the second
half of the 1980s. Somewhere towards the end of the 1990s, the
late majority began arriving.
Those people coming late to a technology have a distinctively
different outlook than those arriving early.
Different Playing Fields, Different Rules
In the not-too-distant past, discussions in comp.object were governed
by certain "rules" (conventions really). For example, it was perfectly
permissible (and, in many cases, encouraged) to cite literature
references to back up your point. This process was an interesting cross
between the scientific method and legal arguments (i.e., using
precedence to argue the meaning of a point).
Real case studies, as well as data and stories from real projects, were
especially prized.
One of the more commonly-invoked rules was "some data is better than no
data." This could be trumped using the "more data is better than less
data rule." Of course, there were endless discussions involving
statistical significance, the context of the data, and what one could
and could not generalize from the information at hand.
Another, very important, rule was that you had to work within the
confines of nature as defined by other disciplines. For example, we
have to accept the limitations of the human mind to learn new things.
We have to understand that not everybody can do everything equally
well. "Too many cooks" can spoil the "broth" regardless of the context.
It is also important to realize the value of particular activities,
tools, and concepts. E.W. Dijkstra and Harlan D. Mills both advised us
that designing programs correctly was far better than testing and
debugging a program into correctnes. This lead to another rule: don't
be sloppy now hoping that your sloppiness will be caught later.
Dilbert's boss ("Mr. Twin Peaks") gave us an "anti-rule" (a rule in
reverse) when he said "Anything that I don't understand has to be
easy." This told us again that we had to respect existing bodies of
knowledge and experience. If we were not willing to investigate a
methodology, or a discipline, for example, we should be very careful in
our assumptions about that item.
Sir Isaac Newton (1642-1727), in a letter to Robert Hooke, reminded us
that "If I have seen further (than you and Descartes) it is by standing
on the ShouldersOfGiants." This taught us to first investigate what
is already known or in place, before setting out in a "new" direction.
Doing this would either give us a "head start," or show us that we did
not need to reinvent the wheel.
The abundant diversity in the object-oriented world also made us aware
that the "playing fields" were not all the same. For example, depending
on who your were talking to, a "class" could be a simple template, a
template plus a creation mechanism, or a set of items constructed using
a particular template. Some people talked about "classes and
inheritance," while others sought out "exemplars and delegation." Types
and classes are the same thing for one group, while being fundamentally
different for another.
One of the more interesting and important playing field distinctions,
with regard to object-oriented software engineering, is that among the
artificial intelligence sector, the Smalltalk sector, and, for lack of
a better label, the C++ sector. (Eiffel, Ada, Java, and other
programming languages are all used by people who could be said to fall
into this sector.)
"Software Engineering" in its most exacting form lives in the C++
sector. The denizens of this sector typically value metrics,
methodologies, engineering knowledge and disciplines, technical
literature, and formalized approaches in general. They strongly tend to
value science and engineering over "magic." For these people, it is not
enough to do something well. You have to know that you did it well.
Lord Kelvin (William Thomson), who lived from 1824 to 1907, speaks for
the C++ sector when he says
"When you can measure what you are speaking about, and express it
in numbers, you know something about it; but when you cannot
measure it, when you cannot express it in numbers, your knowledge
is of a meager and unsatisfactory kind: it may be the beginning of
knowledge, but you have scarcely, in your thoughts, advanced to the
stage of science."
When he came up with FLEX, Alan Kay was looking for a computer for the
rest of us, i.e., a computer for non-programmers. He carried this idea
forward to Dynabook ("Dynamic Book"), and then to Smalltalk. Ideas like
a simple language syntax and semantics, a graphical user interface, and
the mouse could all be combined to bring the computer closer to the
masses.
It is no accident that many Smalltalk advocates see little value in
formalized methodologies, metrics, and other artifacts of the C++
sector. Since it is so easy to quickly create and modify applications
why would anyone bother with some rote process? As knowledge of
object-oriented software engineering became more widespread,
Smalltalkers were mercilessly harassed by people asking them "what
methodology to you use?"
When Rebecca Wirfs-Brock and her cohorts came up with "responsibility
driven design" (RDD) in the late 1980s, she provided some relief. Now,
when asked about one's choice of methodologies, Smalltalkers could (and
did) say "Wirfs-Brock." Kent Beck, Ward Cunningham, and others
contributed items such as class-responsibility-collaboration cards to
the Smalltalk arsenal.
To be sure, those in the Smalltalk sector can become very nitty-gritty
technical when discussing such things as typing, the mathematics of
inheritance, and reflective systems.
The terminology, concepts, and approaches found in the artificially
intelligence sector differ significantly from those in the other two
sectors. The Common Lisp Object System (CLOS) boasts full-blown
metaclasses (classes whose instances themselves are classes), for
example. The word "class" also takes on a different connotation,
meaning the set of all instances of a pattern.
The artificial intelligence sector is more free-form than the Smalltalk
sector. Progress is usually made in a heuristic (experimentation,
evaluation, or trial-and-error) manner. Methodologies and metrics are
remotely important, if at all.
The Arrival of the XP Contingent
Extreme Programming came from the Smalltalk sector. Like RDD ten years
earlier, it is an approach that comes from the programming language
level upwards -- as opposed to many methodologies that arise in the C++
sector which, almost exclusively, work from high-level concepts
downward. XP is unapologetically code-centric.
Rather than evolving it from a more complex methodology (e.g., the
Rational Unified Process), Kent Beck took what he knew from his decades
of software development experience and crafted an approach that would
greatly amplify what he considered good programming practices. If you
understand the Smalltalk sector of the object-oriented community, many
facets of XP will look and sound familiar.
Kent is very concerned with the human element in software development.
So, there is a huge dollop of people-related issues in XP. Some of
these people issues are fairly tame. For example, XP advocates keeping
the number of hours worked to a reasonable level, e.g., 40 hours per
week.
Some of the XP people issues do not initially make much sense to people
in from the C++ sector. Specialization of skills to the point of
creating a new job category is not allowed. There is also a lot of talk
about intelligence and job descriptions.
About the only way I can begin to describe these (to me) very strange
ideas is this. Somewhere in the past, somebody came by and disparaged
coders, i.e., people whose primary job function was writing code. In a
reaction to this abuse, several things happened:
* Non-coding activities became trivialized, and were considered
less important than coding. So, for example, analysis and design
activities that had non-code components pretty much disappeared.
The products of these processes were mocked as "pretty pictures."
Software testing was something that anyone could learn, and
anyone could do well.
* Horror stories, not always untrue, of documentation for
documentation's sake became almost a mantra. Another commonly-
recited point was that non-code documentation always gets out
of date, so there is no sense in producing it (or more than a
very small amount of it) at all.
(Apparently, one is not supposed to point out the differences
between a bad idea and a bad implementation of a good idea.)
* Many (most?) aspects of one's job description became intelligence
issues. Attempting to take away any activity from a programmer
was viewed as saying the programmer was not intelligent enough
to perform the activity. Issues such as having too many different
things to do, lack of demonstrated talent or desire, or mindset
conflicts were all ignored.
* Classic software project management became daemonized. Self
management (or emergent management) was the new norm.
You could say that very few, if any, restrictions were placed upon
programmers.
Another confusing (to me) situation involves XP advocates using terms
found in the C++ sector, but with different definitions, and without
any metrics. For example, there is a lot of talk about "quality."
However, when asked for definitions or metrics to support the quality
claims, XP advocates are not very forthcoming.
There are other points of contention that I have with XP. However, in
my discussions, I have become acutely aware that we are playing by two
very different sets of rules, on two very different playing fields.
It is not that my values and my perspectives are right, and XP's are
wrong. It is just that we are so far apart on the issues that it would
take forever to get anywhere close, and that is assuming that that is a
desirable goal.
Adopting XP in its current form, goes against most of my training,
research, and experience. It also conflicts violently with my training
as a scientist. Further, XP can evolve and improve on its own without
any "assistance" from Ed Berard. Thank you.
People like Topmind and Elliot see themselves as crusaders, fighting
for a cause. While I do crusade, I have no vested interest in, nor
desire to see, XP go away. In addition, commenting (positively or
negatively) on XP does not further my career, nor does it put any money
in my pocket.
Kent Beck is very right. We are talking across a paradigm gulf.
I am not interested in any more "XP battles." If you want to say that
XP is righteous has defeated the evil Ed, be my guest. Now, and in the
future, I will make every attempt to duck XP questions and any
references to XP technology.
I will be glad to discuss my views on software engineering in general,
and on object-oriented software engineering in particular.
Let the celebration begin!
-- Ed
----
''In 2000 he posted this to news:comp.object.''
''After you read several pages of misdirection and insinuation disguised as erudition, you will read 'I am not interested in any more "XP battles"'.''
''So he seems to have reversed his stance, with his recent "AgileZealots" career shift.''
That's interesting background on him, thanks. If I'm not mistaken, he started news:comp.object (?). I don't monitor it ... has XP continued to dominate over the past 3 years, and has Ed resumed commenting on XP there?
For those who haven't yet read the paper, it covers misconceptions '''about''' agile advocates ("anti-agile misconceptions") as well as misconceptions '''by''' agile advocates ("agile zealot misconceptions"). The stated goal of the paper is "not to discredit agile methods, or their advocates. Rather, the goal is to recalibrate the on-going exchange of technical ideas." I read his paper in that light, all 110 pages, and found it thought-provoking. (About half of the paper is references.) YMMV.
''Kent Beck is very right. We are talking across a paradigm gulf.''
In the past year, I have found a lot of presentations and papers by people (even from the SoftwareEngineeringInstitute) looking harder for compatibility than for incompatibility. I don't consider myself an expert on the same level as KentBeck or EdwardBerard; I'll readily admit that I still have a lot to learn and experience, and I definitely expect my opinions on these matters to evolve over time as a result. Right now I'm personally not convinced that the paradigm gulf between agile and everything else is so large as to be unspannable. I visualize the bridge as a spectrum, shades of gray as opposed to black & white. (Or rainbow, like the CrystalMethodologies?) I'd love to hear others' thoughts on this.
There's probably a more appropriate [existing?] Wiki page than this for a general discussion of the ParadigmGulf [ComparingParadigms, ExtremeParadigmShift, MixingParadigms, ParadigmShift, SoftwareDevelopmentImprovementParadigmShift, TimeForaParadigmShift ?]. Anyone have a recommendation, and/or care to refactor?
-- KarenSmiley, Sept. 19, 2003
----