In the business world, we're often faced with EnterpriseApplicationProblems;
applications that are unreliable, fail to satisfy business needs well, and/or are difficult to change.

How can we address these problems?

First, we can attack the problem at the source, and start WritingApplicationsThatRunWell.

For the mess we're stuck with today, we'll need strategies for AdministeringIllBehavedApplications, then everything will go away.

-----

If the people doing production support of the running application are allowed effective feedback into the development effort,
one can move applications from the "ill-behaved" category to the "written to run well" category above.
This approach is supported by IncrementalDelivery strategies, including ExtremeProgramming.

-----
The question came up in a q&a session in a college course on Object Oriented Algorithms - How are large Software projects organized?  Students who are about to enter the Job Market want to know. Is there a pattern of organization one can expect to find in the RealWorld, or are there many workable patterns that might be encountered by an entry-level programmer?

'''One answer:''' Large software projects generally develop a platform framework -- a set of libraries and components for common application code.  This often begins as a platform abstraction layer (for portability) but soon grows to include logging, reporting, system health monitoring, configuration persistence and other things needed for running real production applications.  Preventing bugs from shipping is preferable, but when that fails then one needs as much information as possible about what the system was doing when a problem occurs.  Asking users doesn't cut it.  The app  should be tracking itself, or at least have a tracking ability (disabled by default) that can be turned on.

''See BigBallOfMud for one common (but bad) pattern.''

----

First, a quote from an article, Schadler, Ted. principal analyst, Forrester Research "Commentary: Ten tips for killer Web services" http://news.com.com/2030-7345-5136264.html

"In the context of discussing how to manage and administer deployed production software, an organization needs to have some "levers and knobs" to turn. In other words, "Put a power meter and steering wheel on every Web service". Schadler further suggests that it should be possible to ask, "Who's using the service? How often are they requesting it? How many requests succeed? How many fail?". In order to do this, he lists three needs:

* visibility
* accountability
* hands-on control"

The basic administration tools enable a view into system status and recording of diagnostic events. Many of the events will be recorded an accumulator and min/max extremes may be recorded. For example, as an important dynamic collection changes, we may want to know the number of elements and the high/low water marks. Many types of information may be monitored.

Types of monitored values and events
* system initialization, restart, shutdown
* object creation events (only for key resource allocation classes)
* call to monitored method
* semantically significant failures
* significant state change (of a class or the system) 
* configuration values
* outputs
* side effects
* probes -- an assertion that, when it fails, generates an error event that gets logged
* Performance
** number of operations over time 
* Resource Usage
** high and low water marks 
* audit
** transaction completion
** suspect events
** errors 

--StevenNewton