Probably an AntiPattern. Could someone with more WikiExperience tell me if I should format it as such? Is it a duplication of another AntiPattern?

Tomography is the practice of firing a ray into a substance and gathering all the data, then blurring out the parts you don't want, allowing you to see only a small slice of the object. Think MRIs. 

TestingByTomography happens when you write IntegrationTest''''''s for a system but you don't make the system UnitTest''''''able. Your tests entirely consist of running the entire program and checking to see if it did what it said it was going to do, looking at only one or two small parts. 

For example, if you have a class that encodes a file into a new format, but you didn't design it with UnitTest''''''ing it in mind, your only testing option may be to run the encoder and then examine the file for only one small part of itself. This is particularly obvious with large systems, where you run the system through one cycle and then see if, say, it connected to a server (by checking logs). 

This is BadPractice because it doesn't really do what you expect it to do. These tests rely on the whole system working (or not working), not the party you're testing. In these kinds of test harnesses, you tend to find that if you break something during a refactoring, ALL the tests begin to fail. Imagine our X-Ray hitting a lead wall at some point. The tests output ceases to be helpful, and we've basically eliminated most of the benefits of testing.   

TestFirst development is a way to prevent this. Since you make sure every method is going to be tested, you ''must'' code in a fashion which allows UnitTest''''''ing of each component and each individual method. You'll naturally begin to isolate out external data sources into modules which abstract that detail (for instance, Clock.currentTime instead of gettimeofday), and allows for each module to be tested independently. 

----

I HaveThisPattern.  And while I agree that it's probably an AntiPattern in the general case, it's proved useful for the project in question.  The UnitTest''''''s have proven to be a good way to detect and back out changes that totally break everything, and can often localize the problem enough for me to debug it.

I'm doing a decent sized FanFiction site, all in PhpHypertextProcessor with a MySql backend.  There is little coupling between each page beyond the database (PHP is stateless, after all).  There is also little in the way of computation - it's basically a PutTheDamnDataOnTheDamnScreen app.  When there is something to compute, I've typically tossed it into its own class (DecoratorPattern, mostly) and unit-tested that separately.

The bulk of the unit tests are regexp tests on the generated HTML pages, using a known test database.  they basically run the page generation all the way through.  There's a test class for each page generator.  Since page generation usually consists of a couple SQL queries and a few calls to a third-party TemplatingInPhp library, I consider this acceptable granularity.

The annoying thing about this setup is that tests tend to be rather brittle on template changes.  I try to test only data in the database (TestingByTomography, after all, means throwing away the uninteresting data), but sometimes I need to make sure that fields come out in certain relations to other page elements, like form tags.  I also need to count how often data appears, because the most common error seems to be missing or unnnecessarily repeated data.  This also leads to brittleness.

It's still faster than clicking through the whole site every time though.  -- JonathanTang

Ahh, thanks. I was unclear. TestingByTomography is bad when it's the only kind of test you have. When your only source of information on the system's running state is, in fact, its running state, then you have a problem. IntegrationTest''''''s are important, but you need UnitTest''''''s as well, otherwise bug tracing can be kinda hard. -- DaveFayram

It's not clear to me that Jonathan's story is precisely the same topic, although it's clearly related; in general GUI-related testing has a lot of unique issues.

I like the name you chose, "testing by tomography", but it's granting too much credit. Tomography is based on very precise algorithms drawn from very exacting mathematics. The kind of testing you're talking is, by contrast, very ad hoc, and it's very hard to do what one thinks really should be done (lack of appropriate NullObjects, etc).

So I wonder if "testing by tomography" might better be reserved for an extremely methodical approach as-yet uninvented.

Perhaps instead you could call it TestingByColonoscopy. ;-)

Incidentally, it seems to me that what you describe has always been the de facto approach to testing whenever massive unit testing is not in place. -- DougMerritt

Hmm. Tomography seems to me to be really descriptive, but you may be right, it gives it too much credit. I came up with the name when I was playing blackbox on my copy of xemacs to eliminate stress over having to ''*insert hand waving from boss*'' "Just write some unitTests, but don't refactor that much." In the end, I felt about as sure of the system's integrity as the position of the damnable little balls in the BlackBox. That's what happens when UnitTest''''''ing becomes a marketing pitch. :\ -- DaveFayram