TestingFramework is an object-oriented approach to ProgrammerTest''''s pioneered in KentBeck's 1994 ''TestingFramework'' package for Smalltalk, later renamed SUnit (SmalltalkUnit). Beck's approach and code have since been adopted and ported to numerous other environments and languages. Have an object (TestCase) manage the execution of a single test. Each test should be isolated from the others, so it creates all its data before executing and destroys it when it's done (setUp() and tearDown()). Composites of tests (TestSuite) allow many tests to be run together. Passing a CollectingParameter (TestCollector/TestResult) to the tests as they run allows the collection of statistics. That's the framework - three objects (TestCase, TestSuite, TestResult). There are simpler ways of running automated tests, and many, many more complicated ways, but this architecture seems to hit a sweet spot. See also TestingFrameworks, FrameworkForIntegratedTest, CategoryMockObjects - for MockObject generation frameworks, http://en.wikipedia.org/wiki/XUnit ---- Originated in * http://www.xprogramming.com/testfram.htm ''Simple Smalltalk Testing: With Patterns'' KentBeck (reprint of the October 1994 *Smalltalk Report* article) * Chapter 30 "Simple Smalltalk Testing", in ''Kent Beck's Guide to Better Smalltalk'' by KentBeck, Donald G. Firesmith (Editor), Cambridge University Press, December 1998, ISBN 9780521644372 ---- '''Source code RegressionTesting libraries, by language:''' (see also a similar list at http://xprogramming.com/software ) * '''AdaLanguage:''' ** AdaUnit - Ada * '''ApacheAnt:''' ** AntUnit = task library for the ApacheAnt build tool. * '''AppleScript:''' ** AsUnitForAppleScript - AppleScript * '''ActionScript:''' ** AsUnit - (ASUnit) for ActionScript ''[in progress]'' ** AsTuce - (ASTUce) for ActionScript 3.0 * '''CeeLanguage:''' ** ACT - Advanced CUint Test for C on LINUX, Windows, Embedded system See http://kldp.net/projects/act ** CeeUnit - "cUnit" for C. A Unit testing framework for C that depends on glib. ** cfix - xUnit testing framework for Win32. Minimizes effort to author tests. Tests are compiled into a DLL rather than an EXE. A testrunner is provided to (selectively) run fixtures of one or more DLLs. GPL licensed. See http://cfix.sourceforge.net/ ** CGreen [CeeGreen] - Open source Unit testing framework for C. Includes process separation of tests and mock functions. See http://sourceforge.net/projects/cgreen/ ** CheckFramework - A mature well-tested unit test framework for C ** CUT - C Unit Tester system. See http://www.falvotech.com/content/cut for version 2.4, and http://www.falvotech.com/content/cut/3 for CUT 3.0pre1. Portable across all platforms which support Python; does '''not''' rely on any libraries (except its own), and is '''not''' an SUnit-clone (it relies on a TestCollector to aggregate tests). Suitable for embedded unit testing. CUT 3 promises to be easier to use from within IDEs via error messages resembling the error reports from popular compilers. ** ctest - http://astranetwork.com/products/ctest/ ** CUnit [CuNit] - Standard 'C', Platform independent unit testing framework - http://cunit.sourceforge.net/ ** CuTest - C. Works on Windows with MSVC and BCC32 (does not depend on glibc) ** InTheGuard - A simple C test harness intended for testing C libraries during installation. ** Min''''''Unit - A minimal unit testing framework for C - http://www.jera.com/techinfo/jtns/jtn002.html ** Cutter [CutterTestingFramework] - An easy to use unit testing framework for C - http://cutter.sourceforge.net/ ** CUnitWin32 [CeeUnitWin32] - Lean and mean framework with truly independent tests (e.g. global/static variables are initialised between tests). See http://code.google.com/p/cunitwin32/ ** C2Unit - Yet another C unit testing framework - http://github.com/cwyang/c2unit * '''CeePlusPlus (WhySoManyCeePlusPlusTestFrameworks, ConsiderationsForAndComparisonOfCeePlusPlusTestFrameworks):''' ** cfix - xUnit testing framework for Win32. Minimizes effort to author tests. Tests are compiled into a DLL rather than an EXE. A testrunner is provided to (selectively) run fixtures of one or more DLLs. GPL licensed. See http://cfix.sourceforge.net/ ** CppTestHarness (a modernized and upgraded flavor of CppUnitLite) ** CppUnit does C# as well as C++; 1.12 version shows some weird runtime errors ** CppUnitLite ** CppUtxOverview - an excellent alternative to CppUnit, though you need to implement parts yourself. ** Cutee, C++ Unit Testing Easy Environment. See http://codesink.org/cutee_unit_testing.html ** CxxTest ** Managed C++ - CsUnit (available for free under GPL) ** RwCppUnit - C++ with RogueWave and no StandardTemplateLibrary ** TestSoon - C++ with Macro Metaprogramming. See http://testsoon.sf.net ** TUT - C++ Templates. See http://tut.dozen.ru ** EasyUnit ** AnyUnit - addin for Microsoft Visual Studio.NET to support any C++ unit testing framework through customizable layouts. See http://www.anyunit.com ** UnitTest++ (this should be first in the list :-) --RS) - See http://www.gamesfromwithin.com/articles/0603/000108.html ** CUnitWin32 [CeeUnitWin32] - Lean and mean framework with truly independent tests (e.g. global/static variables are initialised between tests). See http://code.google.com/p/cunitwin32/ ** CuteTest - with eclipse plug-in, modern header-only design ** CatchTest - modern header-only design * '''CeeSharp:''' ** CsUnit (available for free under GPL) ** And see also the list of test frameworks for .NET, listed below. * '''CommonLisp:''' ** See the list of CommonLisp test frameworks at http://www.cliki.net/Test%20Framework ** See the misnamed CommonLispUnit which describes the CL RegressionTest framework RT by RichardWaters ** CLOS-unit a CLOS implementation for CLOS/Lisp unit testing. See http://a.die.supsi.ch/~pedrazz/clos-unit ** CommonLispXpTest ** EmacsLisp (RegressEl), LispUnit, CommonLispUnit -- ''(Not "StarUnit" libraries, but useful, nonetheless.)'' * '''ColdFusion''': ** cfcUnit - A unit testing framework for ColdFusionComponents. See http://www.cfcunit.org ** cfunit - ColdFusion. See http://www.macromedia.com/devnet/mx/coldfusion/articles/cfunit.html ** CFTestingKit - ColdFusion. See http://cftestingkit.sourceforge.net ** CFUnit - ColdFusion. See http://cfunit.sourceforge.net ** MXUnit - ColdFusion. Unit Test Framework and Eclipse Plug-in for CFMX 6-7-8. See http://mxunit.org ** RocketUnit - ColdFusion. Unit Testing Framework. Single CFC implementation. See http://rocketunit.riaforge.org/ * '''DelphiLanguage:''' ** BubelenProject - Bubelen is a Delphi integrated system for building and maintaining bubbles. A bubble is a piece of code for testing, developing, profiling, debugging, and stress testing. Bubelen is open source. ** DelphiUnit - A couple of different Delphi testing frameworks * '''EiffelLanguage:''' ** EiffelUnit - Unit test framework for Eiffel (formally known as ETest or EiffelTest) * '''GaucheLanguage:''' ** GaUnit - for Gauche * '''GroovyLanguage:''' ** GroUnit - for Groovy * '''HaskellLanguage:''' ** QuickCheck - an automated "agitator"-style test generator ** HaskellUnit - xUnit-style tester, for TestDrivenDevelopment * '''IconLanguage:''' ** IconUnit - IconLanguage unit test suite * '''IoLanguage:''' ** IoUnitTest - for the IoLanguage * '''JavaLanguage:''' ** ConcordiOn - acceptance testing from plain English descriptions of behaviour. See http://www.concordion.org/ ** Lobo, Continuous Tuning - for performance testing and monitoring. See it at http://www.oncast.com.br/dev/lobo/index_en.htm. ** TestEnGee (TestNG) - Advanced testing framework for Java. See http://www.testng.org/ ** JavaUnit - "JUnit" library for Java. (a Wiki-unfriendly name). There's also JakartaCactus: a server-side unit testing framework that extends JUnit. ** actiWATE - Java, for testing web applications, emulates browser, supports JavaScript. See http://www.actiwate.com/ ** DbUnit = JUnit extension targeted for database-driven projects. ** Jameleon - An Automated Testing Tool - http://jameleon.sourceforge.net/ (in Java, for general acceptance testing) ** JBehave (JayBehave) - for BehaviorDrivenDevelopment ** JwebUnit = FacadePattern refactoring on top of HttpUnit and JavaUnit. ** QeTest - for Java, hosted in Xalan-J ** TagUnit - for JSP custom tags ** WebUI for JUnit - JUnit UI framework allows running your unit tests through web browser. http://www.softxp.com/webui/ ** HtmlUnit - based on the apache httpclient utilities, for use within JUnit ** J2MEUnit - for Java 2 Micro Edition - http://j2meunit.sourceforge.net/ ** a good list of JavaLanguage testing tools - http://opensourcetesting.org/unit_java.php ** JutLand ("Java Unit Testing - Light, Adaptable 'n' Discreet") by KevlinHenney -- [not downloadable?] * '''JavaScript:''' ** JsUnit - JavaScript TestingFramework ** JsSpec - for BehaviorDrivenDevelopment ** IeUnit - JavaScript testing framework for InternetExplorer ** AsTuce - (ASTUce) for JavaScript, JScript, ActionScript, etc. ** TestSimple - TAP-based testing framework ** JsnUnit - JavaScript TestingFramework for AspDotNet client and server * '''KayLanguage:''' ** KayUnit - for K (KayLanguage) and KSQL * '''LingoScriptingLanguage:''' ** LingoUnit - for LingoScriptingLanguage * '''LuaLanguage:''' ** lunit - A unit testing framework for Lua - http://www.mroth.net/lunit/ * '''Matlab (MatlabLanguage)''' ** The MATLAB Unit Test Framework - http://www.mathworks.com/help/matlab/matlab-unit-test-framework.html ** xUnit - http://www.mathworks.co.uk/matlabcentral/fileexchange/22846 ** MUnit - http://xtargets.com/cms/index.php?option=content&task=view&id=19 ** mlUnit_2008a - http://www.mathworks.co.uk/matlabcentral/fileexchange/21888 ** mlUnit - http://sourceforge.net/projects/mlunit/ ** Unit testing tools - http://www.mathworks.com/matlabcentral/fileexchange/7404 ** MATUnit - http://www.mathworks.com/matlabcentral/fileexchange/7979 ** munit - http://www.mathworks.com/matlabcentral/fileexchange/7845 * '''ObjectiveCee:''' ** ObjcUnit - ObjectiveCee ** OcUnit - ObjectiveCee ** UnitKit - Strictly speaking, of Mac OS X only, as it is tightly integrated with XCode. * '''PerlLanguage:''' ** PerlUnit - for Perl - a fairly straight port of JavaUnit to Perl ** PerlTap - the basis of the standard Perl testing framework. ** LectroTest - automatic, specification-based testing for Perl. ** PerlActor - automated executable acceptance test framework for Perl. ** PerlTestClass - another xUnit framework for Perl, built on top of PerlTap * '''PhpLanguage:''' ** PhpUnit - for PHP3 ** SimpleTest - PHP unit tester and WebTester with MockObject generation * '''PLSQL (PeeEllEssQueueEllLanguage):''' ** PlUnit - unit testing framework for PL/SQL Oracle 8i and higher ** PlSqlUnit - (aka utPLSQL) for PL/SQL ''(Oracle 7.3.4 and higher)'' * '''PowerBuilder:''' ** PowerBuilderUnit (PBUnit) - for PowerBuilder * '''PowerShell:''' ** PowerShellScriptsForTesting * '''PythonLanguage:''' ** DocTest - for Python: test cases serve as documentation examples and vice versa. ** PythonUnit - for PythonLanguage (various offerings) ** TestOob - extensions for PythonUnit ** PyTest - the framework used for PyPy ** Pikzie [PikzieTestingFramework] - An easy to write/debug Unit Testing Framework for Python. - http://pikzie.sourceforge.net/ * '''R (ArghLanguage):''' ** RUnit - http://cran.r-project.org/web/packages/RUnit/index.html * '''RebolLanguage:''' ** RebolUnit - for RebolLanguage * '''RubyLanguage:''' ** RubyTestUnit - for RubyLanguage ** RSpec - for RubyLanguage - http://rspec.info * '''SchemeLanguage:''' ** SchemeUnit - as minimalistic as Scheme itself * '''UnixShell''''''s:''' ** ShUnit - for Bourne-derived shells * '''SmalltalkLanguage:''' ** SmalltalkUnit - the original; for the SmalltalkLanguage * '''SmlLanguage:''' ** SmlUnit (SMLUnit is included in the SML# distributions) * '''SQL-PL''' ** db2unit - Unit testing framework for IBM DB2 for LUW: http://github.com/angoca/db2unit * '''Tcl (ToolCommandLanguage):''' ** DejaGnu - The GNU testing framework, based on Expect (which in turn is built on Tcl). ** XoUnit - for Extended Object Tcl (ExtendedObjectTcl) * '''TransactSQL:''' ** TsqlUnit - for Transact-SQL (Microsoft SQL Server) * '''VisualBasic:''' (VbClassic) ** AspUnit - test ActiveServerPages (from VisualBasic) ** ComUnit - for Microsoft VisualBasic and COM ** CsUnit - VisualBasic .NET (available for free under GPL) ** Excel''''''Unit - see VbaUnit ** SimplyVbUnit - a VB6 unit testing framework ** VbaUnit - for VisualBasicForApplications (which gets around VbUnit's incompatibility with VBA) ** VbLiteUnit - a really light-weight test platform for VB and VBA that works pretty darn well. ** VbUnit - VisualBasic (for VB6) ''[See Also DotNetUnit and "DotNet platform" below]'' * '''VisualBasic:''' (VbDotNet) ** See the list of test frameworks for .NET, listed below. * '''VisualFoxPro:''' ** FoxUnit - an OpenSource unit testing framework for VFP. See http://www.foxunit.org * '''WebObjects:''' ** WoUnitTest - for WebObjects * '''XbasePlusPlus:''' ** XbUnit - for Xbase++ * '''XML (ExtensibleMarkupLanguage):''' ** XML Schema Unit Test (SUT): http://sut.sourceforge.net/ ** XmlUnit - for assertions about XML using Junit and Java JAXP ** XsltUnit - for XSLT Stylesheets * '''XoTclExtension:''' ** XoUnit - for '''Extended Object Tcl''' (ExtendedObjectTcl) of the ToolCommandLanguage (Tcl) '''Multi-Language Testing Frameworks:''' * TetWorks - (aka TET) Open Group multi-lingual test framework '''For the DotNet platform (a grouping)''' * CsUnit - for the DotNet platform (CsUnit is available for free under the GPL) * DotNetUnit - for the DotNet platform * DotUnit - for the DotNet platform * HarnessIt - .Net languages (commercial tool with "a modest fee") * MbUnit - Unit testing and Model-Based testing. See http://mbunit.tigris.org * NUnitAsp - Extension to NUnit for testing ASP.NET applications: http://nunitasp.sourceforge.net/ * NunitFramework - NUnit for the .Net platform. See http://sourceforge.net/projects/nunit * ExUnity - "X-Unity" testing for .Net platform (name Wikized), commercial tool * PowerShellScriptsForTesting - to write tests in PowerShell, which can access, run and test functionality in the DotNet CLR. * ZaneBug - Advanced unit testing tool for .Net platform (NUnit compatibility). * "Open Source Testing Tools in C#", a listing: http://csharp-source.net/open-source/testing-tools * XunitDotNet - NUnit (NunitFramework replacement): http://www.codeplex.com/xunit '''For HTTP:''' * HttpUnit - for web server testing * HtmlUnit - for web server testing * WebInject - Web/HTTP Test Framework - http://www.webinject.org * Selenium - client-side web application testing - http://selenium.thoughtworks.com/ * DieselTest - in Delphi 5, for Windows NT * Tellurium - Automated Web Testing Framework - http://code.google.com/p/aost/ '''For the ZopeApplicationServer:''' * See ZopeUnit for list of TestingFramework''''''s that run in the ZopeApplicationServer. '''Works in all languages:''' * PoorMansTestingFramework * NeUnit - a GUI front end for multiple back end TestingFrameworks. Currently C/C++ & Python. '''Data Driven:''' * ConcordiOn - acceptance testing from plain English descriptions of behaviour. See http://www.concordion.org/ * FitWiki / FIT / FrameworkForIntegratedTest = multi-language table driven acceptance testing tool * FitNesse = FIT and tools, all rolled into one * ForteToolUnit - Forte 4GL (TOOL) * HASTE = HighlevelAutomatedSystemTestEnvironment * JxUnit = XML driven testing (for Java) '''Automated Test Generation:''' * JTest by Parasoft (JavaTest) * Agitar's Agitator (AgitarsAgitator) * NTest by Incenteus * QuickCheck for the HaskellLanguage. '''Model-Based Tesing (MBT):''' * TestOptimal, xml based scripts, Data-Driven Testing(DDT), requirements tracking, see http://TestOptimal.com/tutorials/Overview.htm '''"lint" Type "Smell Detectors":''' * PeeEmDee (PMD) -- http://pmd.sourceforge.net/ '''PerformanceTests:''' * DieselTest - scripted web testing in Delphi 5, for Windows NT * TheGrinder - "The Grinder, a Java Load Testing Framework" * JMeter (JavaMeter) - http://jakarta.apache.org/jmeter/index.html * LoadSim - http://loadsim.sourceforge.net/ * LoboPerformanceMonitoringTool - http://www.oncast.com.br/dev/lobo/index_en.htm * Mercury Interactive's LoadRunner - http://www-svca.mercuryinteractive.com/products/loadrunner * OpenSta (Open System Testing Architecture) - A distributed software testing architecture based on CORBA. * RUBiS - "RUBiS can be used from a web browser for testing purposes or with the provided benchmarking tool." - http://rubis.objectweb.org/ * TestMaker - http://www.pushtotest.com/ptt ---- See also: * http://xprogramming.com/software - RonJeffries's list. (If the URL fails, use UrlMangling.) * http://www.manageability.org/blog/stuff/open-source-automated-test-tools-written-in-java/view - "Open Source Automated Test Tools Written in Java" * http://www.opensourcetesting.org/unit_java.php - "Java unit testing tools" * http://java-source.net/open-source/testing-tools - "Open Source Testing Tools in Java" * http://java-source.net/open-source/web-testing-tools - "Open Source Web Testing Tools in Java" * http://csharp-source.net/open-source/testing-tools - "Open Source Testing Tools in C#" * http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks - Wikipedia list, much like this page ---- How do I do test-then-code with C in embedded systems? Is there anywhere I can go where people are discussing this? I'm getting the idea that it's possible - even on simple PIC or AVR processors. Has anyone tried it? ''see ExtremeProgrammingBareMachine.'' ---- You've cobbled up an equivalent under Bourne shells with functions. Contact me for details. One trick I found useful has been to put unit tests (for shell files to be sourced) after a "am I being sourced" guard, and run the unit tests if being executed. - JosephBeckenbaumer. -- KFrede ---- I need an EmailUnit ... pop, smtp, imap, ... DavidEscala See http://sourceforge.net/projects/smtpunit/ -- MarkusSrank ---- As some of our test suites get large, we need to speed up our tests so we can run them quickly. ToplinkInMemory allows us to stub out the database so that we can continue to run all tests before checking in! -- TedOgrady ---- One of the pieces of information that is demonstrably missing about xUnit lore is ''when'' the first version of sUnit came into existence. It can't be any later than 1995, I'd expect, since that's when C3 happened, and thus, XP came into being. I'm pretty sure sUnit existed at least at that time. But, did it exist even before then? If so, when? -- SamuelFalvo ---- I would appreciate it if folks (or that Shark thingie :) ) would stop removing my updates to CUT, a unit testing package that '''I''' wrote. Thank you. :) -- SamuelFalvo ---- CategoryTesting, CategoryFramework