TODO
:Introduce ITestFactory, deprecate @Factory
:Introduce Annotation Transformers
===========================================================================
5.1
Added: @Test(sequential = true)
Fixed: TESTNG-102 (Incorrect ordering of @BeforeMethod calls when a dependency is specified)
Fixed: TESTNG-101 (HTML output contains nested tags and a missing
tag)
Added: support for specifying test-only classpath (http://forums.opensymphony.com/thread.jspa?messageID=78048&tstart=0)
Fixed: TESTNG-93 (method selectors filtering @BeforeMethod)
Fixed: TESTNG-81 (Assert.assertFalse() displays wrong expected, actual value)
Fixed: TESTNG-59 (multiple method selectors usage results in no tests run)
Fixed: TESTNG-56 (invocation of @Before/AfterClass methods in parallel/sequential scenarios)
Fixed: TESTNG-40 (failures suite does not contain @Before/After Suite/Test methods)
Fixed: TESTNG-37 (allow passing null parameter value from testng.xml)
Fixed: TESTNG-7 (display classname when hovering method)
Eclipse plug-in
Added: run contextual test classes with parameters from suite definition files
Added: TESTNG-100 (Show HTML reports after running tests)
Added: TESTNG-97 (Double click top stack to raise comparison)
Added: TESTNG-84 (plug-in UI for suite option should support absolute path)
Added: TESTNG-20 (copy stack trace)
Fixed: TESTNG-72 (display groups with non-array values)
Fixed: TESTNG-64 (Eclipse plug-in applies added groups to all launch configurations)
Fixed: TESTNG-28 (Cannot select groups from dependent eclipse projects)
Fixed: TESTNG-25 (do not display fully qualified method name when running contextual test class)
Improved behavior:
TESTNG-98 (temporary files have guaranteed fixed names)
TESTNG-95 (Assertion failed comparison trims trailing ">")
TESTNG-70 (TestNG prevents eclipse from opening an older CVS version of a java class)
display of test hierarchy information (TESTNG-29)
===========================================================================
5.0.1
Eclipse plug-in
Added: Output directory for the tests
Added: Can now specify listener classes
===========================================================================
5.0.1
Fixed: reports generated by SuiteHTMLReporter do not work with JDK1.4
===========================================================================
5.0
Added: Ant task: support for JVM, workingDir, timeout
Added: Stack traces can be interactively shown in the HTML reports
Added: Link to testng.xml in the reports
Added: New structure for reports, suites go in their individual directory
Added: @Test(suiteName) and @Test(testName)
Added: The stack traces in reports do not include TestNG frames (system property testng.exception)
(see: http://groups.google.com/group/testng-dev/browse_thread/thread/9f4d46ade10b0fda)
Fixed: Exit with error when no methods are run
(see: http://groups.google.com/group/testng-dev/browse_thread/thread/3c26e8a5658f22ac)
Added: List of methods in alphabetical order
Fixed: Class-scoped annotations were not recognized when inherited
Added: Deprecated @Configuration and introduced @BeforeSuite/Test/Class/TestMethod
Added: Deprecated @ExpectedExceptions and moved it into @Test
Added: expectedExceptions to @Test, deprecated @ExpectedExceptions
Added: New annotations: @BeforeSuite, @BeforeTest, etc...
Fixed: Was returning an exit code of 0 if a cyclic graph was detected
Added: Interface org.testng.ITest so that tests can declare a name
Fixed: The Text reporter was reporting the square of the actual number of methods
Fixed: Bug reported by Eran about dependencies with an afterClass method
Added: IHookCallBack now receives the ITestResult in its run() method
Added: Name of suite for command line can be set with -Dtestng.suite.name=xxx
Fixed: TestNGAntTask was hardcoding m_haltOnFSP to true
Fixed: Passing a null parameter caused an NPE in the reports
Added: "listener" to the ant task (and documentation)
Added: if patch-testng-sourcedir.properties is found in the classpath
with a property "sourcedir" containing a ; separated list of
directories, this list will override -sourcedir.
===========================================================================
4.7
Added: Maven 2 plug-in
Fixed: Message formattings in TestNG assertion utility class
Fixed: @Factory methods were counted as @Test as well
http://jira.opensymphony.com/browse/TESTNG-51
Fixed: All DataProvider parameters were shown in the HTML report
Fixed: Bug in testng-failed.xml generation
Fixed: bug when using a jar file to load the test classes
Added: alwaysRun for before @Configuration methods
http://jira.opensymphony.com/browse/TESTNG-35
Fixed: groupless @Configurations were not invoked if a method depends on a group
http://jira.opensymphony.com/browse/TESTNG-45
Added: beforeGroups/afterGroups to @Configuration
Eclipse plugin:
Added: last contextual launch is available in Eclipse launcher lists
Fixed: 3.2M5 integration (removed dependency on non-existing class)
Fixed: testng-failures.xml generation
===========================================================================
4.6
Added: Documentation contains the new reports
Added: TestNG.setUseDefaultListeners(boolean)
Added: Descriptions now appear in TextReporter (verbose>=2) and the HTML reports
Added: description attribute to @Test and @Configuration
Added: combined Reporter output in the reports
Added: methods not run in the reports
Added: org.testng.IReporter
Added: threadPoolSize to @Test
Added: Reports now show relative timings (start at 0)
Added: Reports now show different colors depending on the methods' classes
Added: Reports now show all parameters used to invoke the test method
Added: org.testng.Reporter
Added: DataProviders can accept a Method as first parameter
Fixed: Extraneous implicit inclusion of a method
Eclipse plugin:
Added: Run/Debug as TestNG test from the editor contextual menu
Fixed: TESTNG-24: 'Run as testng test' does not appear of the Test annotation does not have a group
Fixed: TESTNG-18: Eclipse plugin ignores Factory annotation
Fixed: TESTNG-21: Show differences when double clicking assertion exceptions
Added: UI allows setting orientation (even more space)
http://forums.opensymphony.com/thread.jspa?threadID=17225&messageID=33805#33805
===========================================================================
4.5
Core:
Fixed: Methods were not implicitly included, only groups
Fixed: Bug with failed parent @Configuration don't skip child @Configuration/@Test invocations
Fixed: Bug with overridding @Configuration methods (both parent and child were run)
Fixed: Bug when overriding beforeClass methods in base class (cyclic graph)
Added: Support for JAAS (see org.testng.IHookable)
Fixed: Problem with nested classes inside
Fixed: ArrayIndexOutOfBoundsException for jMock
Added: dependsOnMethods can contain methods from another class
Fixed: JUnitConverter required -restore, not any more (option is now a no-op)
Fixed: JUnit mode wasn't invoking setName() on test classes
Added: Regular expressions for classes in
Added: Distributed TestNG
Fixed: Command line parameters and testng.xml are now cumulative
Fixed: Reports now work for multiple suites
Fixed: Was ignoring abstract classes even if they have non-abstract instances
Fixed: If setUp() failed, methods were not skipped
Fixed: Was not clearly indicating when beforeSuite fails
Added: @Configuration.inheritGroups
Fixed: inconsistency between testng.xml and objects regarding method selectors
Eclipse plug-in:
New look for the progress view.
===========================================================================
4.4
Core:
Fixed: testng-failures.xml was not excluding methods from base classes
Fixed: Bug in suites of suites for JUnit mode
===========================================================================
4.3
Core:
Fixed: testng-failures.xml was not excluding methods from base classes
Fixed: Bug in suites of suites for JUnit mode
Added: Excluded groups on command line and ant task
Fixed: When including a group, implicitly include groups depended upon
Fixed: When depending on several groups, wasn't skipped if one of them failed
Fixed: Failures weren't reported accurately in the JUnitReports report
Fixed: Wasn't throwing an exception if depending on a non-existing group
===========================================================================
4.2
Core:
Fixed: wasn't excluding methods in base classes
Added: alwaysRun for tests (soft dependencies)
Fixed: Class-level enabled=false were not honored
Fixed: Bug with multiple dataproviders on same class
Fixed: Bug with dataprovider defined in the parent class
Fixed: Bug with dataprovider defined in a subclass
Fixed: Bug with dataprovider defined in an abstract class
Fixed: testng-failures generation was excluding the methods even if a failed test depended on it
===========================================================================
4.1
Core:
Added: @DataProviders can return Iterable
Fixed: Superclass test methods were not called in the presence of a class @Test
Added: Reporter class to log messages in the HTML reports
===========================================================================
4.0
Core:
Fixed: suite methods now invoked only once in a hierarchy
Added: @DataProvider and @testng.data-provider
Fixed: Interleave order now respected for before/afterClass methods
Added: Can now invoke java -jar testng-2.6.jar <...>
Added: Support for BeanShell
Added: Method Selectors (IMethodSelector)
Fixed: In the absence of dependencies, @Configuration methods respect inheritance
Fixed: Bug in multithreaded dependencies on methods
Fixed: dependsOnGroups wasn't working on regular expressions
Fixed: Bug in when directories contain spaces in their names
Fixed: Introduced a JDK5 dependency in the JDK1.4 build (getEnclosingClass())
Fixed: Output directory in ant task was not honored if it didn't exist
Fixed: Problem with timeout according to
http://forums.opensymphony.com/thread.jspa?threadID=6707
Eclipse plug-in:
Fixed: Wasn't handling linked directories correctly
Fixed: Bug in QuickFix implementation
Added: Quick Fix for JUnit conversion (Annotations and JavaDoc)
Fixed: Methods Run as TestNG test
Added: Package level Run as TestNG test
Fixed: Resources from the linked directories are using a wrong path when
passed to command line TestNG
IDEA plug-in:
Added: Support for JDK 1.4 (both projects and IDEA itself)
Fixed: Classes that contained only configuration were ignored
===========================================================================
2.5
Added: ITestListener.onTestStart(ITestResult)
Added: Support for
Added: Resource files for easier ant taskdefs
Fixed: @Configuration methods were not invoked with individual test methods
Fixed: Bug with ExpectedExceptions
Fixed: Didn't support nested factory classes
Fixed: NPE if -target is omitted with JDK 1.4
Fixed: @Configuration failures in a class would cause other classes to fail
Added: alwaysRun
Fixed: beforeTestClass/afterTestClass were broken for a pathological case
Added: @Configuration(alwaysRun)
Added: JUnitConverter task
Fixed: < and > characters in reports were not escaped
Eclipse plug-in:
Fixed: Class dialog wasn't showing @Factory classes
IDEA plug-in:
First release!
Documentation:
Added: Brand new look!!!
Added: Section on testng.xml
Fixed: Numbering of sections
===========================================================================
2.4
Changed: New package: testng.org
Fixed: Bug with @ExpectedException occuring the parallel mode
Fixed: Bug with parameters and beforeTest
Added: IInstanceInfo support
Fixed: methods were not excluded when included by groups
Fixed: testng-failures.xml is now including also the beforeSuite/afterSuite methods
Fixed: generating the testng-failures.xml is now working as expected
Fixed: Factories call all the tests even if some of them fail along the way
Fixed: Better JUnit support (wasn't creating individual instances)
Fixed: dependsOnGroups didn't work across different classes
Added: command line (and Ant) -groups option
Added: @Parameters (and made parameters attribute deprecated)
Added: Parameters for constructors
Fixed: Better interleaving of before/afterTestMethods
Fixed: Ant task
Fixed: TestNGException thrown when TestNG conditions are not fulfilled
Documentation:
- New assert classes
- New ways to launch
- JUnitConverter documentation
- new beforeSuite/afterSuite
===========================================================================
2.3
Fixed: Spaces are now legal in JavaDoc comments
Added: documentation for @Factory
Fixed: factories were called multiple times
Added: beforeSuite and afterSuite
Fixed: inheritance and scope now working properly for annotations
Fixed: dependsOnMethods wasn't working for 1.4
Added: Better stack traces
Added: Better syntax for included/excluded methods
Fixed: Better verbose support
Fixed: Various fixes for the Eclipse plug-in
Added: Can specify a class name on the command line
Fixed: Default package bug in JUnitConverter
Added: Regression tests for JUnitConverter
Added: -quiet option to JUnitConverter
===========================================================================
2.2
Fixed: Wasn't handling several testng.xml files correctly
Fixed: Renamed -src to -sourcedir
Fixed: Complains if no sourcedir is specified in 1.4
Added: In 1.4, don't require annotations="javadoc"
Fixed: If setUp fails, complain and mark test methods as skips
Fixed: Dependent methods weren't working for 1.4
===========================================================================
2.1
Added: Parser can accept an InputStream for testng.xml
Fixed: expected-exceptions now fails if test passes
Fixed: reports now use the suite name in HTML
Added: invocationCount and successPercentage
Added: dependsOnMethods
Added: timeOut works in non-parallel mode
===========================================================================
2.0
Added: port on JDK 1.4
===========================================================================
1.3
Added: new view: classes (still experimental)
Added: timeout on methods
Added: thread-count
Added: TestNG is now multithread, see "parallel" in
===========================================================================
1.2
Added: JUnitConverter
Fixed: Bug with afterClasses (test: AfterClassCalledAtTheEnd)
===========================================================================
1.1
Added: new links for methods and groups in the HTML report
Added:
Added: to
===========================================================================
1.0
Fixed: Updated to the new DTD
Fixed: Suite table of contents displays failures first
Fixed: Bug in afterTestClass
Added: Validating testng.xml
Added: Scoped parameters
Added: testng.xml
Removed: Property quiet
Changed: Verbose is now an integer
Added: Dependent methods
===========================================================================
0.9
Added: Groups of groups
Added: Groups for Configuration methods
Added: Parameters
===========================================================================
0.2
Fixed: Merged TestMethod and TestClass into Test
Added: HTML report
Added: Regexps for groups
Fixed: Inheritance of methods
Fixed: ExpectedException is now called ExpectedExceptions