Skip to content

TestNG Framework

Jump to bottom
Roman edited this page Feb 11, 2020 · 8 revisions

WARNING: We no longer use testNG, see jUnit which is very similiar

https://testng.org/doc/documentation-main.html (https://www.tutorialspoint.com/testng/index.htm)

JUnit integration

  • JUnit tests can easily been run in the testNG environment
  • just import the needed function into the test class and specify the test as junit in the testng.xml file
import static org.testng.AssertJUnit.assertEquals;
...
<test name = "JUnitTests" junit="true">

Annotations

@BeforeSuite The annotated method will be run only once before all tests in this suite have run.

@AfterSuited The annotated method will be run only once after all tests in this suite have run.

@BeforeClass The annotated method will be run only once before the first test method in the current class is invoked.

@AfterClass The annotated method will be run only once after all the test methods in the current class have run.

@BeforeTest The annotated method will be run before any test method belonging to the classes inside the tag is run.

@AfterTest The annotated method will be run after all the test methods belonging to the classes inside the tag have run.

@BeforeGroups The list of groups that this configuration method will run before. This method is guaranteed to run shortly before the first test method that belongs to any of these groups is invoked.

@AfterGroups The list of groups that this configuration method will run after. This method is guaranteed to run shortly after the last test method that belongs to any of these groups is invoked.

@BeforeMethod The annotated method will be run before each test method.

@AfterMethod The annotated method will be run after each test method.

Parameters for all above

  • alwaysRun: For before methods (beforeSuite, beforeTest, beforeTestClass and beforeTestMethod, but not beforeGroups): If set to true, this configuration method will be run regardless of what groups it belongs to. For after methods (afterSuite, afterClass, ...): If set to true, this configuration method will be run even if one or more methods invoked previously failed or was skipped.
  • dependsOnGroups: The list of groups this method depends on.
  • dependsOnMethods: The list of methods this method depends on.
  • enabled: Whether methods on this class/method are enabled.
  • groups: The list of groups this class/method belongs to.
  • inheritGroups: If true, this method will belong to groups specified in the @Test annotation at the class level.
  • onlyForGroups: Only for @BeforeMethod and @AfterMethod. If specified, then this setup/teardown method will only be invoked if the corresponding test method belongs to one of the listed groups.

@DataProvider Marks a method as supplying data for a test method. The annotated method must return an Object[ ][ ], where each Object[ ] can be assigned the parameter list of the test method. The @Test method that wants to receive data from this DataProvider needs to use a dataProvider name equals to the name of this annotation.

Parameters:

  • name: The name of this data provider. If it's not supplied, the name of this data provider will automatically be set to the name of the method.
  • parallel: If set to true, tests generated using this data provider are run in parallel. Default value is false.

@Factory Marks a method as a factory that returns objects that will be used by TestNG as Test classes. The method must return Object[ ].

@Listeners Defines listeners on a test class.

Parameters:

  • value: An array of classes that extend org.testng.ITestNGListener.

@Parameters Describes how to pass parameters to a @Test method.

Parameters:

  • value: The list of variables used to fill the parameters of this method.

@Test Marks a class or a method as a part of the test.

Parameters:

  • alwaysRun: If set to true, this test method will always be run even if it depends on a method that failed.
  • dataProvider: The name of the data provider for this test method.
  • dataProviderClass: The class where to look for the data provider. If not specified, the data provider will be looked on the class of the current test method or one of its base classes. If this attribute is specified, the data provider method needs to be static on the specified class.
  • dependsOnGroups: The list of groups this method depends on.
  • dependsOnMethods: The list of methods this method depends on.
  • description: The description for this method.
  • enabled: Whether methods on this class/method are enabled.
  • expectedExceptions: The list of exceptions that a test method is expected to throw. If no exception or a different than one on this list is thrown, this test will be marked a failure.
  • groups: The list of groups this class/method belongs to. invocationCount The number of times this method should be invoked. invocationTimeOut: The maximum number of milliseconds this test should take for the cumulated time of all the invocationcounts. This attribute will be ignored if invocationCount is not specified.
  • priority: The priority for this test method. Lower priorities will be scheduled first.
  • successPercentage: The percentage of success expected from this method singleThreaded If set to true, all the methods on this test class are guaranteed to run in the same thread, even if the tests are currently being run with parallel="methods". This attribute can only be used at the class level and it will be ignored if used at the method level. Note: this attribute used to be called sequential (now deprecated).
  • timeOut: The maximum number of milliseconds this test should take.
  • threadPoolSize: The size of the thread pool for this method. The method will be invoked from multiple threads as specified by invocationCount. Note: this attribute is ignored if invocationCount is not specified

Benefits of Using TestNG Annotations

  • TestNG identifies the methods it is interested in, by looking up annotations. Hence, method names are not restricted to any pattern or format.
  • We can pass additional parameters to annotations.
  • Annotations are strongly typed, so the compiler will flag any mistakes right away.
  • Test classes no longer need to extend anything (such as TestCase, for JUnit 3).

For further needed knowledge please read in the documentation specified on the top.

Execution

testng.xml (or similar other file name) needs to be specified for a test suite and can be executed separately with this file or all test-suites specified in the maven-sunfire-plugin via mvn test and automatically with TravisCI

Example form our test testing:

  • test class:
public class ExampleTests {
  @Test(groups = {"active"})
  public void printSomethingUnnecessary() {
    System.out.println("Test method one");
  }

  @Test(groups = {"active"})
  public void printSomethingUnnecessary1() {
    Assert.assertEquals(1, 1);
  }

  @Test(groups = {"active"})
  public void printSomethingUnnecessary2() {
    Assert.assertEquals(1, 1);
  }

  @Test(groups = {"active"})
  public void printSomethingUnnecessary3() {
    Assert.assertEquals(1, 1);
  }

  // Failing test, is representable as a bad example and TravisCI will fail the build

  @Test(groups = {"broken"})
  public void testMethodTwo() {
    Assert.assertEquals(1, 2);
  }

  @Test(groups = {"broken"})
  public void printSomethingUnnecessaryToo() {
    System.out.println("Test method two");
  }
}
  • testng.xml
<?xml version = "1.0" encoding = "UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="SuiteTest" verbose="1">
    <test name="Tests">
       // groups can be defined as Parameter in @Test method (@Test(groups = { "active" })) and clustered in the define tag to a "super"-group
        <groups>
            <define name="include-group">
                <include name="active" />
            </define>
            <define name="exclude-group">
                <include name="broken" />
            </define>
            // they can be either executed of not while running the tests
            <run>
                <include name="include-group" />
                <exclude name="exclude-group" />
            </run>
        </groups>
        <classes>
            //specified test class
            <class name="org.bptlab.cepta.ExampleTests" />
        </classes>
    </test>
</suite>

With maven

Surefire-Plugin executes the suite tests saved in the XML file

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-surefire-plugin</artifactId>
	<version>3.0.0-M3</version>
	<configuration>
		<!-- Suite testng xml file to consider for test execution -->
		<suiteXmlFiles>
			<suiteXmlFile>src/test/testng.xml</suiteXmlFile>
		</suiteXmlFiles>
	</configuration>
</plugin>

Listener

... is watching the execution of the tests and prints in the console according to the result of the test

public class DotTestListener extends TestListenerAdapter {
  private int m_count = 0;

  @Override
  public void onTestFailure(ITestResult tr) {
    log("F");
  }

  @Override
  public void onTestSkipped(ITestResult tr) {
    log("S");
  }

  @Override
  public void onTestSuccess(ITestResult tr) {
    log("#");
  }

  @Override
  public void onFinish(ITestContext testContext) {
    // what should happen after finishing all tests
  }

  @Override
  public void onStart(ITestContext testContext) {
    // what should happen at the beginning of testing start
  }

  private void log(String string) {
    System.out.print(string);
    if (++m_count % 40 == 0) {
      System.out.println("");
    }
  }
}

The listeners has also to be specified in the configurations of our surefire-plugin

<properties>
  <property>
    <name>listener</name>
    <value>org.bptlab.cepta.DotTestListener</value>
  </property>
</properties>
Clone this wiki locally