How to execute only failed TestNG tests in Eclipse

While running testNG suite, once we are done with one round of execution of tests in the test suite, It can be helpful to execute only the failed tests during the first round of execution to reduce test failures due to flakiness/synchronization issues.

Below steps shows how to achieve this in eclipse.

• Run the required test suite through eclipse as shown below.

• Once the execution is completed and there are failures in the execution, we may require to rerun the tests.

• Refresh the project. folder test-output should be displayed in the project explorer.

• File testng-failed.xml should be available in the folder.

• Run the test suite in similar manner as shown above, i.e Run As TestNG suite.

How to run multiple test suites in TestNG?

Suppose we have individual test suite for different features/modules in TestNG, we can run multiple testNG test from a master suite xml file by calling the individual tests as shown below:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="MainTestSuite">
    <suite-files>
        <suite-file path="suiteFeatureA.xml" />  
        <suite-file path="suiteFeatureB.xml" /> 
        <suite-file path="suiteFeatureC.xml" />  
        <suite-file path="suiteFeatureD.xml" />      
    </suite-files>
</suite>

This way, we can run multiple test suites from a master suite file in TestNG.

Creating TestNG.xml to run Selenium test suites.

TestNG is a testing framework used to design automated test suites for functional/unit tests in Java. Below are few reasons which makes TestNG, a strong framework for automation. 

• TestNG provides annotations to distinguish tests from methods to be called before the test and after the tests.

• TestNG provides assertions to validate test conditions.

• TestNG provides data Provider annotation to create data driven automated tests.

• TestNG test information is added in an xml file. We can easily create a suite of testNG tests in a master suite xml file and run the tests from eclipse or from Jenkins using ant files. 

• XML file denotes a test or suite of tests to be executed.

Pre-condition: Below article are useful to setup and create first TestNG test in selenium.

• How to Setup TestNG in eclipse.

• Creating first testNG test in eclipse

Focus of this article:

• Creating TestNG xml file to run test suite.

• How to run TestNG xml file in eclipse.

Let us take an example, suppose there are two classes in a Java Project.  we can create a test suite using testng xml file and run the test suite from eclipse as shown in the below image.

Creating TestNG.xml file for different scenarios:

A. Run all the tests in both classes NewTest and NewTest1. This will execute all the methods with beforetest annotation in both the classes, followed by methods with test annotation, and then methods with afterTest annotation.

 <?xml version="1.0" encoding="UTF-8"?>  
 <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">  
 <suite name="Suite1">  
  <test name="Test">  
   <classes>  
    <class name="testNG.NewTest"/>  
       <class name="testNG.NewTest1"/>  
   </classes>  
  </test>  
 </suite>  

B. Run test in class NewTest and then the methods in NewTest1 class:

 <?xml version="1.0" encoding="UTF-8"?>  
 <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">  
 <suite name="Suite1">  
  <test name="Test1">  
   <classes>  
    <class name="testNG.NewTest"/>  
         </classes>  
  </test>  
  <test name="Test2">  
   <classes>  
    <class name="testNG.NewTest1"/>  
   </classes>  
  </test>  
 </suite>  

C. Executing all tests in a package with name as TestNG(Note: TestNG is the name of package in our example, it can be any name other than TestNG also)

 <?xml version="1.0" encoding="UTF-8"?>  
 <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">  
 <suite name="Suite1">  
 <test name="Test1">  
   <packages>  
    <package name="testNG" />  
   </packages>  
  </test>  
 </suite>  

D. Executing all tests based on the group Name including and excluding specific tests

 <?xml version="1.0" encoding="UTF-8"?>  
 <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">  
 <suite name="Suite1">  
 <test name="Test1">  
  <groups>  
   <run>  
    <include name="testemail"/>  
    <exclude name="testnoemail"/>  
   </run>  
  </groups>  
   <classes>  
    <class name="testNG.NewTest"/>  
    <class name="testNG.NewTest1"/>  
   </classes>  
 </test>  
 </suite>  

E. Executing specific methods by including methods in the class:

 <?xml version="1.0" encoding="UTF-8"?>  
 <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">  
 <suite name="Suite1">  
 <test name="Test1">  
 <classes>  
   <class name="testNG.NewTest">  
    <methods>  
     <include name="emailval" />  
    </methods>  
   </class>  
   <class name="testNG.NewTest1"/>  
 </classes>  
 </test>  
 </suite>