I have 100 test cases, if 20% or first 20 test cases all fail how can I stop the execution ? Already have testng ITestResult where should I break the build?
@Test(retryAnalyzer = ReTryFail.class, dataProvider = "SanityTCTest", dataProviderClass = utility.Xlsdataprovider.class, groups = "Dashboard", alwaysRun = true) public void Sanity_TC001(LinkedHashMap<String, String> data) throws InterruptedException, SQLException { Some code } @Test(retryAnalyzer = ReTryFail.class, dataProvider = "SanityTCTest", dataProviderClass = utility.Xlsdataprovider.class, groups = "Dashboard", alwaysRun = true) public void Sanity_TC002(LinkedHashMap<String, String> data) throws InterruptedException, SQLException { Some code } @Test(retryAnalyzer = ReTryFail.class, dataProvider = "SanityTCTest", dataProviderClass = utility.Xlsdataprovider.class, groups = "Dashboard", alwaysRun = true) public void Sanity_TC003(LinkedHashMap<String, String> data) throws InterruptedException, SQLException { Some code } ///////////////////////////////
Where can I break this suite if Result, “FAIL is over 20 ? DO i require to create new class or can I add in same below?
@AfterMethod(alwaysRun = true) public void reporterDataResults(ITestResult Result) throws InterruptedException { boolean flag = false; Testfail = TestResultStatus.Testfail; /*System.out.println("test fail flag in AfterMethod: " + Testfail); */ if (Result.getStatus() == ITestResult.SKIP) { Resmark.put(Result.getName(), ""); captureScreenShot(Result, "SKIP", getDriver()); Reporter.log(Result.getName() + " is SKIPPED"); Add_Log.info(Result.getName() + " is SKIPPED"); TestResultTL.put(Result.getName(), "SKIP"); if (!(getDriver() == null)) { closeWebBrowser(); } } else if (Result.getStatus() == ITestResult.FAILURE) { Collection<String> values = TestResultStatus.mpaskeysnew.get(Result.getName()); String resultout = String.join(" | ", values); System.out.println(resultout); Resmark.put(Result.getName(), resultout); captureScreenShot(Result, "FAIL", getDriver()); Reporter.log(Result.getName() + " is FAIL"); Add_Log.info(Result.getName() + " is FAIL"); if (!(getDriver() == null)) { closeWebBrowser(); } TestResultTL.put(Result.getName(), "FAIL"); } else { captureScreenShot(Result, "PASS", getDriver()); Resmark.put(Result.getName(), ""); Reporter.log(Result.getName() + " is PASS"); Add_Log.info(Result.getName() + " is PASS"); if (!(getDriver() == null)) { closeWebBrowser(); } TestResultTL.put(Result.getName(), "PASS"); } Testskip = false; TestResultStatus.Testfail = false; }
Advertisement
Answer
You can implement ISuiteListener
and in onFinish
method you’ll have access to ISuite
and ISuiteResult
Then you can do
public void onFinish(ISuite suite) { final Map<java.lang.String,ISuiteResult> res = suite.getResults(); for (ISuiteResult r : res.values()) { context = r.getTestContext() ; failedTestCases =context.getFailedTests().size(); } }
size()
will give you number of failed test for that suite. Once you know that number you can implement to stop execution using strategies in this
If your test cases are in different suites then in each call to onFinish
method you can count number of failed test cases per suite and based on that stop execution.
One other alternative is to implement ITestListener
. In onTestFailure
method you have access to ITestResult
You can count how many times onTestFailure
method is called and based on that stop execution. I think implementing ITestListener
is more suitable and easy in your case.
Here , I edited to explain how you’d implement listener
import org.testng.ISuiteListener; public class listener implements Itestlistener { public int i = 0; public void onTestFailure(ITestResult result) { result.getName(); i++; //your break logic goes here if (i ==20){ // do something or call some function to stop execution } } }
You can read more about testng listeners here .
For your EDIT above (if you want to go that way). Though I still think you should implement listener, which is more cleaner. It will be called only when the test fails.
But do the same thing as I did in onTestFailure
method , add a counter and increase it in else if
.
public int i = 0; //do this in your class
then in your method
else if (Result.getStatus() == ITestResult.FAILURE) { i++; //increase counter here Collection<String> values = TestResultStatus.mpaskeysnew.get(Result.getName()); String resultout = String.join(" | ", values); System.out.println(resultout); Resmark.put(Result.getName(), resultout); captureScreenShot(Result, "FAIL", getDriver()); Reporter.log(Result.getName() + " is FAIL"); Add_Log.info(Result.getName() + " is FAIL"); if (!(getDriver() == null)) { closeWebBrowser(); } TestResultTL.put(Result.getName(), "FAIL"); if (i==20){ // stop execution here } }