Skip to content

QTAF TestRail Plugin⚓︎

One possibility to export and document test results is TestRail. In this article we show you how to integrate QTAF in TestRail, how to link your test cases written in Java with the test cases defined in TestRail and how QTAF can automatically save the test results of your test runs in Xray.

Configuration⚓︎

TestRail's API is employed to integrate automated tests and submit the results of the implemented test cases. This API operates on an HTTP basis and can be leveraged using any framework or programming language. The TestRail QTAF Plugin is implemented in Java.

In this article, we'll guide you on creating test runs in TestRail, linking your test runs to the test cases specified in TestRail, and how QTAF can automatically record the test results of your runs in TestRail.

More Details of using the TestRail API see TestRail API Manual

Creating Test Runs in TestRail⚓︎

Assuming you've already outlined some test cases in TestRail (note: creating a test case in TestRail isn't covered in this tutorial. For instructions on that, refer to the following video: TestRail's Test Cases), to craft a test run, navigate to the "TEST RUNS & RESULTS" tab, select "Add Test Run", and pick the corresponding test suite.

create test runs

After creating the test run, you should append the test cases to it. There are three methods to achieve this:

add test cases to test run

QTAF and TestRail Integration⚓︎

This section shows how the results, attachments and report could be sent to TestRail.

This segment illustrates how to transmit results, attachments, and reports to TestRail.

Firstly, besides qtaf-core, you need to integrate the qtaf-testrail-plugin dependency into your project. Ensure you have the most recent version numbers of these modules, which can be verified at the Maven Repository

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
<dependencies>
    <dependency>
        <groupId>de.qytera</groupId>
        <artifactId>qtaf-core</artifactId>
        <version>${qtafVersion}</version>
        <scope>compile</scope>
    </dependency>
    <dependency>
        <groupId>de.qytera</groupId>
        <artifactId>qtaf-testrail-plugin</artifactId>
        <version>${qtafVersion}</version>
        <scope>compile</scope>
    </dependency>
</dependencies>

Next, you need to store TestRail credentials within the qtaf.json file. Insert the following section into the file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
{
  ...
  "testrail": {
    "enabled": true,
    "url": "xxxxxxxx",
    "projectId": "xxxxx",
    "authentication": {
      "clientId": "xxx@xxx.xxx",
      "clientSecret": "xxxxxxxxxx"
    }
  }
  ...
}

Following describes the keys and values of the TestRail configuration section: | Key | Description | |--------------|---------------------------------------------------------------------------------------------------------| | enabled | Remember to activate the TestRail Plugin by toggling the enabled flag. | | url | The HTTP Url to the TestRail server instance. | | projectId | The TestRail internal projectID, if it is necessary. You also can define it in the TestRail annotation. | | clientId | The login credential of the TestRail API. | | clientSecret | The password credential of the TestRail API. |

After setting up TestRail and saving the credentials, utilize the TestRail annotation to mark the test case. Below is an illustrative example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
import de.qytera.qtaf.testrail.annotations.TestRail;

@TestFeature(
        name = "DoGoogleSearch",
        description = "Perform a search for text 'Hello World'"
)
public class DoGoogleSearch extends TestContext {
    @Test(
            testName = "TestGoogleSearch",
            description = "Type 'Hello World' and click search button",
            groups = {"Group 1", "Group 2"}
    )
    @TestRail(caseId = {"11111"}, runId = "1111")
    public void TestCaseGoogleSearch()
    {
        // test logic here
    }
}

To map a test class to a test in TestRail, apply the @TestRail annotation to the appropriate Java method. The caseId and runId attributes of the annotation should match the test case ID and test run ID in TestRail.

Images are provided to guide you in locating the test case ID and test run ID.

  • To get the test run ID, click on "TEST RUNS & RESULTS" tab, choose your test run, top in the left corner you can find the ID, it is framed with black in the following image:

get test run id

  • To get the test case ID, click on "TEST SUITES & CASES" tab, choose your test suite, then you test case, top in the left corner you can find the ID, it is framed with black in the following image:

get test case id

In order to also document the test steps, methods must be called in the test that are provided with the annotation @Step of the QTAF framework. In this example, we assume that there are two methods that are defined in a separate class called GoogleFunctionsPage.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
public class GoogleFunctions extends TestContext
{
    @Step(
            name = "Search value",
            description = "Search for a value"
    )
    public void doSearch(String searchValue) {
        driver.findElement(By.xpath(UsedObject.SearchInputXPath())).sendKeys(searchValue);
        driver.findElement(By.name(UsedObject.GoogleSearchButtonName())).click();
    }

    @Step(
            name = "Go to Google home page",
            description = "Navigate to the Google home page"
    )
    public void goToGoogleHome() {
        driver.get(UsedObject.GoogleHome());
    }
}

We then call these methods in the test case.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
@TestFeature(
        name = "DoGoogleSearch",
        description = "Perform a search for text 'Hello World'"
)
public class DoGoogleSearch extends TestContext
{
    @Test(
            testName = "TestGoogleSearch",
            description = "Type 'Hello World' and click search button",
            groups = {"Group 1", "Group 2"}
    )
    @TestRail(caseId = {"11111"}, runId = "1111")
    public void TestCaseGoogleSearch()
    {
         // 1. Open Google
        googleFunctionPage.goToGoogleHome();

        // 2. Execute Search
        googleFunctionPage.doSearch("Hello World");
    }
}

QTAF registers the call of these methods and logs the call in the background. Here, no IDs need to be defined for the test steps. QTAF assigns the called methods in the order of the steps defined in TestRail.

Now execute your test cases as usual. After the execution of your test cases is finished, QTAF will show you the following statement in the console:

1
INFO  de.qytera.qtaf.core - Results are uploaded to testRail

This means that the upload to TestRail was successful.