Skip to content

Configuration⚓︎

All Xray plugin options can be specified within the xray property of the configuration JSON.

Note

The following icons describe whether the individual settings apply to Xray server or Xray cloud only:

  • Setting applicable to/relevant for Xray cloud only
  • Setting applicable to/relevant for Xray server only

All settings without such icons are applicable to both versions.

authentication⚓︎

In order to upload results to Xray, QTAF needs to authenticate to it.

Tip

Have a look at this graph to quickly set up Xray and Jira authentication. 

graph LR
    A{Xray<br/>instance};
    B("xray.clientId=<i>id</i><br>xray.clientSecret=<i>secret</i><br><hr>jira.username=<i>user@company.com</i><br>jira.apiToken=<i>token</i>");
    A ---->|&nbsp cloud &nbsp| B;
    A ---->|&nbsp server &nbsp| D;
    D("xray.bearerToken=<i>token</i><br><hr>jira.apiToken=<i>token</i>");
    classDef code-node text-align:left;
    class B,D,E code-node;

Please note that basic authentication for Xray server and Jira server is not supported by QTAF.

xray⚓︎

Xray server or Xray cloud credentials can be specified here.

Note

Consult Xray's official documentation on how to set up:

bearerToken ⚓︎

The Jira PAT of the user QTAF should use for interacting with Xray.

Environment variable
XRAY_AUTHENTICATION_XRAY_BEARERTOKEN
Type
string
Example 
1
2
3
4
5
6
7
"xray": {
  "authentication": {
    "xray": {
      "bearerToken": "BigSecretToken"
    }
  }
}

1
XRAY_AUTHENTICATION_XRAY_BEARERTOKEN="BigSecretToken"

clientId ⚓︎

The ID of the user all requests will be done in behalf of.

Environment variable
XRAY_AUTHENTICATION_XRAY_CLIENTID
Type
string
Example 
1
2
3
4
5
6
7
"xray": {
  "authentication": {
    "xray": {
      "clientId": "12345ABCDEF"
    }
  }
}

1
XRAY_AUTHENTICATION_XRAY_CLIENTID="12345ABCDEF"

clientSecret ⚓︎

The secret of the user all requests will be done in behalf of.

Environment variable
XRAY_AUTHENTICATION_XRAY_CLIENTSECRET
Type
string
Example 
1
2
3
4
5
6
7
"xray": {
  "authentication": {
    "xray": {
      "clientSecret": "xyzSuperSecretxyz"
    }
  }
}

1
XRAY_AUTHENTICATION_XRAY_CLIENTSECRET="xyzSuperSecretxyz"

jira⚓︎

Additional Jira credentials might be necessary too, depending on how you configure the plugin.

username ⚓︎

The username for Jira Cloud authentication. It is usually the Email address of the user, as described here.

Environment variable
XRAY_AUTHENTICATION_JIRA_USERNAME
Type
string
Example 
1
2
3
4
5
6
7
"xray": {
  "authentication": {
    "jira": {
      "username": "fred@example.com"
    }
  }
}

1
XRAY_AUTHENTICATION_JIRA_USERNAME="fred@example.com"

apiToken⚓︎

The API token granting access to the API of the Jira instance. More information:

Environment variable
XRAY_AUTHENTICATION_JIRA_APITOKEN
Type
string
Example 
1
2
3
4
5
6
7
"xray": {
  "authentication": {
    "jira": {
      "apiToken": "MyLittleJiraSecret"
    }
  }
}

1
XRAY_AUTHENTICATION_JIRA_APITOKEN="MyLittleJiraSecret"

enabled⚓︎

Enables or disables the plugin entirely. If set to false, the plugin will be skipped completely and no results will be uploaded.

Environment variable
XRAY_ENABLED
Type
boolean

Accepted strings (case-insensitive):

  • "true" "1" "y"

  • "false" "0" "n"

Default
false
Example 
1
2
3
"xray": {
  "enabled": "y"
}

1
XRAY_ENABLED="y"

projectKey⚓︎

The Jira key of the project to upload results to. It is used for identification of relevant test cases, meaning that only test cases with an @XrayTest annotation containing the project key will be tracked and eventually uploaded to Xray.

Environment variable
XRAY_PROJECTKEY
Type
string
Example 
1
2
3
"xray": {
  "projectKey": "PRJ"
}

1
XRAY_PROJECTKEY="PRJ"

resultsUpload⚓︎

These settings determine the plugin's behaviour regarding the results upload.

assignee⚓︎

The Jira user to assign created test executions to. The following values should be provided here:

  • The account ID, usually of the form 61f8f589e688d6007068a792. You can retrieve account IDs by exporting an issue as XML where the user is visible (reporter, assignee, ...). The XML element containing the account ID will then look something like this: 
    1
    <assignee accountid="61f8f589e688d6007068a792">John Miller</assignee>
  • The username used for login, for example miller_j. You can retrieve usernames of other users by exporting an issue as XML where the user is visible (reporter, assignee, ...). The XML element will then look something like this: 
    1
    <assignee>miller_j</assignee>

Note

If the configured assignee is null, the test execution issues will not be assigned to anyone.

Environment variable
XRAY_RESULTSUPLOAD_ASSIGNEE
Type
string
Default
null
Example 
1
2
3
4
5
"xray": {
  "resultsUpload": {
    "assignee": "61f8f589e688d6007068a792"
  }
}

1
XRAY_RESULTSUPLOAD_ASSIGNEE="61f8f589e688d6007068a792"

customStatus⚓︎

These status settings map QTAF test (execution) statuses to corresponding Jira issue or Xray test and step statuses.

step⚓︎

These settings allow mapping QTAF test steps to Xray step statuses.

Tip

If you have custom step statuses set up in Xray, you should provide their names here.

failed⚓︎

The Xray status of failed steps.

Environment variable
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_STEP_FAILED
Type
string
Default
"FAIL"
"FAILED"
Example 
1
2
3
4
5
6
7
8
9
"xray": {
  "resultsUpload": {
    "customStatus": {
      "step": {
        "failed": "ERROR"
      }
    }
  }
}

1
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_STEP_FAILED="ERROR"
passed⚓︎

The Xray status of passed steps.

Environment variable
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_STEP_PASSED
Type
string
Default
"PASS"
"PASSED"
Example 
1
2
3
4
5
6
7
8
9
"xray": {
  "resultsUpload": {
    "customStatus": {
      "step": {
        "passed": "DONE"
      }
    }
  }
}

1
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_STEP_PASSED="DONE"
pending⚓︎

The Xray status of pending steps.

Environment variable
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_STEP_PENDING
Type
string
Default
"EXECUTING"
Example 
1
2
3
4
5
6
7
8
9
"xray": {
  "resultsUpload": {
    "customStatus": {
      "step": {
        "pending": "EXECUTING"
      }
    }
  }
}

1
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_STEP_PENDING="EXECUTING"
skipped⚓︎

The Xray status of skipped steps.

Environment variable
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_STEP_SKIPPED
Type
string
Default
"TODO"
Example 
1
2
3
4
5
6
7
8
9
"xray": {
  "resultsUpload": {
    "customStatus": {
      "step": {
        "skipped": "TODO"
      }
    }
  }
}

1
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_STEP_SKIPPED="TODO"

test⚓︎

These settings allow mapping QTAF test statuses to Xray statuses, for example when setting the status of a test inside a test test execution issue.

Tip

If you have custom statuses set up in Xray, you should provide their names here.

failed⚓︎

The Xray status of failed tests.

Environment variable
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_TEST_FAILED
Type
string
Default
"FAIL"
"FAILED"
Example 
1
2
3
4
5
6
7
8
9
"xray": {
  "resultsUpload": {
    "customStatus": {
      "test": {
        "failed": "ERROR"
      }
    }
  }
}

1
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_TEST_FAILED="ERROR"
passed⚓︎

The Xray status of passed tests.

Environment variable
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_TEST_PASSED
Type
string
Default
"PASS"
"PASSED"
Example 
1
2
3
4
5
6
7
8
9
"xray": {
  "resultsUpload": {
    "customStatus": {
      "test": {
        "passed": "DONE"
      }
    }
  }
}

1
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_TEST_PASSED="DONE"
pending⚓︎

The Xray status of pending tests.

Environment variable
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_TEST_PENDING
Type
string
Default
"EXECUTING"
Example 
1
2
3
4
5
6
7
8
9
"xray": {
  "resultsUpload": {
    "customStatus": {
      "test": {
        "pending": "EXECUTING"
      }
    }
  }
}

1
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_TEST_PENDING="EXECUTING"
skipped⚓︎

The Xray status of skipped tests.

Environment variable
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_TEST_SKIPPED
Type
string
Default
"TODO"
Example 
1
2
3
4
5
6
7
8
9
"xray": {
  "resultsUpload": {
    "customStatus": {
      "test": {
        "skipped": "TODO"
      }
    }
  }
}

1
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_TEST_SKIPPED="TODO"

testExecutionIssue⚓︎

The Jira status of test execution issues created during upload.

failed⚓︎

The Jira status to transition test execution issues to if tests failed. If the status is null, the issue will have the default issue status of issues created in the project.

Environment variable
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_TESTEXECUTIONISSUE_FAILED
Type
string
Default
null
Example 
1
2
3
4
5
6
7
8
9
"xray": {
  "resultsUpload": {
    "customStatus": {
      "testExecutionIssue": {
        "failed": "Review"
      }
    }
  }
}

1
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_TESTEXECUTIONISSUE_FAILED="Review"
passed⚓︎

The Jira status to transition test execution issues to if all tests passed. If the status is null, the issue will have the default issue status of issues created in the project.

Environment variable
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_TESTEXECUTIONISSUE_PASSED
Type
string
Default
null
Example 
1
2
3
4
5
6
7
8
9
"xray": {
  "resultsUpload": {
    "customStatus": {
      "testExecutionIssue": {
        "passed": "Done"
      }
    }
  }
}

1
XRAY_RESULTSUPLOAD_CUSTOMSTATUS_TESTEXECUTIONISSUE_PASSED="Done"

environments⚓︎

QTAF can add test environments to created test execution issues. The following settings can be used to control the way QTAF assigns test environments.

Info

For more information about test environments, please see:

driver⚓︎

Toggles whether QTAF should add the name of the Selenium driver used to execute the tests as test environment.

Environment variable
XRAY_RESULTSUPLOAD_ENVIRONMENTS_DRIVER
Type
boolean

Accepted strings (case-insensitive):

  • "true" "1" "y"

  • "false" "0" "n"

Default
true
Example 
1
2
3
4
5
6
7
"xray": {
  "resultsUpload": {
    "environments": {
      "driver": false
    }
  }
}

1
XRAY_RESULTSUPLOAD_ENVIRONMENTS_DRIVER="false"

enabled⚓︎

Toggles whether QTAF should include test environments during test execution issue creation. Setting this option to false will result in no test environments being added to the test execution issue.

Environment variable
XRAY_RESULTSUPLOAD_ENVIRONMENTS_ENABLED
Type
boolean

Accepted strings (case-insensitive):

  • "true" "1" "y"

  • "false" "0" "n"

Default
true
Example 
1
2
3
4
5
6
7
"xray": {
  "resultsUpload": {
    "environments": {
      "enabled": false
    }
  }
}

1
XRAY_RESULTSUPLOAD_ENVIRONMENTS_ENABLED="n"

os⚓︎

Toggles whether QTAF should add the name of the operating system the tests were executed on as test environment.

Environment variable
XRAY_RESULTSUPLOAD_ENVIRONMENTS_OS
Type
boolean

Accepted strings (case-insensitive):

  • "true" "1" "y"

  • "false" "0" "n"

Default
true
Example 
1
2
3
4
5
6
7
"xray": {
  "resultsUpload": {
    "environments": {
      "os": false
    }
  }
}

1
XRAY_RESULTSUPLOAD_ENVIRONMENTS_OS="0"

scenarioImageEvidence⚓︎

Toggles whether QTAF should attach screenshot evidence to the test execution issue. Setting this to true will result in QTAF adding all screenshots it takes during a test to the test case inside the test execution.

Info

For more information about test execution evidence, please see:

Environment variable
XRAY_RESULTSUPLOAD_SCENARIOIMAGEEVIDENCE
Type
boolean

Accepted strings (case-insensitive):

  • "true" "1" "y"

  • "false" "0" "n"

Default
false
Example 
1
2
3
4
5
"xray": {
  "resultsUpload": {
    "scenarioImageEvidence": true
  }
}

1
XRAY_RESULTSUPLOAD_SCENARIOIMAGEEVIDENCE="y"

scenarioReportEvidence⚓︎

Toggles whether QTAF should attach the HTML report it generates as evidence to the test execution issue.

Info

For more information about test execution evidence, please see:

Environment variable
XRAY_RESULTSUPLOAD_SCENARIOREPORTEVIDENCE
Type
boolean

Accepted strings (case-insensitive):

  • "true" "1" "y"

  • "false" "0" "n"

Default
false
Example 
1
2
3
4
5
"xray": {
  "resultsUpload": {
    "scenarioReportEvidence": true
  }
}

1
XRAY_RESULTSUPLOAD_SCENARIOREPORTEVIDENCE="1"

testPlanKey⚓︎

Test executions can automatically be assigned to an existing test plan. QTAF uses this setting to retrieve such configured test plans. When set to null, QTAF will not add created test execution issues to any test plan.

Environment variable
XRAY_RESULTSUPLOAD_TESTPLANKEY
Type
string
Default
null
Example 
1
2
3
4
5
"xray": {
  "resultsUpload": {
    "testPlanKey": "PRJ-123"
  }
}

1
XRAY_RESULTSUPLOAD_TESTPLANKEY="PRJ-123"

tests⚓︎

The settings below allow configuring the way test cases themselves are updated during test results upload.

iterations⚓︎

When uploading data-driven test results, QTAF will automatically use Xray's iterations in test executions. Settings related to these iterations are listed here.

parameters⚓︎

Xray can add parameters to data-driven test iteration results, which provide information about the names and values of the concrete test case parameters. Parameter settings are listed here.

Info

For more information about iteration parameters, please see:

maxLength⚓︎

Xray enforces a hard limit on the length of both parameter names and values, which is different for Xray cloud and Xray server and from version to version. It is recommended to provide reasonable values like 30 here for both name and value to keep things readable and to prevent Xray from rejecting test result uploads altogether. QTAF will automatically truncate parameter names and values of the iterations to the numbers provided here.

Warning

If set to null, QTAF will not truncate parameter names or values whatsoever, possibly resulting in failed result uploads. Make sure to manually identify the limit your Xray instance is enforcing through the UI!

Environment variables
XRAY_RESULTSUPLOAD_TESTS_ITERATIONS_PARAMETERS_MAXLENGTH_NAME
XRAY_RESULTSUPLOAD_TESTS_ITERATIONS_PARAMETERS_MAXLENGTH_VALUE
Type
number
Default
null
Example 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
"xray": {
  "resultsUpload": {
    "tests": {
      "iterations": {
        "parameters": {
          "maxLength": {
            "name": 42,
            "value": 50
          }
        }
      }
    }
  }
}

1
2
XRAY_RESULTSUPLOAD_TESTS_ITERATIONS_PARAMETERS_MAXLENGTH_NAME=42
XRAY_RESULTSUPLOAD_TESTS_ITERATIONS_PARAMETERS_MAXLENGTH_VALUE=50

info⚓︎

Settings regarding general test case information.

keepJiraSummary⚓︎

When uploading test results, QTAF can automatically rename existing test issues to the test case's name defined within QTAF. This setting toggles whether this is allowed to happen or whether the test case's existing summaries should be kept.

Environment variable
XRAY_RESULTSUPLOAD_TESTS_INFO_KEEPJIRASUMMARY
Type
boolean

Accepted strings (case-insensitive):

  • "true" "1" "y"

  • "false" "0" "n"

Default
false
Example 
1
2
3
4
5
6
7
8
9
"xray": {
  "resultsUpload": {
    "tests": {
      "info": {
        "keepJiraSummary": true
      }
    }
  }
}

1
XRAY_RESULTSUPLOAD_TESTS_INFO_KEEPJIRASUMMARY="y"
steps⚓︎

When uploading test results, QTAF can automatically update or change the steps defined within existing test issues with the steps it executed itself. These settings control how and whether this is allowed to happen.

update⚓︎

Toggles whether QTAF is allowed to update existing test issue steps in Xray. Setting this to true will result in QTAF replacing all existing steps with the ones it executed itself. To not modify any existing steps, set this to false.

Environment variable
XRAY_RESULTSUPLOAD_TESTS_INFO_STEPS_UPDATE
Type
boolean

Accepted strings (case-insensitive):

  • "true" "1" "y"

  • "false" "0" "n"

Default
false
Example 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
"xray": {
  "resultsUpload": {
    "tests": {
      "info": {
        "steps": {
          "update": true
        }
      }
    }
  }
}

1
XRAY_RESULTSUPLOAD_TESTS_INFO_STEPS_UPDATE="true"
merge⚓︎

Toggles whether QTAF should merge all steps it executed into a single step before updating test case steps in Xray. The merged step will then contain a list of steps, possibly for all test case iterations if a test was executed more than once:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
# Step 1 (the only remaining step)
  == ITERATION 1 username="denise" password="xyz12" address="main avenue"
    1. Click register
    2. Enter username
    3. Enter password
    4. Enter address
    5. Click submit
  == ITERATION 2 username="george" password="12345" address=""
    1. Click register
    2. Enter username
    3. Enter password
    4. Click submit
In situations like above where individual test runs might not always consist of exactly the same steps (for example when skipping empty form fields), Xray would not be able to properly display the test results inside the test execution issue without the merged steps:

Steps defined in test case issue Steps reported in test execution (iteration 1) Steps reported in test execution (iteration 2) Problem
  1. Click register
  2. Enter username
  3. Enter password
  4. Click submit
  1. Click register
  2. Enter username
  3. Enter password
  4. Enter address
  1. Click register
  2. Enter username
  3. Enter password
  4. Click submit
 Fifth step truncated in iteration 1 by Xray.
  1. Click register
  2. Enter username
  3. Enter password
  4. Enter address
  5. Click submit
  1. Click register
  2. Enter username
  3. Enter password
  4. Enter address
  5. Click submit
  1. Click register
  2. Enter username
  3. Enter password
  4. Enter address
  5. Click submit
 Mismatched steps in interation 2: the address was never actually entered and Xray marked the fifth step as TODO because only four steps were actually executed in iteration 2.

Merging the iterations' steps can therefore help for data-driven testing.

Environment variable
XRAY_RESULTSUPLOAD_TESTS_INFO_STEPS_MERGE
Type
boolean

Accepted strings (case-insensitive):

  • "true" "1" "y"

  • "false" "0" "n"

Default
false
Example 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
"xray": {
  "resultsUpload": {
    "tests": {
      "info": {
        "steps": {
          "merge": true
        }
      }
    }
  }
}

1
XRAY_RESULTSUPLOAD_TESTS_INFO_STEPS_MERGE="true"

service⚓︎

Defines whether the targeted Xray instance is an Xray cloud or Xray server instance. QTAF requires this setting because Xray's APIs need to be addressed slightly differently.

Environment variable
XRAY_SERVICE
Type
"cloud" or "server"
Example 
1
2
3
"xray": {
  "service": "server"
}

1
XRAY_SERVICE="server"

url⚓︎

To properly connect to Xray, QTAF needs to know where the Xray and Jira instances can be found.

jira⚓︎

Defines the base URL of the Jira instance. For Jira cloud, it is usually of the form https://your-domain.atlassian.net (without the /jira part, see here). For Jira server, you can have a look here to determine your base URL.

Environment variable
XRAY_URL_JIRA
Type
string
Example 
1
2
3
4
5
"xray": {
  "url": {
    "jira": "https://example.org/development/jira"
  }
}

1
XRAY_URL_JIRA="https://example.org/development/jira"

xray ⚓︎

Defines the base URL of the Xray instance.

Environment variable
XRAY_URL_XRAY
Type
string
Example 
1
2
3
4
5
"xray": {
  "url": {
    "xray": "https://xray.jira.company.com"
  }
}

1
XRAY_URL_XRAY="https://xray.jira.company.com"