In order to offer quality software, we have to ensure optimisation of the automatic processes and test the software thoroughly. For this, we create analyses and reports that help us to successfully identify possible errors.
Therefore the use of test management solutions and tools is crucial for the creation of testing plans and strategies for our software.
At WATA Factory, Xray provides us with the optimal tools to structure, organise and log the execution of manual and automated tests in our systems.
What is Xray for Jira?
Xray is a Jira add-on that helps us track the entire test lifecycle: Planning, design, execution and reporting From all this, we get very detailed diagrams and reports as a result. Thanks to this solution, team leads and testers can improve project management (or product development) and workflow by checking the status of requirements and the progress of test runs.
Xray’s integration with Jira allows us to index tests in real time and take control of the entire testing process to achieve full coverage and quickly identify issues. In addition, Xray supports BDD methodology and continuous integration (CI) through technologies such as Cucumber and Jenkins.
What types of issues does Xray offer us?
The integration of Xray with Jira opens up the possibility of creating new types of issues that can be used to define the key stages of the test lifecycle.
Depending on which phase of the test plan we are in, we can use the following types of issues for testing:
Test Planning
- Test Plan. Enables better management of our tests and provides us with a table of all tests added to the test plan.
Test Designing
- Precondition. Helps to define the conditions that must be met for a test to be performed.
- Test. This is the main issue to create our tests as a sequence of steps, paired with the conditions, inputs and expected outcome of the test.
- Test Set. Enables the organisation of a series of tests.
Test Execution and Reporting
- Test execution. This issue enables the monitoring of a set of tests to check whether said tests work as expected in a certain context and environment. This issue also creates, when running the tests, the test reports that show the requirements coverage of our project.
How is Xray used?
The following steps are usually followed to create a test environment for an Xray project:
1. Test Specification
Three different types of tests can be created with Xray: Manual, Generic and Cucumber.
With Manual tests in Xray, one manually defines the list of individual steps, formulates a description of the test, specifies the software version and reports the results.
The creation of automatic tests in Xray can be done with Generic and Cucumber tests.
While Generic tests allow us to create unstructured and step-less tests, Cucumber tests can be defined in a way that is understandable to both the technical team and the non-technical business side, based on Behaviour Driven Development (BDD) with the Gherkin language.
It should be noted that Generic tests can also be used for manual testing (e.g. exploratory testing).
2. Organisation of tests
The organisation of tests is an important step in managing the testing environment of our project, as the number of tests usually increases significantly. A good organisation will allow us to find each test in an efficient way.
With Xray, we organise our tests with the Issue Test Set and/or using Test Repository.
The Issue Test Set allows us to create test lists grouped by logic, which can be used as the basis for creating Test Executions or Test Plans. We can create multiple combinations of Test Sets depending on the requirements of our project’s test environment.
Using the Xray Test Repository helps us to organise our tests based on a hierarchical tree structure of folders and subfolders, which provides a better overview of our tests and their execution.
3. Planning of tests
The test planning phase is crucial for defining a good test strategy in our project based on certain aspects, such as the requirements we want to validate and how we want to validate them, the types of tests to be developed, the resource allocation or the execution plan.
For this process, we can use the Issue Test Plan in Xray, where we can select all the tests we want to place in a particular plan. In this issue we find a table that shows the tests assigned to the plan and their executions.
4. Execution of tests
Once we have defined our tests and test suites, we can execute them. Xray’s Issue Test Execution monitors and checks whether the tests selected for execution have passed or failed and validates the execution if the former is the case.
The execution result informs us of the progress of execution for each test that passed, failed, or was selected for execution. Each test has its own execution details so that progress and execution results can be easily tracked.
With manual tests, the status of the execution is defined manually in the Test Execution after the execution in the corresponding environment.
If, on the other hand, the automatic test execution is done by Continuous Integration (CI), the Test Executions are automatically generated with the test results.
5. Analysis of test runs
Xray provides an intuitive and descriptive dashboard for analysing and evaluating the test results. The Tests Executions automatically generate statistics and charts showing a detailed summary of the passed and failed tests in our project.
This allows developers and testers to closely review the execution of our tests and monitor the coverage of our test environment.
On the other hand, the Xray Dashboard is an ideal way to present the project status to our customers to promote transparent and trustful communication and collaboration.
Xray at WATA Factory
At WATA Factory, we define our testing environments around Xray for Jira tools and solutions to deliver quality software based on automated, organised as well as resilient testing strategies.