Execution Reports

All the execution reports for a project can be found on the executions page within the app’s website. These reports are sorted based on the most recent executions.

Fig: Executions page

Fig: Executions page

Executions run with multiple datasets are marked with a filled stacked disc icon at the beginning of the row, distinguishing them from the individual test executions.

Clicking on an execution’s name opens the associated report. If it is an execution run with multiple datasets, it opens the consolidated report, otherwise, it opens the single dataset report as shown below:

Fig: Viewing execution report

Fig: Viewing execution report

To download the execution report, simply click on the download icon located in the “Report” column.

Let’s delve into the details of both the execution type reports below.

Types of Execution Reports

Single Dataset report

The Single Dataset report that is generated contains the execution details such as Test Name, Result, Started At time, Dataset information, Environment details and more. It also includes details of each step - step name, input & output parameters, component name, step result, log messages and so on.

Displayed below is a sample report for such an execution:

Fig: Single Dataset Report

Fig: Single Dataset Report

For this report, Detailed and Summary report PDFs are available for download. Simply click the download button at the top right and choose the report of your choice.

Detailed Report PDF

The Detailed report PDF includes execution details such as Test Name, Result, Started At time, Dataset information, environment details and more. It also comprises detailed step information, encompassing step names, input and output parameters, log messages, and additional details.

Following is the link to the sample PDF.

Sample Detailed Report PDF

Summary Report PDF

The summary report PDF contains execution details such as Test Name, Result, Started At time, Dataset information, environment details and more. It also contains the step wise summary of the test steps which includes the step name, component name, total duration and result.

Following is the link to the sample PDF.

Sample Summary Report PDF

Consolidated report for Multiple Datasets

The consolidated report that is generated for the test run with multiple datasets, provides a summarized result distribution chart along with execution details of each dataset. This report offers a comprehensive overview of the test’s performance across different datasets.

The following is a sample consolidated report for an execution:

Fig: Consolidated report

Fig: Consolidated report

The report contains a pie chart displaying the Passed/Failed/Cancelled/Not Run result distribution of the datasets executed. It also displays other execution details like Environment, Started At time, Total Duration, User email and so on. The consolidated summary report PDF can be downloaded by clicking on the download button besides the execution result at the top right.

Additionally, a table displays individual dataset execution details. By clicking on the dataset name, you can view the specific report for that dataset’s execution. For each dataset, both a summary and a detailed report are available to download via their respective download icons.

Consolidated Summary Report PDF

The summary report PDF contains the result distribution pie chart and information of the datasets and a step wise summary of each dataset executed. It offers the advantage of easy sharing via email or as an attachment, facilitating seamless collaboration and comprehensive test result discussions among team members.

Below is the sample PDF.

Sample Consolidated Report PDF

Understanding Step Numbering

In useMango™, each step of a test execution is assigned a specific number to provide clarity and structure to the testing process. This numbering helps track and identify the sequence of actions within the test.

Step Numbering of Steps

Each step in the test is numbered sequentially based on the order of execution. For example:

  1. Open browser
  2. Click Login
Fig: Step Numbering In Reports

Fig: Step Numbering In Reports

When you click the dropdown for any step, you can view additional details like identifiers, execution time, assigned values, step name, and log messages for further insights.

Step Numbering for Loop Steps

When a test involves a loop, the numbering format adapts to reflect the loop structure. A loop icon will appear next to the step number to indicate it’s part of a loop.

Loop Step Numbering

The loop step is numbered based on its position in the test.

For example: 2: Login (indicates the loop is the second step in test)

Fig: Loop Step Numbering

Fig: Loop Step Numbering

Inside the loop, each iteration is numbered with the letter I indicating “iteration,” followed by the iteration number.

For example: 2I1

2: The loop step number.

I: Indicates it’s part of an iteration.

1: The first iteration of the loop.

Step Numbering Within Iterations

Steps executed within each iteration are numbered based on their position in the overall test sequence.

Fig: Step Numbering Within Iterations

Fig: Step Numbering Within Iterations

For example: 2I1-3

2: The loop step number.

I: Indicates it’s part of an iteration.

1: The iteration number.

3: The step’s position in the overall test sequence.

Step Numbering of Composite Components

When a composite component is executed as part of a test, the steps within it are numbered relative to the component’s position in the overall test sequence.

For example: 2: Composite Component for Login (indicates the composite component is the second step in test)

Fig: Step Numbering of Composite Component

Fig: Step Numbering of Composite Component

Step Numbering Within Composite Components

Each step inside the composite component is further numbered sequentially.

For example: 2-1

2: The composite component step number.

1: The first step inside composite component.