Release August 2020

Version 5.2.129

In this update

Data-driven testing

Until now, feeding data into useMango tests when they run has been possible but not very straightforward (you really needed to be using ALM for test management and you had to manually attach a data file to a test set).

For this release we’ve made it easy and very practical to set up alternative scenarios for a test. Typical situations in which you may need to use scenarios are:

  1. you want to test your system more thoroughly by running one test with lots of different input values and validate that the system performs correctly for each different input value;

  2. you want to test your system running in different environments, using the same test process but having to use alternative parameters to target each different environment.

The changes to useMango that enable test scenarios are:

  • The parameters panel in the Test Designer has a new button that opens up the test page for Scenario editing.

Test Designer parameters panel with Scenarios button

  • The new test page on the app website displays summary details of a test and contains a Scenarios section where data for the test’s scenarios can be managed.

Each scenario is named and provides space to enter a value for each parameter in the test.

As well as adding data for individual scenarios, you may upload a CSV file containing data for many scenarios together. The CSV file should use a header row containing the name of the test parameter for each included column.

  • When running a test which had input parameters from the application, you can now pick any of the defined scenarios to run and useMango will use the data provided for that scenario as inputs to the test when executed.

  • useMango jobs in Jenkins run all scenarios for a test

  • The test report displays information on the scenario that was executed

Recent execution history

The test library in useMango now displays the results from up to 5 most recent executions.

Recent test runs indicated in the Test Library

This enables you to see at a glance which tests have passed or failed and, by clicking the on the run indicator, lets you open up the test report directly in your browser.

Website test pages

The application website has new pages for listing and viewing tests within a project.

  • When you log into your account in the web application you will now see the list of the projects in your account.
  • Select a project and then click the Tests menu item to see the list of tests within the selected project.
  • Click on a test to open the test details page. This page shows summary information about the test and gives access to the most recent execution results for the test. The test page also lets you view and edit the any scenario data that may exist if the test has input parameters.

New report design

When looking at execution reports you will now see a revised report design. This new design has some new content and an improved layout to make it easier to review the test’s activity during execution. The new report:

  • shows information related to the scenario that was run. The scenario name and the data passed to the test are displayed
  • if you have included a test step that uses a composite component then the report is now simplified to show just the summary outcome of the composite component step. The individual steps that ran within the composite component can still be seen by clicking the + icon, which expands the display to reveal all of the child steps
  • step details are now available in-line with each step rather than being listed in a separate section at the bottom of the report. This makes it easier to review the details of a step at the same location within the report.

Notable bug fixes

  • Motor folder temporary files are not cleared
  • Obsolete tests are included in Test Library list when showing all tests
  • Motor incorrectly escapes backslash () in Ruby scripts
  • IncrementVariable component increments the variable twice
  • Login failure message is uninformative
  • Screenshots don’t always appear in the execution report