Data-Driven Testing

Data-driven testing (DDT) lets you set test parameters outside the test script, allowing you to run the same test under different conditions. This means that you can assign different values to the same parameter, making your tests highly flexible and reusable. Parameters can be set with explicit values in a plan or by using a Dataset.

Parameters

Parameters are set at the test plan level. For example, you might have a parameter for a username that is set to Jane Doe in one plan and Admin in another. When the test runs as part of each plan, it will use the parameter values assigned to that plan, allowing you to test various scenarios without modifying the test script itself. If a test in a plan has a parameter that does not have a value, the test will not run successfully and will have a status of "Aborted."

When you run a test without a plan, if there are parameters that do not have a value at the test level, the system will prompt you to enter values.

Entering Parameter Values

When you create the test, enter the name of the parameter in braces { }. This placeholder will be replaced with the actual value of the parameter or dataset assigned to a plan. A parameter can also get a value from a variable. For details, see Variables.

For example, in the following test script, the parameters {Username} and {email} will be replaced by their respective customized values when the test runs.

Copy 
Go to https://example.com
Click "Form"
Enter {Username} in the "Your Name" field
Enter {email} in the "Email" field
Visually check this screen

If you enter confidential information such as a password or PIN, Autonomous automatically masks the value and displays it as a series of dots.

Setting Parameter Values in a Test

When you preview or run a test without a plan, if there are parameters that do not have a value, the system prompts you to enter a value. These values are automatically saved for future test runs. You can also set or modify test parameter values in the Test run settings window.

To Set or Modify Parameter Values in a Test

  1. In the Navigation bar, select Tests.

  2. Open or create a custom flow test that includes parameters.

  3. Click Run Settings.

  4. Enter a value for each parameter in the test.

    The Parameters section appears only if there are parameters in the test that require a definition.

Setting Parameter Values in a Plan

When you run a test as part of a plan, the system uses the parameter values assigned to the plan and ignores any values set at the test level.

To Set Parameter Values in a Plan

  1. In the Navigation bar, select Plans.

  2. Open a plan that includes a custom flow test with parameters.

    Tests with parameters display parameter names in the Test data column. If any parameters do not have assigned values, a icon appears. Click this icon to assign values.

  3. Above the list of tests, click Set parameter values. This option is available only if at least one test in the plan uses parameters.

  4. Enter a value for each parameter and click Set values. This window displays all parameters in the plan that do not have a value set in a dataset.

    If a parameter includes confidential data, for example if the name includes "Password," "Secret," or "PIN," it is automatically masked after the plan is saved.

    A test will not run successfully if it includes parameters that do not have assigned values.

Datasets

Datasets allow you to run tests with multiple sets of parameter values. A dataset is created from an XLS or CSV file where the first row contains the parameter names and subsequent rows contain different values for each parameter. A dataset file can include up to 20 parameters (columns) and 1000 values (rows).

Using a dataset enables comprehensive testing across various scenarios without manually changing parameter values for each test run. This ensures consistency and saves time, especially when testing large data sets or multiple environments.

When you run a plan that includes a dataset, each test in the plan will run multiple times; once for each set of parameter values in the dataset.

Uploading a Dataset

Before using datasets, you must upload a dataset file, assign it a name, and link it to an application. You can then assign the dataset to tests by selecting its name.

To Upload a Dataset

  1. In the Navigation bar, select Test data.

  2. Select the Datasets tab.

  3. Click Add dataset.

    If you haven't created a dataset file yet, click Dataset file template to download a sample CSV file. Modify this file by adding the required parameter names and values.

  4. Enter the following information:

    • Name – Assign a name to the dataset. This name will be used to select the dataset in the plan.

    • Description – Optionally, add a description of the dataset.

  5. Click Upload dataset file and browse to the dataset XLS or CSV file you want to upload.

  6. Click Add dataset.

  7. To verify that the dataset was uploaded correctly, click next to the dataset name and select View.

    A list of values in the dataset appears. If a parameter includes confidential data, for example if the name includes "Password," "Secret," or "PIN," it is automatically masked after the plan is saved.

    You cannot modify values in this window. To make changes, close the preview window, click > Update, and select a new dataset file.

Downloading a Dataset

If you need to change the values in a dataset, you can download a copy of an existing dataset, make changes, and then add the revised dataset to the plan. Note that if a dataset ioncludes masked data such as passwords, you cannot download a copy of the dataset.

To Download a Dataset

  1. In the Navigation bar, select Test data.

  2. Select the Datasets tab.

  3. Click next to the dataset name and select Download.

Assigning a Dataset to a Plan

Once you have associated a dataset with an application, you can add it to a plan within that application.

To Assign a Dataset to a Plan

  1. In the Navigation bar, select Plans.

  2. Open a plan that includes a custom flow test containing parameters.

  3. Next to the name of the custom flow test containing parameters, click

  4. Select the dataset and click Add dataset.

    The list of available datasets only includes datasets associated with the application.

Dataset Example

A common use case for a dataset is testing multiple versions of a site, such as production and staging environments or different language versions. To do this, create a dataset file listing the base URLs for each site to be tested.

In the custom flow test, use the parameter name {URL} as the start URL.

When the dataset is applied to the test, it will run multiple times, each with a different URL from the dataset. The Results page will display each test separately, with the parameter values listed in the Test Data column.