Data-Driven Testing

Data-driven testing (DDT) lets you set test parameters outside the test script, allowing you to run the same test under different conditions. This means that you can assign different values to the same parameter, making your tests highly flexible and reusable. Parameters can be set with explicit values in a plan or by using a Dataset.

Parameters

Parameters are set at the plan level. For example, you might have a parameter for a username that is set to Jane Doe in one plan and Admin in another. When the test runs as part of each plan, it will use the parameter values assigned to that plan, allowing you to test various scenarios without modifying the test script itself.

Entering Parameter Values

When you create the test, enter the name of the parameter in braces { }. This placeholder will be replaced with the actual value of the parameter or dataset assigned to a plan. A parameter can also get a value from a variable. For details, see Variables

For example, in the following test script, the parameters {Username} and {email} will be replaced by their respective customized values when the test runs.

Copy 
Go to https://example.com
Click "Form"
Enter {Username} in the "Your Name" field
Enter {email} in the "Email" field
Visually check this screen

Setting Parameter Values in a Plan

To Set Parameter Values in a Plan

  1. In the Navigation bar, select Plan management > Plans.

  2. Open a plan that includes a custom flow test with parameters.

    Note that tests with parameters will display the parameter names in the Test data column. If there are any parameters without values, a icon will appear. Click this icon to assign values.

  3. Above the list of tests, click Set Parameter Values. This option is available only if at least one test uses parameters.

  4. Enter a value for each parameter and click Set values. This window displays all parameters used in the plan that do not have a value set in a dataset. Note that a test will not run successfully if there are parameters that do not have a value.

Previewing with Parameter Values

When previewing a custom flow test that includes parameters, you must enter values for all parameters that are used in the test. These values are used for the preview but do not affect the actual test run, which will use the parameter values defined in the plan.

To Enter Temporary Parameter Values for Previews

  1. Open a test that includes at least one parameter.

  2. Click to open the parameter values window.

  3. Enter a value for each parameter to use in the test preview.

    If any parameter does not have a value, this window will open automatically when you preview the test.

  4. To save these preview values for future preview runs, select Remember these values for your future preview runs.

  5. Click Set Values to apply the preview values.

Datasets

Datasets allow you to run tests with multiple sets of parameter values. A dataset is created from an XLS or CSV file where the first row contains the parameter names and subsequent rows contain different values for each parameter. A dataset file can contain up to 20 parameters (columns) and 1000 values (rows).

Using a dataset enables you to perform comprehensive testing across various scenarios without manually altering the parameter values for each test run. This ensures consistency and saves time, especially when dealing with large sets of data or multiple environments.

When you run a plan that includes a dataset, each test in the plan will run multiple times; once for each set of parameter values provided in the dataset.

Uploading a Dataset

To use datasets, you should first upload a dataset file, give the dataset a name, and assign it to an application. You can then assign the dataset to tests using the dataset name.

To Upload a Dataset

  1. In the Navigation bar, select Test management > Test data.

  2. Select the Datasets tab.

  3. Click Add dataset.

    If you haven't created a dataset file yet, click Dataset file template to download a sample CSV file. Modify this file by adding the required parameter names and values.

  4. Enter the following information:

    • Name – Assign a name to the dataset. This name will be used to select the dataset in the plan.

    • Description – Optionally, add a description of the dataset.

    • Application – Select an application. Each dataset must be associated with one application and can only be assigned to plans associated with that application.

  5. Click Upload dataset file and browse to the dataset XLS or CSV file you want to upload.

  6. Click Add dataset.

  7. To verify that the dataset was uploaded correctly, click next to the dataset name and select View dataset file.

    Autonomous displays the dataset file.

    You cannot modify values in this window. To make changes, close the preview window and click > Update dataset file.

Assigning a Dataset to a Test

Once you have associated a dataset with an application, you can add it to a plan within that application.

To Assign a Dataset to a Test

  1. In the Navigation bar, select Plan management > Plans.

  2. Open a plan that includes a custom flow test containing parameters.

  3. Next to the name of the custom flow test containing parameters, click

  4. Select the dataset and click Add dataset.

    The list of available datasets only includes datasets associated with the application.

Dataset Example

One common use case for a dataset is testing multiple versions of a site, such as production and staging environments or different language versions. To achieve this, create a dataset file listing the base URLs for each site to be tested.

In the custom flow test, use the parameter name {URL} for the start URL.

When the dataset is applied to the test, it will run three times, each with a different URL from the dataset. The Results page will display each test separately, with the parameter values listed in the Test Data column.