Azure DevOps Extension integration

This document describes how you can run a test that implements Applitools Eyes visual UI testing from within the build environment, and then see the results in the Azure DevOps Build result window.

The article assumes that you already have Azure DevOps installed on your system and that you know how to use it. Note that depending on your version of Azure DevOps, some of the steps or screenshots described here may be slightly different.

Overview

Once you have done the one time setup described in this article you will be able run your visual UI test from within the Azure DevOps environment and see the Eyes results within a Azure DevOps build results window as shown below:

Setting up Azure DevOps to work with Eyes involves the following steps:

1. Preparing your Eyes code
2. Setup your Azure DevOps build
3. Install the Eyes Extension

Each of these is described in the sections that follow.

Preparing your Eyes code

The Eyes extension automatically generates a Batch name and batch ID and assigns them to environment variables. Your code should access the environment variables and use these values, as shown in the following code:

The batch name defined by the Eyes Extension is composed of the project name and the build definition name as defined in the Azure DevOps environment.

Note that the extension uses environment variables in two different ways.

APPLITOOLS_API_KEY is an environment variable that you need to define in your operating system or in Azure DevOps, as explained in the next section.

The environment variables APPLITOOLS_BATCH_NAME and APPLITOOLS_BATCH_ID are defined and set by the Eyes SDK Azure DevOps extension; you should not define these variables in your operating system.

Setup your Azure DevOps build

The Eyes extension needs access to your Eyes API key. It obtains the value of the key from an environment variable called APPLITOOLS_API_KEY You can set the environment key externally via your operating system, or you can do it internally in Azure DevOps  as follows:

1. On the "Builds and Release" page, "Builds" tab select your build by clicking on it.

   

2. Click on the Edit button.

   

3. In the Variables tab, click the Add button.



4. In the left input field type the name of the environmental variable APPLITOOLS_API_KEY, and in the right field type your API key. To obtain your API key see How to obtain your API key. Click on save, and in the Dialog that opens add a comment and click Save.

Installing the Eyes Extension

1. Add the Applitools Eyes build task. In Azure DevOps, click on the "Build and Release" tab, then click on the build definition to which you want to add the Eyes task.



2. Click on the Edit button.



3. Click on Add Task.



4. Click on "Check out our Marketplace".



5. Type "applitools", click the search icon and you should see the Applitools Eyes Integration Extension. Click on the Applitools Eyes Integration Extension.



6. Click on the Install button.



7. Wait a bit....



8. Click "Confirm".



9. Wait a little bit more...



10. Click the "Close" button.



11. Refresh the browser.



12. Click "Add Task".



13. In the search box start typing applitools, when the Applitools Build Task appears, put your cursor over it until the "Add" button is displayed and then click it.



14. The Applitools task appears at the end of the list of tasks.



15. Drag it up so that it is before the "Build Solution".



16. Click "Save & Queue", click "Save", add a comment in the popup and click "Save".

Running your tests

You run your test in Azure DevOps in the usual way :

1. Click the "Queue" button.



2. Click the "Queue" button on the popup.



3. Click on the build number of the current job.



4. This will bring up the Azure DevOps Build results window. Click the Applitools tab, and then, when the test is complete, you will see the main panel of the Applitools Test Manager with the results of the current build loaded.

You will see the results of the current job. To see the results of other builds you can select them from the Azure DevOps "Recently Completed" list. You can also enter full Test Manager mode where you can see all the results of all your tests runs. To do this do as follows:

1. Click any test to see the steps of that test.
2. Click on the step thumbnail to enter the Test Editor.
3. If you click the button, the display will return to Azure DevOps. If you click the Applitools logo, the display will show the main Test Manager panel where you can see all the runs.