Using GitLab with Eyes

Introduction

The Eyes GitLab integration allows you to benefit from Eyes visual testing and branch and merge workflow on projects run using GitLab. This article focuses on how to configure Eyes and GitLab so that work together and assumes that you are already acquainted with working with both Eyes and GitLab.

GitLab integrates with Eyes in the following aspects:
  • All Eyes tests in a specific CI build are automatically grouped together as an Eyes batch, with a batch name that includes the Git commit information.
  • Eyes tests are automatically configured to use and save baselines in branches with names derived from the test's GitLab branch name.
  • The GitLab merge request panel includes a status indicating the latest visual test results and merge status.
  • When you initiate a GitLab merge request, this triggers an Eyes branch merge.

This article has two parts

Getting started

How to configure Eyes to interact with GitLab

Integrating GitLab into Eyes is done using the Test Manager and must be done by a user with Eyes team administration privileges. This integration involves the following steps:

  • On the Admin panel, click on the Teams tile.

  • Click on the name of the team you want to configure.

  • On the Integrations tab, click GitLab.

  • Select your server from the drop-down list. If your server is not on the list, then click the Add button and enter the URL in the dialog that opens.

  • Click the Manage repositories button.

  • If you have not already logged into GitLab, you will see a message asking you to do so. Click the link and enter your GitLab credentials.

  • Once you have logged into GitLab, you can see a list of repositories associated with the logged-in GitLab account.

  • The dialog includes a filter you can use to narrow down the list of repositories.

  • Select the repositories that you require and click Apply.

  • A list of selected repositories is now visible. You can remove a repository from the list by hovering over it in the list and clicking the button.

  • By default, GitLab integration closes a batch automatically once the CI build operation is completed. If your build runs concurrently or consists of multiple tests and you want all of the results to appear together in a single batch, you need to disable automatic closing of the batch and close the batch explicitly when the build is complete in your CI.

    To control the batch closing operation for a repository, select the Show batch closing control checkbox and enable or disable auto-closing of the batch. If you disable batch auto-closing, you must explicitly close the batch.

If your organization uses an Eyes on-premise server, in addition to the above, you should also white list inbound HTTPS traffic from the GitLab server to your Eyes server in your organizationhs firewall.

How to configure GitLab to run Eyes tests

In the GitLab configuration file .gitlab-ci.yml, add the following lines:

Copy 
variables:
    APPLITOOLS_API_KEY: "YourAPiKey"
    APPLITOOLS_BATCH_ID: ${CI_COMMIT_SHA}

Replace the string YourAPiKey with your API key. To get the value of your API key, see How to retrieve your API key.

Typically the .gitlab-ci.yml will also contain at least one job that runs an Eyes test.

How to prepare your code

The Eyes SDK obtains the API Key and the BatchID from the environment variables that are set up in the gitlab-ci.yml, as described above. If you are converting existing tests, then ensure that the existing codes do not explicitly set the API key, batchID or batch name.

Working with GitLab

Once your system is configured as described above, you can use GitLab in the usual way. A GitLab pipeline is run when changes are pushed to a branch in your repository. This pipeline will typically include an Eyes visual test.

The Eyes GitLab integration is based on the GitLab merge request. When there is an open merge request, Eyes adds the following functionality to your GitLab workflow:

  • When GitLab runs a pipeline that includes an Eyes test, Eyes detects that the run is associated with a GitLab merge request and searches for a baseline to use as a reference. Eyes checks for a matching baseline in the following order:
    • On a branch that corresponds to the repository name and source branch defined by the merge request.
    • If there is no such branch, it creates a branch and populates it with the latest baseline for that test from the repository name and target branch defined by the merge request.
    • If there is no such baseline, the test will be marked as New.
  • Eyes sends the GitLab server status information and this is displayed in the GitLab CI results.

How to review branch differences and resolve them

Merging an Eyes source branch with a target branch results in adding new baselines to the target branch or updating its existing baselines according to baseline changes found in the source branch. Changes can include both updated baseline images and new, changed, or removed annotations (e.g. an added ignore region). Eyes compares two baselines and determines whether they can be merged automatically or whether manual user resolution is required.

Initiating the merge

When you click on the scm/applitools/team1 link, the browser switches to the Test Manager Merge branches page reloaded with the source and target branches derived from the merge request. If there are no conflicts, the Test Manager displays a pop-up window, as shown below.

Click the Got it! button to remove the pop-up.

By default, code merge is blocked if there are unsaved changes to the checkpoint as the SCM will not be marked as Passed. To enable or disable this setting, contact Customer Support.

Merging the Eyes baseline

The Eyes merge workflow is fully described in the article The Compare and merge branches page. Below, we will only provide a brief overview of the main merge features.

The Merge branches page shows all of the baselines that have changed in the source branch with respect to the target branch. That is, either they don't exist in the target branch, or they exist in the target branch but have changed in the source branch. Note that baselines that have changed in the target branch and were not changed in the source branch will not be listed and are ignored in the merge process.

At the far left of each row, the status column can show a Merge status such as Conflict, indicating that you need to resolve the baseline.

You resolve conflicts by using the buttons to the right of each row, which allow you to choose whether you want to replace the target baseline with the source baseline or to keep the target baseline unchanged. Choosing one of these changes the Conflict status to a Resolved status. You can also compare the two baselines and manually edit the source baseline to include the changes from both branches.

An alternative to resolving each test individually is to use the checkbox at the start of each row to select rows, and use the buttons in the toolbar at the top left of the window to choose how to resolve the selected rows. The toolbar also has buttons that allow you to delete selected baselines in the source branch or to undo the resolution made for the selected baselines.