Configuring your CI for the Eyes GitHub integration

The article Using Eyes with GitHub describes configurations that need to be set up in the Test Manager to use Eyes with GitHub. The current article describes the configuration that needs to be set up in the CI system you use with GitHub. The configuration consists of two steps:

  • Setting up the start of the build: Setup that is done by the CI as part of the build before the Eyes tests are executed. This sets environment variables that are read by the Eyes SDK to obtain the information required by Eyes to execute the test.
  • Synchronizing multiple concurrent builds: If the CI runs a build using multiple concurrent instances, then it needs to notify Eyes when all of the instances have terminated.

Setting up the start of the build

Eyes needs two values in order to run the visual tests in the builds:

  • Your Eyes API Key: This is passed by setting the environment variable APPLITOOLS_API_KEY to the value of your Eyes API key. To obtain the value of your Eyes API key, see How to obtain your API key.
  • The batch ID: In a standalone Eyes test, the result of each test execution appears in a separate batch in the Test Manager. When multiple Eyes tests are executed using a supported CI, all of the test results will appear together in a single batch. This is achieved by associating them with a common batch ID. The batch ID is passed to Eyes by settings the APPLITOOLS_BATCH_ID environment variable to a value generated by the CI.
For detailed configuration instructions for your CI, click on the relevant link below:

Travis CI

  • In the travis.yml file, add the following line to the script section that runs before an Eyes test runs:
    
    script:
    - export APPLITOOLS_BATCH_ID=`echo ${TRAVIS_PULL_REQUEST_SHA:=$TRAVIS_COMMIT}`
    
  • On your project page:
    • Click on More options and then select Settings.
    • In the Environment Variables section:
      • In the Name field put APPLITOOLS_API_KEY.
      • In the Value field put the value of your API key. To obtain the value of your API key, see How to obtain your API key.
      • Click Add.

CircleCI

  • In the config.yml file in the .circlec folder, add the following command under jobs/build/steps so that it runs before the Eyes test starts:
    
    jobs:
      build:
        steps:
           - run: "export APPLITOOLS_BATCH_ID=`echo $CIRCLE_SHA1`"
    
  • In the project dashboard:
    • Click the gear icon.
    • On the menu on the left, under Build Settings, click Environment variable.
    • Click Add Variable.
    • In the Name field put APPLITOOLS_API_KEY.
    • In the Value field put the value of your API key. To obtain the value of your API key, see How to obtain your API key.
    • Click Add Variable.

Semaphore CI

On the Project page, click Project Settings:

  • In the setup panel, click Add New Command Line, then copy the following line to the field and press enter:
    
    export APPLITOOLS_BATCH_ID=`echo $REVISION`
  • Click on Environment Variables.
    • Click Add.
    • In the Name field put APPLITOOLS_API_KEY.
    • In the Content field put the value of your API key. To obtain the value of your API key, see How to obtain your API key.
    • Tick the Encrypt Content check box.
    • Click Create Variable.

AppVeyor

  • In the appveyor.yml file, add the following line in the init section:
    
    init:
      - ps: $env:APPLITOOLS_BATCH_ID = if ($env:APPVEYOR_PULL_REQUEST_HEAD_COMMIT) { $env:APPVEYOR_PULL_REQUEST_HEAD_COMMIT } else { $env:APPVEYOR_REPO_COMMIT }
    
  • On your project page
    • Click SETTINGS.
    • On the left panel select Environment.
    • In the Environment variables section click Add variable.
    • In the Name field put APPLITOOLS_API_KEY.
    • In the Value field put the value of your API key. To obtain the value of your API key, see How to obtain your API key.
    • Click Save.

Jenkins

For full instructions on integration with Jenkins, see Updating Jenkins Build status.

Navigate to the Jenkins project configuration, go to the Execute shell window and type the following:

export APPLITOOLS_BATCH_ID=`echo ${GIT_COMMIT}`
export APPLITOOLS_API_KEY= "your api key"

How to prepare your code

The configuration described in the previous section sets up the values of the environment variables APPLITOOLS_API_KEY and APPLITOOLS_BATCH_ID to provide the Api key and batch ID respectively. Recent versions of the Eyes SDK reads these environment variables to obtain this information. If you are using an older versions of the SDK that does not do this, then we recommend updating your SDK to the latest version. If this is not possible then add the following code to your test after the eyes instance variable has been instantiated and before the test is opened.


                                

If you are using a version of the SDK that reads these environment variables then ensure that your code does not make any calls to the following methods since doing so would overwrite the automated handling:

Synchronizing multiple concurrent builds

By setting the APPLITOOLS_BATCH_ID environment variable, as described in the previous section, you instruct Eyes to put the results of all tests in a particular CI build in the same Test Manager batch. If your CI runs multiple builds concurrently, then you need to follow two more steps :

  • In the GitHub integration page of the Test Manager, follow the instructions to disable automatic batch closing described in the article How to configure the Eyes GitHub integration. This is a one-time configuration that needs to be done by each team for each repository that has concurrent builds.
  • You need to indicate to Eyes when all of the concurrent builds have completed. You do this by sending the Eyes server an HTTP command. How you do this depends on your environment, for example, on a Linux-based system you can do it as follows:
    
    set eyesserver="eyesapi.applitools.com"  # set this to your Eyes server name.
    set githubserver = "your_gituhub_server" # as entered in the Eyes/github integration screen
    set apikey=$APPLITOOLS_API_KEY
    set batchid=$APPLITOOLS_BATCH_ID         # this was previously set to the CommitSha	
    set url="https://$eyesserver/api/externals/github/servers/$githubserver/commit/$batchid/complete?apiKey=$apiKey	
    
    curl --request DELETE --dump-header - $url
    

    Note that various values represented here as $xxxx are values set in other parts of the integration:

    • $eyesserver: The URL of your eyes server - the same value that you set when defining the Eyes server when you run visual Eyes tests (or the default shown in the snippet above).
    • $githubserver: The GitHub server you entered on the Eyes/GitHub integration screen as described in How to configure the Eyes GitHub integration.
    • $apikey: The same API key you use when running visual Eyes tests.
    • $batchid: The unique value that defines the batch, it is the value assigned to the APPLITOOLS_BATCH_ID environment variable as described in the section Setting up the start of the build above.