GitHub integration - Configuring your CI (Advanced)

In most configurations, the SDK can automatically retrieve the APPLITOOLS_BATCH_ID and APPLITOOLS_BATCH_BUILD_ID information from Git. In this case, see GitHub integration - configuring your CI. If the SDK cannot retrieve this information, you need to manually set up our CI to configure the Eyes SDK with information related to the GitHub pull request or commit request. To allow the Eyes Batch to integrate data received from the GitHub server, you need to set the following Environment variables:

  • APPLITOOLS_API_KEY - Your API key. See How to obtain your API key.

  • APPLITOOLS_BATCH_ID - The GitHub commit SHA of the current build. This also must be set as an environment variable before running Eyes tests.

    • For a pull request, the SHA value must be the last commit in the source branch.

    • For a push action, the SHA value must be the last commit in the branch that was pushed.

  • APPLITOOLS_BATCH_BUILD_ID - (optional) The GitHub build ID. This is required if you run multiple builds in parallel on the same commit SHA, for example, parallel push and PR builds of the same source branch. You can obtain the GitHub build ID from your CI, for example in GitHub Actions, access the build ID using the GITHUB_RUN_ID environment variable which contains the unique identifier for the current workflow run.

    If you run parallel push and pull requests, you need to disable Automatic Batch Closing, (see Disabling automated batch closing) and notify Eyes when a build is complete.

If Eyes cannot recognize the End of Build event, (for example, if your CI system or GitHub cannot identify an End of Build event, or you have multiple concurrent builds), you also need to add a cURL command to notify Eyes when all builds are complete. For details, see Notifying Eyes when a build is complete.

Below are procedures to set required variables and other required settings in common CIs.

GitHub Actions

  • In the workflow YAML file, add the following:

    Copy 
    env:
        APPLITOOLS_API_KEY: ${{ secrets.APPLITOOLS_API_KEY }}
        APPLITOOLS_BATCH_ID: ${{ github.event.pull_request.head.sha || github.sha }}

    • The example above assumes that you have access to the GitHub secrets service and have stored your Applitools API key in the GitHub secret name APPLITOOLS_API_KEY.

      To obtain your API key, see How to obtain your API key.

    • The APPLITOOLS_BATCH_ID value is the Git commit ID (SHA).

    • If you use the public cloud, the value of APPLITOOLS_SERVER_URL is 'https://eyesapi.applitools.com'.

      If you run the Eyes server on a private cloud or on-premise system, then change the value assigned to the environment variable APPLITOOLS_SERVER_URL to the URL of your Eyes server.

  • If your GitHub workflow includes two or more concurrent Eyes jobs, add another dependent job that waits for these jobs to complete and notifies Eyes that the batch has completed. For example, if you have three jobs called runtests1, runtests2, runtests3, add the following job to the YAML file:

    Copy 
    batch-completion-notification:
      # This runs after all the test jobs have completed
      needs: [runtests1, runtests2, runtests3]
      runs-on: ubuntu-latest
      steps:
        - name: Update Applitools batch status
          uses: wei/curl@v1.1.1
          with:
          args: -X POST -d -H "accept:*/*" https://eyesapi.applitools.com/api/externals/github/servers/github.com/commit/${{env.APPLITOOLS_BATCH_ID}}/complete?apiKey=${{env.APPLITOOLS_API_KEY}}

    If you use the optional APPLITOOLS_BATCH_BUILD_ID environment variable:

    Copy 
    batch-completion-notification:
      # This runs after all the test jobs have completed
      needs: [runtests1, runtests2, runtests3]
      runs-on: ubuntu-latest
      steps:
        - name: Update Applitools batch status
          uses: wei/curl@v1.1.1
          with:
          args: -X POST -d -H "accept:*/*" https://eyesapi.applitools.com/api/externals/github/servers/github.com/commit/${{env.APPLITOOLS_BATCH_ID}}/${{env.APPLITOOLS_BATCH_BUILD_ID}}/complete?apiKey=${{env.APPLITOOLS_API_KEY}}

    See Notifying Eyes when a build is complete for details.

Travis CI

  • In the travis.yml file, add the following line to the script section that runs before an Eyes test runs:

    Copy 
    script:
    - export APPLITOOLS_BATCH_ID=$(echo ${TRAVIS_PULL_REQUEST_SHA:=$TRAVIS_COMMIT})

  • On your project page, click on More options and then select Settings.

  • In the Environment Variables section, enter the following:

    • In the Name field enter APPLITOOLS_API_KEY.

    • In the Value field enter the value of your API key. To obtain the value of your API key, see How to obtain your API key.

  • Click Add.

CircleCI

  • In the config.yml file in the .circlec folder, add the following command under jobs/build/steps so that it runs before the Eyes test starts:

    Copy 
    jobs:
      build:
        steps:
           - run: 
               command: |
                  export APPLITOOLS_BATCH_ID=$(echo $CIRCLE_SHA1)
                  run_the_test

  • In the project dashboard, click the gear icon

  • On the menu on the left, under Build Settings, click Environment Variables.

  • Click Add Variable.

  • In the Name field enter APPLITOOLS_API_KEY.

  • In the Value field enter the value of your API key. To obtain the value of your API key, see How to obtain your API key.

  • Click Add Variable.

Semaphore CI

  • On the Project page, click Project Settings.

  • In the setup panel, click Add New Command Line, then copy the following line to the field and press enter:

    Copy 
    export APPLITOOLS_BATCH_ID=$(echo $REVISION)

  • Click on Environment Variables and click Add.

  • In the Name field, enter APPLITOOLS_API_KEY.

  • In the Content field, enter the value of your API key. To obtain the value of your API key, see How to obtain your API key.

  • Select the Encrypt Content check box.

  • Click Create Variable.

AppVeyor

  • In the appveyor.yml file, add the following line in the init section:

    Copy 
    init:
      -ps: $env: APPLITOOLS_BATCH_ID =
      if ($env: APPVEYOR_PULL_REQUEST_HEAD_COMMIT) {
        $env: APPVEYOR_PULL_REQUEST_HEAD_COMMIT
      }
    else {
      $env: APPVEYOR_REPO_COMMIT
    }

  • On your project page, click SETTINGS.

  • On the left panel select Environment. and in the Environment Variables section click Add Variable.

  • In the Name field, enter APPLITOOLS_API_KEY.

  • In the Value field enter the value of your API key. To obtain the value of your API key, see How to obtain your API key.

  • Click Save.

Azure DevOps

  • In Azure, select Pipelines and then click on the name of your pipeline.

  • Click Edit.

  • Click Variables and set the variable applitools.api.key with your API Key. To get the value of your API key, see How to retrieve your API key.

  • Add the following to the azure-pipelines.yml file:

    Copy 
    variables:
      - name: commitSha
      # Use Build.SourceVersion if we aren't in a PR
        ${{ if ne(variables['Build.Reason'], 'PullRequest') }}:
         value: $(Build.SourceVersion)
      # Use System.PullRequest.SourceCommitId if we are in PR
        ${{ if eq(variables['Build.Reason'], 'PullRequest') }}:
         value: $(System.PullRequest.SourceCommitId)

     env:
       APPLITOOLS_API_KEY: $(applitools.api.key)
       APPLITOOLS_BATCH_ID: $(commitSha) 

for example:

Copy 
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml

trigger:
- master

pool:
  vmImage: ubuntu-latest
variables:
  - name: commitSha
  # Use Build.SourceVersion if we aren't in a PR
    ${{ if ne(variables['Build.Reason'], 'PullRequest') }}:
     value: $(Build.SourceVersion)
  # Use System.PullRequest.SourceCommitId if we are in PR
    ${{ if eq(variables['Build.Reason'], 'PullRequest') }}:
     value: $(System.PullRequest.SourceCommitId)

steps:
- script: echo Hello, world!
  displayName: 'Run a one-line script'

- script: |
  
  # Run Applitools visual tests here

  displayName: 'Run a multi-line script'
  env:
    APPLITOOLS_API_KEY: $(applitools.api.key)
    APPLITOOLS_BATCH_ID: $(commitSha)

Jenkins

Navigate to the Jenkins project configuration, go to the Execute shell window and enter the following:

Copy 
export APPLITOOLS_BATCH_ID=$(echo ${GIT_COMMIT})
export APPLITOOLS_API_KEY= "your api key"

For full instructions on integration with Jenkins, see Updating Jenkins build status.

iOS integration

When using Images or XCUI, you can include commit information in the following format within the Eyes Open message: $(git log -1 --format=%aI,%H), for example 2024-05-26T15:34:50+03:00,6fe5e1897ef6f0308c04d39f6de259263b2722e0.

You can export this information using the Command Line Integration or by adding it through a Run Script phase in the Build Phases of your test target.

In addition, APPLITOOLS_PARENT_BRANCH_BASELINE_SAVED_BEFORE stores the baseline date of the parent branch for comparison purposes. You can include this information in the command line or through the same Run Script phase.

Command Line Integration

To export the commit information in the command line, incorporate the following commands into your Xcode build command:

Copy 
APPLITOOLS_LATEST_COMMIT_INFO="$(git log -1 --format=%aI,%H)" xcodebuild test -scheme ...

To include the baseline date of the parent branch:

Copy 
MERGE_BASE_HASH=$(git merge-base $(git rev-parse --abbrev-ref HEAD) %PARENT_BRANCH%)
APPLITOOLS_PARENT_BRANCH_BASELINE_SAVED_BEFORE="$(git show -q --format=%aI $MERGE_BASE_HASH)" xcodebuild test -scheme ...

Xcode Project Integration using a Run Script

To add a Run Script phase to your Xcode project:

  • Launch Xcode and open your project.

  • In the Project Navigator, select the target for which you want to add the script.

  • Open the Build Phases tab, click + and select New Run Script Phase.

  • In the newly added Run Script phase, copy and paste the following script:

    Copy 
    # Retrieve the latest commit information
    COMMIT_INFO=$(git log -1 --format=%aI,%H)

    # Define the build directory for the plist file
    BUILD_DIR_PLIST="$(dirname "$TARGET_BUILD_DIR")/Applitools.plist"

    # Update the plist file with the latest commit information
    /usr/libexec/PlistBuddy -c "Delete :APPLITOOLS_LATEST_COMMIT_INFO" "$BUILD_DIR_PLIST"
    /usr/libexec/PlistBuddy -c "Add :APPLITOOLS_LATEST_COMMIT_INFO string $COMMIT_INFO" "$BUILD_DIR_PLIST"

    # Set Applitools server URL if not already set
    if [ -z "$APPLITOOLS_SERVER_URL" ]; then
      APPLITOOLS_SERVER_URL="https://eyesapi.applitools.com"
    fi

    # Fetch branch names if not already set
    if [ -z "$CURRENT_BRANCH_NAME" ] || [ -z "$PARENT_BRANCH_NAME" ]; then
      if [ -n "$APPLITOOLS_API_KEY" ] && [ -n "$APPLITOOLS_BATCH_ID" ]; then
        RESPONSE=$(curl "$APPLITOOLS_SERVER_URL/api/sessions/batches/$APPLITOOLS_BATCH_ID/bypointerId?apiKey=$APPLITOOLS_API_KEY")
        CURRENT_BRANCH_NAME=$(echo "$RESPONSE" | sed -n 's/.*"branchName":"\([^"]*\)".*/\1/p')
        PARENT_BRANCH_NAME=$(echo "$RESPONSE" | sed -n 's/.*"parentBranchName":"\([^"]*\)".*/\1/p')
      fi
    fi

    # Determine current and parent branch names (Hardcoded Example)
    #CURRENT_BRANCH_NAME=$(git rev-parse --abbrev-ref HEAD)
    #PARENT_BRANCH_NAME=$( [ "$CURRENT_BRANCH_NAME" = "develop" ] && echo "master" || echo "develop" )

    # Get the merge base hash and baseline date
    MERGE_BASE_HASH=$(git merge-base $CURRENT_BRANCH_NAME $PARENT_BRANCH_NAME)
    BASELINE_DATE=$(git show -q --format=%aI $MERGE_BASE_HASH)

    # Update the plist file with the baseline date
    /usr/libexec/PlistBuddy -c "Delete :APPLITOOLS_PARENT_BRANCH_BASELINE_SAVED_BEFORE" "$BUILD_DIR_PLIST"
    /usr/libexec/PlistBuddy -c "Add :APPLITOOLS_PARENT_BRANCH_BASELINE_SAVED_BEFORE string $BASELINE_DATE" "$BUILD_DIR_PLIST"

    # Update the plist file with the current branch name
    /usr/libexec/PlistBuddy -c "Delete :APPLITOOLS_BRANCH" "$BUILD_DIR_PLIST"
    /usr/libexec/PlistBuddy -c "Add :APPLITOOLS_BRANCH string $CURRENT_BRANCH_NAME" "$BUILD_DIR_PLIST"

    # Update the plist file with the parent branch name
    /usr/libexec/PlistBuddy -c "Delete :APPLITOOLS_PARENT_BRANCH" "$BUILD_DIR_PLIST"
    /usr/libexec/PlistBuddy -c "Add :APPLITOOLS_PARENT_BRANCH

  • If you use the Eyes Public server, the URL is https://eyesapi.applitools.com. If you have a dedicated on-prem server, replace this with a URL in the format https://<My Organization>eyes.applitools.com.

  • Ensure that the new Run Script phase is placed after the Compile Sources phase to ensure proper execution during the build process.

  • Save your changes and build your project to apply the script.

If needed, you can add other Applitools environment variables to the Applitools.plist file in the same manner.

Note that integrating the Applitools.info into the build process does not modify your source code; it is added directly, ensuring that your source code remains unaffected.

Preparing your code for a commit push action

You need to add an API call to the CI script that will inform Applitools each time there is a commit push action.

Copy 
curl -L -d '' -X POST "$APPLITOOLS_SERVER_URL/api/externals/github/push?
apiKey=$APPLITOOLS_API_KEY&CommitSha=$APPLITOOLS_BATCH_ID&BranchName=<Branch name>
&ParentBranchName=<Parent branch name>"

  • <Branch name> represents the full GitHub branch name, including the organization, the repository, and the branch in the format my-org/my-repo/my-branch.

    Examples (Click to expand):

    Copy 
    curl -L -d '' -X POST "$APPLITOOLS_SERVER_URL/api/externals/github/push?
    apiKey=$APPLITOOLS_API_KEY&CommitSha=$APPLITOOLS_BATCH_ID&BranchName=${CIRCLE_PROJECT_USERNAME}/${CIRCLE_PROJECT_REPONAME}/${CIRCLE_BRANCH}
    &ParentBranchName=<Parent branch name - optional>"

    Copy 
    curl -L -d '' -X POST "$APPLITOOLS_SERVER_URL/api/externals/github/push?
    apiKey=$APPLITOOLS_API_KEY&CommitSha=$APPLITOOLS_BATCH_ID&BranchName=${GITHUB_REPOSITORY}/${GITHUB_REF_NAME}
    &ParentBranchName=<Parent branch name - optional>"

  • <Parent branch name> is optional. The default is the master branch of the repository.

  • The APPLITOOLS_API_KEY argument must be passed in the query and cannot be passed in the body.

  • <APPLITOOLS_BATCH_ID> is the GitHub commit SHA of the current build.