Working with Bitbucket

Once your system is configured to work with Bitbucket, use your CI and Bitbucket in the usual way. A CI job is normally triggered by a push to your repository, which then runs your visual UI tests.

Pull request workflow

When there is an open pull request, Eyes adds the following functionality to your Bitbucket workflow:

  • When your CI initiates a build that includes an Eyes test, Eyes detects that the run is associated with a Bitbucket pull request and searches for a baseline to use as a reference. Eyes checks for a matching baseline in the following order:

    • On a branch that corresponds to the repository name and source branch defined by the pull request.

    • If there is no such branch, it creates a branch and populates it with the latest baseline for that test from the repository name and target branch defined by the pull request.

    • If there is no such baseline, the test is be marked as "New".

  • Eyes sends Bitbucket status information that is displayed on the Bitbucket Pull requests panel as shown below. Similar information is also displayed in the Bitbucket Commit panel. The name of the Applitools team is included in the build name.

  • The item labeled tests/applitools/team1 indicates the results of the visual UI test that ran as part of this build.

    1. indicates that all the visual UI tests passed.
    2. indicates that some differences were found and need to be resolved, that the test failed to complete successfully, or that the merge process identified conflicts.
    3. indicates that the test is in progress.

  • The item labeled test/applitools/team1 is a link. Click on the link to open the Test results screen of the Eyes Test Manager, which shows the results of the tests in the most recent build related to that pull request.

    When dealing with differences, if you Accept all the steps of all the tests, then when you go back to the Bitbucket Builds panel, the (error/difference) is replaced by a (passed).

  • The symbols adjacent to the item labeled scm/applitools/team1 represents the merge conflict status. An icon indicates that there are merge conflicts. After you resolve any conflicts Eyes will send an updated status to Bitbucket, and you will see an icon next to the scm/Applitools/team1 item.

  • You can merge the baseline changes of the source branch with the target branch in the following ways:

    • Click the Merge button on the Compare and merge branches page in the Test Manager (enabled only after all conflicts have been resolved). This operation is independent of any merge process in Git and occurs immediately, even if the Bitbucket merge was not done.
    • Merge the Bitbucket pull request as usual. This first merges the Git files and, once this has completed successfully, Eyes is notified that it should merge the visual UI test baselines.

      • By default, code merge is blocked if there are unsaved changes to the checkpoint as the SCM will not be marked as Passed. To enable or disable this setting, contact Customer Support.

Reviewing and resolving branch

Merging an Eyes source branch with a target branch results in adding new baselines to the target branch or updating its existing baselines according to baseline changes found in the source branch. Changes can include both updated baseline images and new, changed, or removed annotations, (for example, an added ignore region).

Eyes knows how to compare two baselines and determine whether they can be merged automatically or whether manual user resolution is required. For example, an ignore region added to a source baseline step that did not change in the target baseline can be automatically added to the target baseline, but if another ignore region was added to the same step in the target baseline, user intervention is required to determine whether to keep only one of the added regions, or to keep both of them.

As described above, the item on the Bitbucket pull request page marked “scm/applitools/team-name” indicates the merge status of the Eyes baselines in the source branch.

When you click on the link on this item, the browser opens the Test Manager Merge branch page reloaded with the source and target branches derived from the pull request. If there are no conflicts, then the Test Manager displays a pop-up window stating this.

The Merge branches page shows all the baselines that have changed in the source branch with respect to the target branch. That is, either they do not exist in the target branch, or they exist in the target branch but have changed in the source branch. Baselines that have changed in the target branch and were not changed in the source branch are not be listed and are ignored in the merge process.

At the far left of each row, the status can show a value of Conflict, indicating that the baseline needs to be resolved. In most cases you can resolve the conflict by updating the pull request (rebase and merge main or master branch into the pull request branch), and rerun the tests.

If rebasing does not resolve the conflict, use the buttons on the right of each row, which allow you to choose whether you want to replace the target baseline with the source baseline or keep the target baseline unchanged. You can also compare the two baselines and manually edit the source baseline to include the changes from both branches.

An alternative to resolving each test individually is to use the checkbox at the start of each row to select rows and then use the buttons in the toolbar in the top left of the window to choose how to resolve the selected rows. The toolbar also has buttons that allow you to delete selected baselines in the source branch or to undo the resolution made for the selected baselines.

Once you have resolved all conflicts, the status of the scm/applitools item in the Bitbucket pull request page changes to green (passed), and you can proceed to merge the branches, either from within the Test Manager using the Merge button, or via the Bitbucket pull request page as described above.

When working on a branch that is a child of another branch (the parent branch) and the child branch doesn't have a baseline, Eyes will use a fallback baseline from the parent branch. For details, see Baseline management on a child branch.

Commit push workflow

When a commit is pushed to a branch, Eyes adds the following functionality to your Bitbucket workflow:

  • When your CI initiates a build that includes an Eyes test, Eyes detects that the run is associated with a Bitbucket commit and searches for a baseline to use as a reference. Eyes checks for a matching baseline in the following order:

    • On a branch that corresponds to the repository name and the name of the branch that the commit was pushed to.
    • If there is no such branch, it creates a branch and populates it with the latest baseline for that test from the repository name and the master branch as defined in APPLITOOLS_PARENT_BRANCH. If the master branch has not been defined it will use the Eyes default branch.
    • If there is no such baselines the test is be marked as "New".

  • Eyes sends Bitbucket status information that is displayed on the Bitbucket Commit panel as shown below. The name of the Applitools team is included in the build name.