Skip to main content

Remarks & Issues

Applitools offers two different regions that can be applied to test results that promote team collaboration:

  • Remark Region: Leave a comment/start a discussion about specific section of a Visual AI test result.
  • Bug Region: Discuss a bug flagged in a specific section of a Visual AI test result. Connect the bug to JIRA or Rally and configure how you want to handle this bug until it is resolved.

Remarks​

There are two types of Remarks:

  • Step Remark Regions: Applied to a specific section of a Visual AI test result.
  • Test Remarks: General conversation around a test as a whole.
Terminology

A "Step" is a Visual AI checkpoint that is apart of a "Test". "Steps" can be thought of as the screenshots that are captured during your Visual AI tests. A "Test" can contain any number of "Steps".

Step Remark Regions​

Creating a remark region​

You can add a remark to a specific step within a test by drawing a region around the specified section of the test result. Follow the steps below to add a remark region:

  1. Open up the step view for the test result in question
  2. Click on the icon shown below Remark Icon
  3. Draw a rectangle around the region you're interested in Remark Region
  4. Add a title for the Remark
  5. Click the Create button

Deleting a remark Region​

To delete a remark region, follow the steps below:

  1. Click on the remark region
  2. Click the delete trash can button Delete Remark

Test Remarks​

Creating a Test Remark​

Follow the steps below to create a Remark at the Test level:

  1. Open up the batch that contains the test you're interested in
  2. Click on the Remark icon at the end of the row of the test you're interested in Test Remark
  3. Optionally add a title of the conversation
  4. Type a message and click the Post button

Conversations​

Both Remark regions and Test Remarks contain a conversation. All team members with access to the test results can participate in the conversation.

Notifications​

When you participate in a Remark conversation, you will be notified via email when there are subsequent updates made to the conversation by other team members. At any point, you can toggle the notifications using the "Follow" checkbox shown below:

Follow Checkbox

Tagging Users​

To get a specific user's attention, you can tag them in a message by typing the '@' symbol followed by their name.

Tagged User

The tagged user will then be sent an email notification along with a link to the conversation in question.

Linking Conversations​

You can copy a direct link to the Remark conversation by clicking on the icon shown below:

Share Button

Bug Region​

Bug regions are a collaboration tool that let you track a bug that was found in your Visual AI test results.

Creating a bug region​

You can add a bug to a specific step within a test by drawing a region around the specified section of the test result. Follow the steps below to add a remark region:

  1. Open up the step view for the test result in question
  2. Click on the icon shown below Bug Icon
  3. Draw a rectangle around the region you're interested in Bug Region
  4. Add a title for the Bug
  5. Click the Create button

Deleting a bug region​

To delete a bug region, follow the steps below:

  1. Click on the bug region
  2. Click the delete trash can button

Delete Bug Region

Bug regions are different from Remark regions in that they can be linked to an external bug tracking platform such as JIRA or Rally.

note

You must have configued the JIRA or Rally integration for your Team for this to work. See the docs below for how to set up these integrations as well as how to link bugs.

Conversations​

Conversations work the same way as Remark Regions, see this section above.

Auto Fail Tests​

With a bug region, you can choose to Fail tests automatically until the bug region is removed. This helps with keeping track of known issues that have been flagged by the Applitools Visual AI but not yet fixed.

To automatically fail the tests, check the box shown below:

Fail Tests

When this box is checked, this test will be marked as failed if the only visual difference found between the baseline and checkpoint images is the one covered by the bug region. If another visual difference is found, the test will still come back as Unresolved ensuring that you don't miss any other Visual bugs as you continue working.

Snoozing Failures​

You can optionally choose to pause automatic failures until the specified time. This is useful if you'd like to prevent this test from failing your automation tests while you work to resolve the underlying issue. To enable this feature, check box shown below:

Snooze Failures

Once the specified date has been passed, the tests will go back to failing automatically until the bug region is removed or a new date is specified.

Issues And Remarks View​

To quickly see all tests that have Remarks or Issues associated with them, you can select the Issues & Remarks view shown below:

I&M View

By default the view shows all the issues and remarks in the batch, but you can use the filter to narrow down the regions displayed based on various criteria.

Each step displayed, includes a thumbnail of the step overlaid with the position of the region(s), and a list of issues and remarks made on that step, for each issue or remark you can see its title, the user that created it, the date and time it was created and the most recent comment in the discussion thread added.

In the Issues and remarks view, you can click on either the details area or the thumbnail and the Test Manager will open the remarks dialog or issues dialog in the Test editor, allowing you to add a comment and make changes.