Tests requiring attention page

The Tests requiring attention page lists test run in your team, in all branches and all applications, that did not pass and require your review. This is an easy way for you to see at a glance your most urgent tasks.

In this page, a test represents a unique combination of test name, viewport size, browser, and operating system and only counts the last run. This means that if there are multiple tests with the same combination, they will be listed as a single item on the Tests requiring attention page.

Filters

At the top of the page you can filter the list of tests displayed by the following parameters, applying a filter impacts the entire page.

  1. Date Range – Select Last 7 days, Last 30 days, Last 60 days, or a custom date range.

  2. Applications – Limit the results to selected applications.

  3. Branch – Limit the results to selected branches.

Click the Share link icon to copy a link for the current page to the clipboard. The link includes filters and grouping that has been applied.

Tests overview

This area summarizes the number of tests which require attention.

The numbers represent the number of tests which did not pass for the following reasons:

  1. Unresolved tests – Tests that contain steps with differences that have not yet been reviewed.

  2. Tests with missing steps – Tests that had steps removed. This means that an existing baseline image has no corresponding checkpoint image. This may be caused by a URL that was removed from your website. To understand how Eyes identifies missing steps, see Tests results and steps.

  3. New tests – Tests that are run for the first time and do not have a previous baseline to compare results to.

  4. Unsaved tests – Tests with changes that have not been saved.

  5. Aborted tests – Tests that were aborted before all the steps were completed. A test may have been aborted manually by a user, or aborted by the system as Eyes was not able to complete the test. If a test was aborted by the system, you need to investigate whether there is a technical error such as a network error or the website is down.

  6. Empty tests – Tests that did not produce any results as there were no checkpoints.

  7. Failed tests with issues – Tests that had a bug region marked and were manually marked as rejected by a user. These are issues that need to be resolved by your development team.

Number of test runs that did not pass

This area shows the five tests with the greatest number of test runs that did not pass. If no filter has been applied, this includes all branches and environments. There is a link to the insights screen which shows additional information about the tests.

List of tests requiring attention

The table lists tests that did not pass in the latest test run, ordered by the number of runs that did not pass. You can click on a row to view details and resolve the issue without needing to open it from the test batch.

Quick Filters

At the top of the list of tests, you can select which types of tests should be displayed.

If multiple options are selected, tests that belong to any of the selected options are displayed. If no options are selected, all results are displayed. You can limit the test displays by any of the following.

  1. Unresolved tests – Tests that contain steps with differences that have not yet been reviewed.

  2. New – Tests that are run for the first time and do not have a previous baseline to compare results to.

  3. Aborted – Tests that were aborted before all the steps were completed.

  4. Empty – Tests that did not produce any results.

  5. Issues found – Tests that had a bug region marked and were manually marked as rejected by a user.

  6. New steps – Tests that have new checkpoints that do not have a corresponding baseline.

  7. Missing steps – Tests that have baselines which are missing checkpoints.

You can also search by test, branch, application, or batch name.

To apply an advanced filter, click to open the Filter test results dialog box. For details, see Filtering test results.

Resolving tests

You can click on a row to view a thumbnail of steps in the test. From this thumbnail you can click on an icon to do the following:

  • Accept differences in step
  • Reject differences in step
  • Highlight differences – displays a red circle around differences in the step that need to be reviewed
  • Show baseline / Show checkpoint – toggles between the checkpoint and baseline images
  • Open in step editor – opens the image in the step editor where you can see a full size screenshot and accept or reject changes, annotate regions, or mark a bug region.

Drilling down to step details

Click to perform the following actions on a step:

  • Filter by – Filters the results by browser, operating system, step name, or viewport.

  • Open in test editor.

  • Open the step editor.

  • Show baseline image – toggles between the checkpoint and baseline images

  • Show test details

  • Show baseline history

Adding comments

Click to add a comment to the test. You can tag other team members using a @ symbol.

Changing the layout of the table

You can select which columns are displayed and group items by selected criteria, for example you may want to group results by browser and then browser version.

To select columns to display, click and select the required columns.

To group items, click , select the required groups and click to move them to the active groups area.