Batch completion notifications

The EyesBatch completion notifications feature allows you to request that notifications are sent to one or more people when the execution of a batch of tests has completed. The notifications are sent using Slack and/or email, so anyone who is interested, and authorized to receive updates on the build status and quality of a batch of tests has access to this information even if they don't have access to Eyes.

You control how and when Eyes sends batch completion notifications using the Test Manager Admin page. You can specify which applications under test trigger notifications and whether to notify on every batch completion or just those that encountered issues or differences. You can also configure whether to send notifications via email, Slack, or both. When running tests, you use the SDK (or environment variables) to control whether notifications should be sent for that particular batch execution. This allows you to avoid sending notifications on test maintenance/creation runs.

The notifications sent include the following details:

• The team name, the batch name, and the date and time the batch started

• A summary of the test results: how many tests passed, how many tests require resolution, and how many tests were aborted

• A link to the test results in the Test Manager

For an example of a notification and more details see Notification details below.

Setting up batch completion notifications

To set up Eyes so that it sends notifications:

• Enable batch notification using the SDK or environment variables. This is described in detail in Setting up batch completion notifications in the SDK.

• Select which applications are notified, in which scenarios, and via which communication methods. Only account or team admins can change these settings.

  • To send notifications to a Slack workspace and channel, see Batch completion notifications - Slack integration.
  • To send notifications to an email list, see Batch completion notifications - Email integration.

Notification details

Each notification includes a summary of the batch status information and a link to the full batch test results in the Test Manager. Here is an example of an email notification:

The batch status information includes the following details:

1. Team name

2. Batch name

3. Batch execution start date and time

4. Total number of tests

5. The number of Passed tests as an absolute number and as a percentage of Total tests

6. The number of Unresolved tests as an absolute number and as a percentage of Total tests

7. The number of aborted tests

8. The number of unresolved differences identified out of the total number of steps

Automatic batch completion

In SDKs that support the ClassicRunner or VisualGridRunner, the SDK notifies the Eyes Server that the batch has completed when it completes the getAllTestResults call. In order to catch a situation in which a test has crashed or is hung, unterminated batches that have been inactive for more than two hours or have been running for more than four days are automatically terminated by Eyes, at which point notifications are sent as configured.