Eyes organizes batches with the same Batch sequence name into chronological order. Eyes collects, analyzes, and displays information and statistics for each batch sequence within the Scope and Date Range set on the Batch statistics tab in the Insights page.
For more information, see Setting the Scope and Setting the Date Range.
The Insights Batch Statistics tab lists batch sequences. The image below shows an example of how this tab displays batch sequences.
This page deals with the following batch statistics related activities:
Viewing batch sequence status information
Each batch sequence appears in a separate row in the batch sequence list. Each row shows the status and execution time of the last batch in the sequence, along with a summary of all the batch runs in the sequence for a defined time period, including the number of passed, failed and unresolved batches, and the total number of batch runs.
Click a row to expand it and view charts and analytics that provide further insights and statistics on the batch sequence.
Batch sequence statistics
The Insights Batch Statistics tab displays batch sequences and their statistics over time. Click a batch sequence in the list to expand the row to display charts with various batch sequence status information and statistics. All statistics are based on the current status information for a particular metric.
Each chart is color-coded and accompanied by a legend. Hover over the chart to view information on the data metrics or status for that part of the chart.
This section provides a brief description of each chart.
Test results status
A Pie chart that shows the pass rate for all tests executed over a set period. The pass rate represents the ratio of tests that passed compared to those that did not, expressed as a percentage.
Diff status
A pie chart that shows the percentage of Diffs currently accepted, rejected, and unresolved for the selected batch sequence.
Baseline status
Shows the status of steps and changes on the baseline, including the number of saved steps, unsaved steps, and unsaved changes.
-
Saved steps: Steps saved to the baseline.
-
Unsaved steps: Steps that were never saved to the baseline.
-
Unsaved changes: Steps with changes not yet saved to the baseline.
Tests
Indicates the total number of tests in the batch and the number completed, running, and aborted.
Checkpoints
Shows the total number of checkpoints executed in the batch and the number that are matching, mismatching, or new.
-
Matching: Checkpoints with no detected differences.
-
Mismatching: Checkpoints with detected differences.
Browsers
A bar chart that shows the number of batches executed in each browser as part of the batch sequence.
Devices
A bar chart that shows the number of tests rendered on each emulated device as part of the batch sequence.
Operating systems
A bar chart that shows the number of batches executed on each operating system as part of the batch sequence.
Viewports
A bar chart that shows the number of tests rendered for each viewport size in the batch sequence.
Test assignment
A bar chart that shows the number of tests that were assigned to each team member.
Issues and remarks
A bar chart that shows the number of issue and remark regions that are applied in steps that were checked as part of the batch sequence. The Issues and Remarks bars are color-coded to show the number of Issues or Remarks that are New or that were already Existing. If Issue or Remark regions were applied to a batch sampled at a point in time within the set Date Range, then those Issues and Remarks are listed as New.
Setting the Batch sequence name
To group batches into a Batch sequence Eyes requires that batches be associated with a Batch sequence name. The Batch sequence name can be set and associated with a batch in several ways.
-
You can set the Batch sequence name explicitly ins the SDK using the
setSequenceName. Do this when setting up the batch and before callingeyes.open -
If you do not set the Batch sequence name explicitly in the SDK and the environment variable APPLITOOLS_BATCH_SEQUENCE is defined then the SDK uses the value of that environment variable.
-
If you use an Applitools supported SCM or CI integrated plugin then it automatically sets the environment variable to the target branch. This results in batch statistics for each branch.
The following SCM and CI integrated plugins are supported:
Applitools supported SCM applications:
-
GitHub
-
GitLab
-
Atlassian Bitbucket
Applitools supported CI applications:
-
Microsoft Azure
-
Jenkins
-
Bamboo
No batch sequence information page
If no sequenced batches were run in the specified time period, for a given team, then Eyes cannot display any status information or batch statistics in the Batch Statistics tab.
In this case, the Batch Statistics tab displays information about batch sequences and a link to the documentation that describes how to define batch sequence names.
