As the application under test evolves, so do its tests. New checkpoints are added to test new functionality and checkpoints are removed because the application functionality or flow has changed.
Instead of having a static association between every checkpoint and a corresponding baseline image, Eyes compares a sequence of images captured by checkpoints to a sequence of baseline images, allowing for the fact that checkpoints may have been removed or added. Then, very much like a textual "diff" program, Eyes find images that have been added and removed as well as images that have a corresponding baseline image and either match or mismatch. Eyes then gives the user the tools to both verify existing checkpoints as well as to ensure that the set of baseline images is up-to-date.
When a visual UI test executes, Eyes verifies the entire sequence of checkpoints against the sequence of baseline images. The result of this verification, the test result, consists of a sequence of steps, each denoting a missing checkpoint, a new checkpoint, a checkpoint that matches its baseline counterpart or one that differs from it.
For example, the baseline of a test may consist of the following five images:
Now imagine that the application and test have been changed, and the test generates a different sequence of images, for example:
Comparing the two sequences, we can see that:
Eyes will detect these differences, and it will generate a sequence of steps, as follows:
Each such step represents one of the four possible outcomes:
While you can freely add and remove checkpoints, Eyes assumes that the order of captured images that are not new does not change. If you change the order of two checkpoints, then depending on the content of these checkpoints, Eyes could report two missing steps and two new steps, or it could report two steps with mismatches.
When you define a checkpoint in the code, you can optionally provide a name for that checkpoint. The name is intended to be descriptive, so as to allow users to easily and conveniently navigate and search within the test results. Eyes does not take the name into account when matching checkpoint and baseline images. The name is optional, does not need to be unique, and you can freely change it without impacting the testability of the code.