As the application being tested evolves, so do its tests. New checkpoints are added to test new functionality and checkpoints are removed because the application functionality or flow has changed.
Instead of having a static association between every checkpoint and a corresponding baseline image, Eyes compares a sequence of images captured by checkpoints to a sequence of baseline images, allowing for the fact that checkpoints may have been added or removed. Then, very much like a textual "diff" program, Eyes finds images that have been added or removed as well as images that have a corresponding baseline image and either match or mismatch. Eyes then enables you to both verify existing checkpoints and ensure that the set of baseline images is up to date.
When a visual UI test executes, Eyes verifies the entire sequence of checkpoints against the sequence of baseline images. The result of this verification - the test result - consists of a sequence of steps, each denoting a missing checkpoint, a new checkpoint, a checkpoint that matches its baseline counterpart, or one that differs from it.
For example, the baseline of a test may consist of the following five images:
Now, imagine that the application and test have been changed, and the test generates a different sequence of images. For example:
Comparing the two sequences, we can see that:
Eyes will detect these differences and generate a sequence of steps as follows:
Each step represents one of the four possible outcomes:
While you can freely add and remove checkpoints, Eyes assumes that the order of captured images that are not new does not change. If you change the order of two checkpoints, then depending on the content of these checkpoints, Eyes could report two missing steps and two new steps, or it could report two steps with mismatches.
When you define a checkpoint in the code, you can optionally provide a name for that checkpoint. The name is intended to be descriptive so as to allow you to easily and conveniently navigate and search within the test results. Eyes does not take the name into account when matching checkpoint and baseline images. The name is optional, does not need to be unique, and you are free to change it without impacting the testability of the code.