Match levels

The match level determines the way by which Eyes compares the checkpoint image with the baseline image.

The sections of this article describe three aspects of using the match level:

Available match levels

Eyes provides the following distinct match levels:

Strict

This is the default and recommend match level.

This match level verifies that the page content matches the baseline image closely enough that the human eye would not see any difference. With this level, Eyes detects changes in text, font, color, graphics, and position of elements. It aims to detect differences that are visible to the human eye while ignoring differences in pixel values that are platform dependent due to the rendering software and hardware.

The Strict match level is most effective when used for regression tests on a particular browser/OS with mostly static content.

Layout

This level is useful for validating that the page layout is consistent.

With this match level, Eyes identifies the various page elements in the checkpoint and baseline images, such as text, images, buttons, and columns and verifies that the relative positions of these elements are consistent. This match level can detect elements that have appeared, disappeared, or moved. With this match level, the Eyes matching engine ignores differences in the actual content text and graphics, color, and other style changes.

This match level is most effective when used to validate pages with dynamic content, language localization, and cross-environment testing where a single baseline is used for multiple execution environments (different operating systems, browsers, devices, viewport sizes, and orientations).

Ignore colors

This match level is similar to the strict match level but ignores changes in colors. It is effective when your content is static but the color varies, for example if there are buttons or screen elements that can appear in a variety of colors.

Dynamic

This match level is used for textual verification, it recognizes defined text patterns such as a date, email address, or URL. For example, if text is defined as a date, Eyes confirms that the text is a valid date format, and would not identify a diff if the date changes, only if the date format is not valid. For more inflammation about this match level, see Dynamic match level

Exact

This match level causes the Eyes matching engine to do a pixel to pixel comparison of the checkpoint and baseline images. It is sensitive to differences such as rendering anomalies that are not visible to the human eye. It is not recommended for ordinary verification purposes.

The default match level is Strict.

How to specify a match level when running tests

There are several ways to set the match level, ranging from setting the match level for a number of tests to setting the match level of a specific region of one particular checkpoint:

  • Setting a match level for multiple tests – Use the method setMatchLevel to set a match level in a configuration object, and then use the method eyes.setConfiguration on each Eyes instance before calling the open method. This sets the default match level for all subsequent checkpoints in tests executed by that eyes instance.

  • Setting a match level for a series of checkpoints – Use a call to eyes.setMatchLevel at any point during a test, to change the default match level of all subsequent checkpoints on that eyes instance.

  • Setting the match level in a region of a checkpoint – Set the match level for a defined region or regions of a specific checkpoint using one of the following check Fluent API methods:

    1. strict(regions)

    2. layout(regions)

    3. ignorecolors(regions)

    4. exact(regions)

    Outside of these regions the page is matched using the match level set for that checkpoint, or if not defined, the checkpoint set for the eyes instance.

    For examples of how to set Dynamic match level, see Setting dynamic match level using code.

You can also set a match level region within a particular step using the Eyes Test Manager. This is then applied to all future test runs of the checkpoint for that step. For more information see Defining Regions.

Previewing the effect of a match level

You can preview the impact of a particular match level while reviewing test results in the Eyes Test Manager Step and Test editors. This feature allows you to view the results of a step in a test run at one match level, and to see what would have been the results had the test been run with a different match level. For more details see Preview the step images using match level options.

For further information, see our video, Setting Match Level Sensitivity in Applitools Eyes: