Skip to main content

Learning how Visual AI works

Visual Testing is just a special form of functional testing. With visual testing, assertions detect visual changes in the user interfaces (UIs) under test. Applitools uses Visual AI to find bugs like missing elements, skewed layouts, and overlapping text that are frequently missed by traditional automation techniques. The Ultrafast Grid enables tests to run against any browser or mobile device at lightning speed in the cloud, and the Execution Cloud enables you to automatically heal broken locators in your Selenium tests.

Let's learn the high-level concepts behind visual testing with Applitools. You will apply these concepts when running your first test.

Taking visual snapshots

Testing is interaction plus verification. Test case steps are merely procedures that do things and then make sure they work. With traditional assertions, a test typically needs to scrape elements from a page and then verify element attributes, like text values or visibility. A test needs multiple calls to check different elements. Unfortunately, element selectors can be complicated to write, and they break whenever the page changes. They also don't capture all the visual aspects of a page like layout, color, size, and position.

Visual snapshots are a much more thorough, robust way to test UIs. Instead of writing multiple lengthy assertions, you can write a one-line call to capture a visual snapshot of everything on the page. A snapshot is more than a static image: it captures all the HTML, CSS, and JavaScript that make up the page.

A visual test is simply a test that uses visual snapshots for its assertions. Applitools provides Eyes SDKs for capturing visual snapshots with several test tools and frameworks, such as Selenium, Cypress, and Playwright. The first time you run the test, Applitools Eyes will upload the initial snapshot as a baseline image to the cloud. Then, every time you run the test again, Applitools captures a fresh snapshot called a checkpoint image and uses Visual AI in the cloud to detect any meaningful differences.

The Eyes Test Manager shows side-by-side comparisons for each snapshot. It highlights any differences in magenta.

Baseline vs Checkpoint

Understanding Visual AI

Visual AI is trained to detect differences that human eyes would see. It is much more robust than basic pixel-to-pixel comparisons, which generate lots of false positives for inconsequential changes.

Accepting or rejecting visual differences

Visual checkpoints yield results for human testers to evaluate. If no meaningful differences are detected between the baseline and checkpoint images, then the Eyes Test Manager will mark the checkpoint as accepted or "Passed".

However, if a meaningful difference is detected, then the checkpoint will be marked as "Unresolved". Visual AI can detect changes, but it needs you, as the human testers, to decide if the change is a new feature or a real bug. You will need to explicitly accept or reject the change.

If you reject the unresolved checkpoint by clicking the thumbs-down icon (👎), then the Test Manager will mark the checkpoint as "Failed". The Test Manager will also automatically fail any future occurrences of the same rejected checkpoint image.

Rejecting a checkpoint difference

On the other hand, if you accept the unresolved checkpoint by clicking the thumbs-up icon (👍), then the Test Manager will mark the checkpoint as "Passed", and the current checkpoint image will be used as the new baseline image for future comparisons.

Accepting a checkpoint difference

Remember to save results

Any time you accept or reject checkpoints in the Test Manager, you must click the Save icon to save your results.

Leveraging the Applitools platform

Applitools Ultrafast Test Cloud offers multiple products to support automated testing.

There are some different ways to run tests:

  1. If you are running tests using Selenium WebDriver, you can run your WebDriver sessions remotely using Applitools Execution Cloud, which will automatically wait for elements to be ready and heal any broken locators. Execution Cloud works with both non-Eyes and Eyes tests.

  2. If you are running Eyes tests for web apps, you can perform cross-browser testing using Applitools Ultrafast Grid (UFG) by configuring browser types, versions, and viewports for rendering snapshots. You can target mobile browsers as well as desktop browsers. Your Eyes test runs one time on your machine, and then UFG rerenders captured snapshots on all the target browser configurations in the cloud.

  3. You can also run any Eyes tests the "classic" way entirely on your local machine using Applitools Classic runner. However, we recommend using UFG instead of the Classic runner for easier machine management, wider access to testable configurations, and faster execution times. The Classic runner renders snapshots using only the machine's local configuration.

The quickstart guides provide specific instructions for running tests with each of these platforms.

Choosing your platform

If you aren't sure which way to run web UI tests, try Applitools Ultrafast Grid. They make cross-browser and cross-device testing easier to set up and faster to execute.