Skip to main content

Learning how visual testing works

Visual Testing is just a special form of functional testing. With visual testing, assertions detect visual changes in the user interfaces (UIs) under test. Applitools uses Visual AI to find bugs like missing elements, skewed layouts, and overlapping text that are frequently missed by traditional automation techniques. The Ultrafast Grid and Native Mobile Grid also enable tests to run against any browser or mobile device at lightning speed in the cloud.

Let's learn the high-level concepts behind visual testing with Applitools. You will apply these concepts when running your first test.

Taking visual snapshots

Testing is interaction plus verification. Test case steps are merely procedures that do things and then make sure they work. With traditional assertions, a test typically needs to scrape elements from a page and then verify element attributes, like text values or visibility. A test needs multiple calls to check different elements. Unfortunately, element selectors can be complicated to write, and they break whenever the page changes. They also don't capture all the visual aspects of a page like layout, color, size, and position.

Visual snapshots are a much more thorough, robust way to test UIs. Instead of writing multiple lengthy assertions, you can write a one-line call to capture a visual snapshot of everything on the page. A snapshot is more than a static image: it captures all the HTML, CSS, and JavaScript that make up the page.

A visual test is simply a test that uses visual snapshots for its assertions. Applitools provides Eyes SDKs for capturing visual snapshots with several test tools and frameworks, such as Selenium, Cypress, and Playwright. The first time you run the test, Applitools Eyes will upload the initial snapshot as a baseline image to the cloud. Then, every time you run the test again, Applitools captures a fresh snapshot called a checkpoint image and uses Visual AI in the cloud to detect any meaningful differences.

The Applitools Eyes dashboard shows side-by-side comparisons for each snapshot. It highlights any differences in magenta.

Baseline vs Checkpoint

Understanding Visual AI

Visual AI is trained to detect differences that human eyes would see. It is much more robust than basic pixel-to-pixel comparisons, which generate lots of false positives for inconsequential changes.

Accepting or rejecting visual differences

Visual checkpoints yield results for human testers to evaluate. If no meaningful differences are detected between the baseline and checkpoint images, then the Applitools Eyes dashboard will mark the checkpoint as accepted or "Passed".

However, if a meaningful difference is detected, then the checkpoint will be marked as "Unresolved". Visual AI can detect changes, but it needs you, as the human testers, to decide if the change is a new feature or a real bug. You will need to explicitly accept or reject the change.

If you reject the unresolved checkpoint by clicking the thumbs-down icon (👎), then the dashboard will mark the checkpoint as "Failed". The dashboard will also automatically fail any future occurrences of the same rejected checkpoint image.

Rejecting a checkpoint difference

On the other hand, if you accept the unresolved checkpoint by clicking the thumbs-up icon (👍), then the dashboard will mark the checkpoint as "Passed", and the current checkpoint image will be used as the new baseline image for future comparisons.

Accepting a checkpoint difference

Remember to save results

Any time you accept or reject checkpoints in the dashboard, you must click the Save icon to save your results.

Leveraging the Applitools platform

Applitools Ultrafast Test Cloud offers multiple products to support automated visual testing.

All visual tests use Applitools Eyes to capture snapshots and perform comparisons. However, there are three ways to run tests:

  1. If you are testing web apps, you can perform cross-browser testing using Applitools Ultrafast Grid (UFG) by configuring browser types, versions, and viewports for rendering snapshots. You can target mobile browsers as well as desktop browsers. Your visual test runs one time on your machine, and then UFG rerenders captured snapshots on all the target browser configurations in the cloud.

  2. If you are testing native mobile apps for Android and iOS, you can perform cross-device testing using Applitools Native Mobile Grid (NMG). NMG is analogous to UFG: you run your mobile test once on your machine, and then NMG renders snapshots on multiple different device types and screen orientations.

  3. You can also run any visual tests the "classic" way entirely on your local machine using Applitools Classic runner. However, we recommend using UFG or NMG instead of the Classic runner for easier machine management, wider access to testable configurations, and faster execution times. The Classic runner renders snapshots using only the machine's local configuration.

The quickstart guides provide specific instructions for running visual tests with each of these platforms.

Choosing your platform

If you aren't sure which way to run tests, try Applitools Ultrafast Grid or Applitools Native Mobile Grid. They make cross-browser and cross-device testing easier to set up and faster to execute.