Leveraging Applitools for Seamless Visual Testing in Playwright

As applications become more complex and UI consistency becomes critical, ensuring that the user interface appears as expected across multiple environments is key. Applitools Eyes, when integrated with the Playwright SDK, provides a powerful, efficient, and streamlined approach to visual testing in your Playwright tests. The combination of Playwright, a popular end-to-end testing framework, and Applitools Eyes, a visual AI-powered testing tool, makes visual validation easier, faster, and more scalable.

Applitools Eyes is a visual AI-powered testing tool that helps address these UI consistency challenges. It uses advanced AI-driven image comparison to detect visual differences. Unlike traditional visual testing tools that perform pixel-by-pixel image comparisons, Applitools Eyes uses AI algorithms to determine if the visual differences are actual bugs.

Applitools Playwright SDK introduces several improvements designed to streamline the process of setting up and running visual tests, making the experience more efficient and user-friendly.

In this blog, we will discuss when it is appropriate to use visual testing with Playwright and which cases will be suitable for using Applitools.

Overview of Visual Testing

Visual testing is a technique performed to ensure the visual appearance of a given website or application matches the provided design and layout . This includes the comparison of the actual UI against a standard image or a reference UI image. This type of testing primarily focuses on detecting any visual anomalies, which include alignment issues , font problems, color discrepancies , images, or structural shifts that can disrupt the user experience.

How Visual Testing Differs from Functional Testing

Visual testing and functional testing are both crucial if the goal is to deliver a high-quality application; visual testing checks the visual appeal of the application, while functional testing verifies how the application functions.

Let’s see some major differences between visual and functional testing.

Aspect	Visual Testing	Functional Testing
Purpose	Ensures the layout and design of the UI look as expected.	Verifies that the application’s functions work as intended.
Scope	Focuses on UI elements’ appearance (position, dimensions, colors, fonts, etc.).	Focuses on the application’s behavior, verifying logic, workflows, and system responses.
Approach	Uses image comparison tools to detect visual discrepancies.	Uses test scripts or user simulations to validate functionality.
Techniques	Relies on image or screenshot comparison against baseline images.	Involves input/output verification, user interaction simulation, and data validation.
Tools	Tools like Playwright, Cypress, and other functional testing frameworks.	Tools like Playwright,Cypress, and other functional testing frameworks.
Issues Detected	Identifies layout errors, pixel misalignments, broken images, wrong colours, or responsive design bugs.	Identifies logical errors, broken workflows, malfunctioning features, or incorrect data processing.
Use Case	Suitable for detecting unintended UI changes during development.	Suitable for validating features and ensuring correct application logic.

Before delving into how we can perform visual testing using Playwright and Applitools, let’s review some of the new improvements that Applitools has introduced .

What’s New In Applitools Playwright SDK?

The Applitools Playwright SDK is a library that integrates Applitools’ visual testing capabilities with the Playwright automation framework.

Let’s explore the improvements done by Applitools to streamline the process of setting up and running visual tests, making the experience more efficient and user-friendly. 

Here’s a detailed breakdown of what’s new in the updated SDK:

Test Fixtures:

• Test fixtures Centralize and reuse setup code across multiple tests, making the setup for visual tests more consistent and reducing redundant configuration.
• This feature is particularly valuable when running tests on multiple pages or scenarios, as it simplifies the initialization of Applitools Eyes in each test.

CLI Onboarding:

• The Command-Line Interface (CLI) simplifies the initial setup process by automating configuration steps, helping users quickly integrate Applitools Eyes with Playwright. 
• This is especially helpful for new users, reducing the complexity of the setup and enabling faster onboarding into visual test creation.

Config Object Setup:

• The configuration object setup automates the insertion of Applitools Eyes settings into the Playwright configuration file (e.g., playwright.config.js or playwright.config.ts). 
• This feature eliminates manual setup, reducing the risk of errors, ensuring accurate configuration from the outset, and saving developers valuable time.

Custom HTML Reporter:

• The custom HTML reporter enhances Playwright’s default test report by integrating visual test results from Applitools Eyes. 
• This allows developers to view a direct comparison between the baseline image and the current test screenshot, helping identify visual regressions more effectively. 
• By adding visual results to the standard Playwright report, it simplifies the review process, offering a comprehensive overview of the test outcomes.

Before we deep dive into how Applitools helps in visual testing, let’s first see how we can perform visual testing using Playwright.

Visual Testing Using Playwright

Playwright comes equipped with tools to make visual testing effortless, allowing developers to take and compare screenshots of web pages or specific web page elements. 

Here’s how visual testing works in Playwright:

1. First Run: Playwright captures and saves a screenshot of the page or specific UI elements. This is known as the base image.
2. Subsequent Runs: In the next run, Playwright takes a new screenshot and compares it against the base image.
   • If there are no differences, the test passes.
   • If differences exist, the test fails, flagging the areas where changes occurred.

Prerequisites

Ensure the following tools are installed:

Node.js: Download and install from Node.js.

Visual Studio Code (Optional): Recommended IDE for coding.

Install Playwright:

npm i @playwright/test

Write the Simple Visual Regression Test Using Playwright
Create a new JavaScript file in your test folder, e.g., demo.spec.js.

Use the page.screenshot() method to capture the screenshot and the expect.toMatchSnapshot() assertion from the @playwright/test module to compare images.

const { test, expect } = require('@playwright/test'); 
test('Visual Regression Test Example', async ({ page }) => { 
   // Navigate to the website 
   await page.goto('https://playwright.dev'); 
   // Capture a screenshot of the page 
   const screenshot = await page.screenshot(); 
   expect(screenshot).toMatchSnapshot(); 
}); 

When we execute the above code for the first time, the test case fails with the below error, and Playwright captures and saves a screenshot of the page or specific UI elements. This is known as the base image.

When we execute the same test case again, it will pass.

Visual Regression Test with maxDiffPixels option 

There might be the case when there are some minor differences in the images and the test case starts failing; to handle this situation we have an option, the ‘maxDiffPixels option,’ that we can pass. It allows you to specify the maximum number of pixels that can differ between two images for the comparison to still be considered successful.

const { test, expect } = require('@playwright/test');
test('Visual Regression Test Example', async ({ page }) => {
  // Navigate to the website
  await page.goto('https://playwright.dev');
  // Capture a screenshot of the page
  const screenshot = await page.screenshot();
  expect(screenshot).toMatchSnapshot({ maxDiffPixels: 100 });
}); 

const { test, expect } = require('@playwright/test');
test('Visual Regression Test Example', async ({ page }) => {
  // Navigate to the website
  await page.goto('https://playwright.dev');
  // Capture a screenshot of the page
  const screenshot = await page.screenshot();
  expect(screenshot).toMatchSnapshot({ threshold:0.5} );
}); 

Visual Regression Test with Threshold option

Passing ‘threshold’ as an option is another option to avoid failing the test case. This is particularly useful for handling slight differences that occur due to rendering variations.

const { test, expect } = require('@playwright/test');
test('Visual Regression Test Example', async ({ page }) => {
  // Navigate to the website
  await page.goto('https://playwright.dev');
  // Capture a screenshot of the page
  const screenshot = await page.screenshot();
  expect(screenshot).toMatchSnapshot({ threshold:0.5} );
}); 

So far we have seen how we can automate the visual UI using Playwright. In the next section you will see how we can use Applitools Eyes and Playwright SDK together to automate visual testing.

Visual Testing using Applitools Eyes and Playwright SDK

Visual testing is a crucial aspect of modern software quality assurance. It ensures that a web application’s UI renders correctly across different devices, screen sizes, and browsers. Tools like Applitools Eyes and frameworks like Playwright SDK simplify this process by providing robust solutions for automated visual regression testing.

Below are the steps to set up Applitools Eyes and Playwright SDK.

Install Playwright

npm init playwright@latest

Install Applitools Eyes SDK 

npm install -D @applitools/eyes-playwright

Add the below line in .spec.js file 

To configure your project for Applitools Eyes, we have to add the below line.

import { test } from '@applitools/eyes-playwright/fixture';

In the next section you will see examples to explain how we can use SDK with Playwright tests. 

Before moving in the detail of various types of matchLeavel, let see one basic example to understand how we do the visual testing in Applitool,  and the method used in Applitools, to capture and validate a checkpoint in your application’s UI.

Below is code that integrates Applitools Eyes with Playwright to perform a visual test.

import { test } from '@applitools/eyes-playwright/fixture';
test('My first visual test Using Applitools with matchLevel: Dynamic', async ({ page, eyes }) => {
await page.goto('https://applitools.com/helloworld/');
  // Visual check
  await eyes.check('Landing Page', {
    fully: true,
    matchLevel: 'Dynamic',
  });
});

In the above code we mainly use two things: one is the method eyes. check() and two options: {fully: true}, and {matchLevel: ‘Dynamic’}

eyes.check(): Performs a visual snapshot of the page or a specific element and compares it with the baseline image stored in Applitools.

Options:

• fully: true:
  • Captures the entire page, including scrollable sections.
  • Ensures that all content on the page is included in the visual test.
• matchLevel: ‘Dynamic’:
  • A matching algorithm that focuses on the content while ignoring dynamic changes, such as text or layout differences.
  • Useful for pages with frequently changing content, like user-generated text or dynamic data.

Below are the options that we can pass in our test case. These options provide flexibility and precision, enabling robust visual testing tailored to different scenarios and challenges.

This table summarizes each configuration, making it easier to understand their purposes.

Option	Purpose	Example Use Case
matchLevel: ‘Dynamic’	Ignores minor layout or content changes (e.g., dynamic text, animations).	Testing pages with frequent dynamic content (e.g., dates, real-time updates) without raising false alarms.
matchLevel: ‘Strict’	Detects even small pixel or layout changes for precise visual validation.	Validating critical UI components or designs where every pixel matters (e.g., brand logos, product pages).
region: component	Focuses the visual check on a specific component or region of the page.	Testing a new or modified UI component in isolation (e.g., buttons, headers, or form fields).
fully: true	Captures and validates the entire page, including content outside the viewport.	Ensuring the layout and content of a long webpage or scrolling section is consistent.
ignoreRegions: [dynamicContent]	Excludes specific regions with dynamic content from the validation.	Ignoring areas like ads, live feed sections, or widgets with fluctuating content (e.g., real-time graphs).

Examples with different matchLevel

MatchLevel determines how the captured visual output is compared with the baseline image.Let’s see the examples with different options.

1. Visual Testing with matchLevel: Strict

This ensures pixel-perfect comparison between the baseline and current screenshot, highlighting any visual difference, no matter how small.

test('My first visual test Using Applitools with matchLevel: Strict', async ({ page, eyes }) => {
  await page.goto('https://applitools.com/helloworld/');
  await eyes.check('Landing Page', {
    fully: true,
    matchLevel: 'Strict',
  });
});

• Purpose: Validates the entire page with Strict match level.
• Strict Match Level: Identifies even small differences, such as minor pixel or layout changes, ensuring high precision in visual validation.

2. Visual Testing with matchLevel: Dynamic

Focuses on content consistency by ignoring minor layout differences, such as text movement, while ensuring key visual elements remain unchanged.

In the below example you can see even if the text color is changed after clicking on link ‘?diff2’ test case still works perfectly fine because we have set matchLevel: Dynamic. This ensures the visual test focuses on the content and structure of the text rather than superficial changes like color.

test('My first visual test Using Applitools with matchLevel: Dynamic', async ({ page, eyes }) => {
 await page.goto('https://applitools.com/helloworld/');
 await page.getByRole('link', { name: '?diff2' }).click();
 await eyes.check('Homepage', {
   fully: true,
   matchLevel: 'Dynamic',
 });
});

• Purpose: Validates the entire page with Dynamic match level.
• Dynamic Match Level: Ignores minor content/layout differences (e.g., text changes, animations) and focuses on high-level structural comparisons.
• fully: true: Captures a screenshot of the full page, not just the visible viewport.

3. Visual Testing for a Specific Region/Component

Limits visual comparisons to a particular area or component of the application.

test('My first visual test Using Applitools with particular region/Component', async ({ page, eyes }) => {
  await page.goto('https://applitools.com/helloworld/');
  const component = page.locator('.fancy.title.primary');
  await eyes.check('My Component', {
    region: component,
  });
});

• Purpose: Tests only a specific component or region on the page.
• region: Focuses the validation on the area defined by the component locator (.fancy.title.primary), instead of the entire page.

4. Visual Testing with Ignored Regions

Excludes specific areas from comparison, preventing false positives caused by expected dynamic changes in those regions.

test('Visual test Using Applitools with ignoring the region', async ({ page, eyes }) => {
  await page.goto('https://applitools.com/helloworld/');
  const dynamicContent = page.locator('.fancy.title.primary');
  await eyes.check('Homepage', {
    fully: true,
    matchLevel: 'Strict',
    ignoreRegions: [dynamicContent],
  });
});

• Purpose: Tests the page while ignoring specific dynamic regions.
• ignoreRegions: Excludes certain areas from the visual validation (e.g., areas with dynamic content like dates, ads, or animations).
• fully: true: Captures the entire page for validation.

About dynamic match level

Dynamic match level is a feature in Applitools Eyes that verifies text by matching it against predefined or custom patterns, ensuring the content adheres to a specific format. It focuses on format validation rather than content changes, making it ideal for dynamic text like dates, emails, or numbers.

Type of dynamic match level

Dynamic match levels allow for flexible verification of text by matching it against predefined or custom patterns. The default types include:

• TextField: Text inside input fields.
• Number: Numeric values like ZIP codes or phone numbers.
• Date: Validates text as a date in proper format.
• Link: Hyperlinks or URLs.
• Email: Checks for a valid email address format.
• Currency: Recognizes monetary values.

These types ensure accurate validation by focusing on the format rather than the content changes, such as dynamic updates to dates or numbers.

Custom dynamic pattern

To create a custom dynamic pattern we have to follow below steps in Applitools dashboard 

Here are the steps to create a custom dynamic pattern:

1. Open the Settings Window: In the Page Navigator, select Apps & Tests.

In the list of applications on the left, hover over an application and click > Settings.

2. Add a Custom Type: Click Add custom type.

3. Define the Custom Pattern:

Enter a name and a Regex pattern.

4. Save the Custom Pattern:

In the below screenshot you can see we have set a custom pattern for Zip Code. Once the custom pattern is created, its name can be used in the code to reference the pattern.

Click Add will save the entered pattern. 

5. Apply the Settings:

Once all required patterns have been selected, click Apply settings.

Applitools Dashboard: Execute visual testing using the Applitools Playwright SDK

Applitools’ Dashboard is a powerful interface designed to manage and analyze test results efficiently, especially for visual testing and monitoring user interfaces.

Below are the steps that we normally have in the Applitools Dashboard when we execute any visual test case.

Precondition : Run the below command to export the key 

export APPLITOOLS_API_KEY='your_api_key_here'

Once the key is exported, the next step is to execute the test cases using the below command.

npx playwright test --ui tests/visual.spec.js

First Run: Playwright captures and saves a screenshot of the page or specific UI elements. This is known as the base image.

Subsequent Runs: In the next run, Playwright takes a new screenshot and compares it against the base image.

• If there are no differences, the test passes.
• If differences exist, the test fails, flagging the areas where changes occurred.

In the below screenshot in the Applitools Dashboard, you can see all the above four test cases are executed successfully.

Applitools Playwright SDK with Example in Detail

Let’s take an example of a particular component in the page. In the below screenshot, the component focuses on a specific component or region of the page. You can see only the ’Hello World’ part is tested. In case there is any change in this component, the test case will fail.

In the below example we have updated the component with the text ‘Happy World’ 

test('Visual test Using Applitools with Updating Component UI', async ({ page, eyes }) => {
await page.goto('https://applitools.com/helloworld/');
await page.getByRole('link', { name: '?diff2' }).click();
await eyes.check('Homepage', {
  fully: true,
  matchLevel: 'Strict',
 });
});

When we execute the above code, it will not execute successfully because the UI of the component is changed after clicking on the link ‘diff2.’

To fix the above problem, we have marked it resolved from the Applitools Dashboard. Once we accept the change, it becomes the base image.

Now when we execute the test case again, it gets executed successfully with the updated UI.

When Playwright’s Visual Testing Features Are Useful 

Playwright’s built-in visual testing features are well-suited for scenarios where:

Simple Visual Comparisons:

• Comparing entire pages or specific UI components with baseline images for detecting pixel-level differences.
• Example: Ensuring the homepage layout hasn’t shifted after a CSS update.

Static UIs:

• Testing applications with minimal dynamic or animated content, as these are less prone to false positives caused by animations or transient elements.

Fast Feedback:

• When quick pixel-by-pixel comparisons in CI pipelines are sufficient without advanced AI-powered analysis.
• Example: Smoke testing in a fast-moving development cycle.

Budget-Friendly:

• Playwright’s native screenshot and comparison capabilities don’t require additional tools or subscriptions.

When Playwright’s Visual Testing May Fall Short

Dynamic Content:

• UIs with dynamic or frequently changing elements (e.g., time, randomized content, or animations) can cause false positives due to exact pixel mismatches.
• Example: A dashboard with live-updating graphs.

Complex Visual Changes:

• Playwright’s threshold-based comparison may miss subtle or semantic changes that don’t exceed the defined pixel difference threshold.
• Example: Slightly altered typography or colour changes.

Ignored Regions:

• While Playwright allows element-specific testing, ignoring specific dynamic regions within a larger page requires custom logic.
• Example: Excluding advertisements or live widgets from visual comparison.

How Applitools Handles Situations Better

Applitools offers superior visual testing capabilities compared to Playwright, especially when handling dynamic content, ensuring pixel-perfect validation, focusing on specific components, and reducing false positives. 

Dynamic Content Handling:

• Match Levels: Applitools’ AI-powered matchLevel options (e.g., Strict, Layout, Content) allow intelligent comparison by focusing on structure and ignoring irrelevant pixel differences.
• Example: Testing a real-time dashboard with dynamic data but a consistent layout.

Ignored Regions:

• Applitools allows you to define ignored regions to exclude specific parts of a page from comparison (e.g., ads or timestamps).
• Example: Excluding a “current time” widget from visual testing.

Advanced Visual Validation:

• Features like dynamic regions and AI-based semantic analysis help detect visual regressions that go beyond pixel-by-pixel differences.
• Example: Identifying a button misalignment that might not be detected by Playwright’s threshold settings.

Visual Testing Across Components:

• Allows you to target specific regions or components for testing without manually cropping or capturing screenshots.
• Example: Testing the styling of a navigation bar or modal dialog.

Conclusion

Playwright’s built-in visual testing features are suitable for simple cases where you need pixel precision in image comparisons of static UIs. However, it may struggle with complicated designs, whether they involve minor changes within the site or more intricate design requirements .

Applitools Eyes, which employs artificial intelligence, is more suitable for overcoming these difficulties. It performs extraordinarily in cases with dynamic content; it easily identifies subtle visual regressions and is equipped with extra functionalities such as ignored areas and cross-browser testing. For teams with significant testing and enhancement requirements, where the built-in visual testing capabilities of Playwright may fall short for complex and large-scale application testing, Applitools serves as an invaluable complement. It ensures and elevates the visual quality of dynamic applications, making it an ideal choice for robust testing and quality assurance.

About the Author

Kailash Pathak (Applitools Ambassador | Cypress Ambassador)

Senior QA Lead Manager with over 15 years of experience in QA engineering and automation. Kailash holds certifications including PMI-ACP®, ITIL®, PRINCE2 Practitioner®, ISTQB, and AWS (CFL).

As an active speaker and workshop conductor, Kailash shares his expertise through blogs on platforms like Medium, Dzone, Talent500, The Test Tribe, and his personal site https://qaautomationlabs.com/