This quickstart will show you how to visually test web apps in Python using Playwright. Visual testing can help you catch problems that traditional automation struggles to find. You can also leverage Applitools Ultrafast Grid to run your tests across all the major browsers in a fraction of the time as other cross-browser testing platforms.
After preparing your environment, this three-step quickstart should take about 15 minutes to complete.
If you get stuck on this example, don't suffer in silence! Please reach out to us to get things working. We can also help you get Applitools running in your own project.
Step 1: Preparing your environment
You'll need a few things to run this quickstart:
An Applitools account, which you can register for free.
A recent version of Python 3.
Step 2: Getting your example project
Downloading the example project
The example project is located at https://github.com/applitools/example-playwright-python. Clone this repository to your local machine:
git clone https://github.com/applitools/example-playwright-python.git
Instead of running
git clone, you can download the project as a ZIP file and extract it.
Installing the dependencies
The example project uses pip for package management. For Playwright, you must install project dependencies and browsers. Run the following commands (which work for both example projects on any operating system):
pip3 install -r requirements.txt
This example project already declares the Applitools Eyes SDK as a dependency. If you want to add the Applitools Eyes SDK into an existing Python environment, run the following commands:
pip3 install eyes-playwright
Deciding how to run tests
There are two ways to test the visual snapshots captured by the test:
- Using Applitools Ultrafast Grid for cross-browser testing in the cloud
- Using Applitools Classic runner on your local machine
If you are not sure which one to pick, read Leveraging the Applitools platform. For most cases, we recommend Applitools Ultrafast Grid. The docs for Ultrafast Grid and Classic runner provide more information.
Walking through the code
All test setup and cleanup is handled by a set of pytest fixtures located at
The variables at the top control how tests will run:
Trueto use Ultrafast Grid or
Falseto use the Classic runner.
Setup varies slightly for these different options.
The project contains one visual test case, which is located at
test_acme_bank is a pytest test module that covers login behavior for the ACME Bank demo web app.
It uses the Applitools Eyes SDK
to execute the test across multiple browsers in Applitools Ultrafast Grid.
With the Ultrafast Grid, you can specify different browser types, viewport sizes, and even mobile devices.
Step 3: Running your tests
Setting Applitools variables
Before running the visual test,
you must find your Applitools API key
and set it as an environment variable named
You may set it through your IDE (if applicable),
or you may set it from the command line like this:
- macOS and Linux
If you have trouble setting the
APPLITOOLS_API_KEY environment variable, you can hard-code your API key like this:
# Change the following line in the 'api_key' fixture:
# return os.getenv('APPLITOOLS_API_KEY')
However, be warned: hard-coding secrets is poor practice. Do this only temporarily for debugging, and never commit hard-coded secrets to version control.
You may also need to set your Applitools Eyes server.
By default, tests will use the public Applitools Eyes server at eyes.applitools.com.
However, if your team is using a private Applitools Eyes server,
you can target it by setting the
APPLITOOLS_SERVER_URL environment variable.
(If you are using a free Applitools account, then use the public server.)
Launching visual tests
Run the following command to launch tests:
python3 -m pytest -s -v tests
By default, tests run headlessly.
To run them in headed mode, add the
When you run tests with the Applitools Ultrafast Grid, the tests will run one time on the local machine, and then they will upload snapshots to render on each target configuration in the cloud. The Test Manager will show a separate result for each rendering. When you run tests with the Applitools Classic runner, the Test Manager will show the one snapshot from your local machine.
You can also change the web page to inject visual bugs:
If you rerun the tests, then they should yield "unresolved" results for you to review. Visual differences will be highlighted in magenta. It's up to you to accept (👍) or reject (👎) the changes. Applitools will remember your decisions for future analysis.
Again, it's okay. If you get stuck on this example, don't suffer in silence! Please reach out to us to get things working. We can also help you get Applitools running in your own project.
Taking the next steps with Applitools
Congratulations on completing this quickstart! There's still so much to learn about visual testing with Applitools, but you're off to a great start.
Resources for next steps:
- 🤖 Learning how visual testing works
- ↔️ Setting match levels for visual checkpoints
- 💥 Troubleshooting common issues
- 🐞 Reporting bugs
- 🗺 Detailed overview of visual testing with Applitools