Selenium Java tutorial
1. 🤖 How it works
Applitools SDKs works with existing test frameworks and simply takes screenshots of the page, element, region or an iframe and uploads them along with DOM snapshots to our Eyes server. Our AI then compares them with previous test executions' screenshots (aka Baselines) and tells if there is a bug or not. It's that simple!
1.1 Baseline vs. Checkpoint images
When you first run the test, our A.I. server simply stores those first set of screenshots as Baseline images. When you run the same test again (and everytime there after), the A.I. server will compare the new set of screenshots, aka Checkpoint images, with the corresponding Baseline images and higlights differences in pink color.
1.2 Marking the test as "Pass" or "Fail"
When the AI compares the baseline and the checkpoint image, if it finds a legitimate difference, it'll mark the test as Unresolved. This is because the AI doesn't know if the difference is because of a new feature or a real bug and will wait for you to manually mark it as a Pass/Fail for the 1st time.
If you mark the unresolved checkpoint image as a "Fail", any further runs with similar difference will be automatically marked as "Failed".
If you mark the unresolved checkpoint image as a "Pass", then it means that the difference is due to a new feature and so we update the new checkpoint image as the new baseline and mark the current test as Pass. And going forward we'll compare any future tests with this new baseline.
Applitools AI has been trained with 100s of millions of images. It doesn't do a pixel-by-pixel comparison because it leads to a lot of false positives, but instead simulates real human eyes and ignore normal differences that humans would ignore and only highlight those that humans would highlight as bugs.
ACCURACY: A.I's current accuracy rate is 99.9999%! Which means for most applications that odds that you'll see false-positives is 1 in a million!
A powerful test results dashboard
We provide a state-of-the-art dashboard that makes it very easy for you to analyze differences, report bugs straight from the dashboard and so on.
2. 🖼 Analyzing differences
The following Gifs shows various tools Applitools provide to easily analyze various differences
Highlight differences between the baseline and checkpoint
Zoom into diffs
Toggle differences between baseline and checkpoint
Show both baseline and checkpoint side-by-side
3. 🐞 Reporting bugs (straight into Jira or Github)You can simply select a section on the image and directly file a bug in Jira or Github. No need to manually take screenshots, write steps and explain things!
4. ✅ Prerequisites
Create a free Applitools account and get the Applitools API KEY
Have Java v10 or higher installed
Install Eclipse or IntelliJ editor
Install ChromeDriver on your machine and make sure it's is in the
Here are some resources from the internet that'll help you.
Tip: On Mac, place the
/usr/local/binfolder so Eclipse and IntelliJ can find it.
Install Maven. You can download the Maven binary Zip file from here.
- It should look something like
- Follow the installation instructions from here to add it to the
- It's better if you add it permanently to the environment so when you open a new Terminal the values will persist. Otherwise, you may have to redo it for everytime you open the Terminal. This means you should put it in the
~/.bash_profilefile (Mac) or in System variables in Windows. For more, see the Steps for adding
- The Maven executable is inside
/binfolder of the extracted Maven directory. So you must include
/bin. It should look something like:
Test your Maven setup
- Make sure to restart the Terminal or Command line prompt to load the new environment variables.
mvn -v. You should see something like below:
- It should look something like
Yoy may store the Applitools API Key in the environment variable. While this works for running the tests in the command line, if you are reading it from Eclipse or IntelliJ, then you need to launch those editors from the Terminal to read them. Otherwise, these editors won't load the environment variables!
Mac: export APPLITOOLS_API_KEY='YOUR_API_KEY'
Windows: set APPLITOOLS_API_KEY='YOUR_API_KEY'
5.1 🚀 - Run the existing demo app
- Get the code:
- Option 1:
git clone https://github.com/applitools/tutorial-selenium-java-basic.git
- Option 2: Download it as a Zip file and unzip it.
- Option 1:
- Import the folder as a Maven file in Eclipse or IntelliJ.
Make sure to set
APPLITOOLS_API_KEY(see prerequisits section).
Open Eclipse from Terminal so it can load the environment variables. On Mac, do something like:
You may need to force download Maven dependencies
- In Eclipse:
- Right click on the main Project folder > Maven > Update Project
- Select "Force Update of Snapshots/Releases" checkbox
- Press OK
- In Eclipse:
Change the Applitools API key in the
BasicDemo.java. Get one by logging into Applitools > Person Icon > My API Key
// Change the APPLITOOLS_API_KEY API key with yours: eyes.setApiKey("APPLITOOLS_API_KEY");1
Run the test
- Right click on the Project (or anywhere in the code) > Run As > JUnit Test
In Command line, run the following Maven command:
mvn -Dtest=BasicDemo test
After you run one set of tests, you now have the baselins. Run the same test but with this URL:
https://demo.applitools.com/index_v2.html. This version of the demo app has some visual bugs so you can see how it all works. And the second set of screenshots are called "Checkpoint images".
5.2 🤓 - Add Applitools to an existing project
1. Create a Maven project and add the following dependencies to the pom.xml
<!-- Applitools SDK --> <dependency> <groupId>com.applitools</groupId> <artifactId>eyes-selenium-java3</artifactId> <version>RELEASE</version> </dependency>
2. Import the SDK
import com.applitools.eyes.*; import com.applitools.eyes.selenium.ClassicRunner; import com.applitools.eyes.selenium.Eyes;
3. Initialize the SDK
// Initialize the Runner for your test. EyesRunner runner = new ClassicRunner(); // Initialize the eyes SDK Eyes eyes = new Eyes(runner); // Change the APPLITOOLS_API_KEY API key with yours eyes.setApiKey("APPLITOOLS_API_KEY");
4. Start the test
// Set AUT's name, test name and viewport size (width X height) // We have set it to 800 x 600 to accommodate various screens. Feel free to // change it. eyes.open(driver, "Demo App", "Smoke Test", new RectangleSize(800, 600)); // Navigate the browser to the "ACME" demo app. driver.get("https://demo.applitools.com");
5. Add eyes.checkWindow to take a screenshot
checkWindow or (similar methods such as
checkFrame) take a screenshot and upload it to the server for analysis. And since they are part of the test, each screenshot essentially becomes a test step. Note that you can add as many
check statements as you need after you open the tests.
// This line takes the screenshot eyes.checkWindow("Login Window");
6. End test
// End the test. eyes.closeAsync(); // Close the browser. driver.quit(); // If the test was aborted before eyes.close was called, ends the test as // aborted. eyes.abortIfNotClosed();
7. Get the test results
// Wait and collect all test results TestResultsSummary allTestResults = runner.getAllTestResults(); // Print results System.out.println(allTestResults);
6. 🚀 Try Visual Grid 🔥
In real world testing, you'll need to test your app on multiple browsers, viewports and mobile devices. So you'll need to run and re-run ALL the tests in different browsers and multiple viewports to ensure things are fine - which could take hours and hours to complete. You may also encounter browsers not opening properly, or hanging and what not.
What if you could run in just one browser, just once, and still do cross-browser tests across various browsers, viewports and multiple mobile emulators? That's where Visual Grid [BETA] comes in.
When you execute this tutorial, it'll run the test locally in Chrome browser but will show results in the following 5 combinations:
- Chrome browser (800 X 600 viewport)
- Chrome browser (700 X 500 viewport)
- Firefox browser (1200 X 800 viewport)
- Firefox browser (1600 X 1200 viewport)
- An iPhone4 emulator
... all in ~30 seconds!⏰🚀
Go through our Visual Grid tutorial!🚀