Selenium Java tutorial
1. 🤖 How it works
Applitools SDKs work with existing test frameworks to take screenshots of pages, elements, regions or iframes and upload them along with DOM snapshots to our Eyes server. Our AI then compares them with previous test executions' screenshots (aka Baselines) and reports if there is a bug or not. It's that simple!

1.1 Baseline vs. Checkpoint images
When you first run the test, our A.I. server stores those first set of screenshots as Baseline images. When you run the same test again (and everytime there after), the A.I. server compares the new set of screenshots, aka Checkpoint images, with the corresponding Baseline images and highlights differences in a pink color.

1.2 Marking the test as "Pass" or "Fail"
When the AI compares the baseline and the checkpoint image, if it finds a legitimate difference, it will mark the test as Unresolved. This is because the AI doesn't know if the difference is because of a new feature or a real bug and will wait for you to manually mark it as a Pass/Fail for the 1st time.
If you mark the Unresolved checkpoint image as "Failed", it'll only mark the current test result as Failed.

Note:
To automatically mark the checkpoint as a "Fail" in the future test runs, you need to do the following:
- Annotate at least one of differences as a "bug region"
- Select the "Fail tests" checkbox in the popup window
- Press "Thumbs Up" (not "Thumbs Down") button in the checkpoint image's toolbar (Note: this is counter-intuitive. But what happens is that, we now create a new Baseline along with this bug and "Failed" metadata. So if the same image with the exact bug appears, it'll fail again)
- Press "Save" in the main toolbar
If you mark the Unresolved checkpoint image as a "Pass", then it means that the difference is due to a new feature so we set the new checkpoint image as the new baseline and mark the current test as Pass. Going forward we'll compare any future tests with this new baseline.

Note:
Applitools AI has been trained with 100s of millions of images. It doesn't do pixel-to-pixel comparison as this can lead to a lot of false positives. It instead simulates human eyes that ignore differences that humans can't detect and highlight differences that humans can detect.
ACCURACY: Our A.I.'s current accuracy rate is 99.9999%! Which means for most applications the odds that you'll see false-positives are 1 in a million!
A powerful test results dashboard
We provide a state-of-the-art dashboard that makes it very easy for you to analyze differences, report bugs and much more. For more information on the Applitools dashboard check out these articles.

2. 🖼 Analyzing differences
The following Gifs show various tools Applitools provides to easily analyze various differences
Highlight differences between the baseline and checkpoint

Zoom into differences

Toggle between baseline and checkpoint

Show both the baseline and checkpoint side-by-side

3. 🐞 Reporting bugs (straight into Jira or Github)
You can select a section of the image and directly file a bug in Jira or Github. No need to manually take screenshots, write steps and explain things! To read more about bug regions check out this article.

4. ✅ Prerequisites
Create a free Applitools account and get the Applitools API KEY
Have Java v8 or higher installed
Install Eclipse or IntelliJ editor
Install ChromeDriver on your machine and make sure it's is in the
PATH
.Here are some resources from the internet that'll help you.
Tip:
On Mac, place the
chromedriver
executable in/usr/local/bin
folder so Eclipse and IntelliJ can find it.Install Maven. You can download the Maven binary Zip file from here.
- It should look something like
apache-maven-3.5.4-bin.zip
- Follow the installation instructions from here to add it to the
PATH
.
TIP
- It's better if you add it permanently to the environment so when you open a new Terminal the values will persist. Otherwise, you may have to redo it for everytime you open the Terminal. This means you should put it in the
~/.bash_profile
file (Mac) or in System variables in Windows. For more, see the Steps for addingChromedriver
to thePATH
below. - The Maven executable is inside
/bin
folder of the extracted Maven directory. So you must include/bin
. It should look something like:/Users/raja/apps/apache-maven-3.5.4/bin
Test your Maven setup
- Make sure to restart the Terminal or Command line prompt to load the new environment variables.
- Run
mvn -v
. You should see something like below:
- It should look something like
Note:
Yoy may store the Applitools API Key in the environment variable. While this works for running the tests in the command line, if you are reading it from Eclipse or IntelliJ, then you need to launch those editors from the Terminal to read them. Otherwise, these editors won't load the environment variables!
Mac: export APPLITOOLS_API_KEY='YOUR_API_KEY'
Windows: set APPLITOOLS_API_KEY='YOUR_API_KEY'
5. 🚀 Run Tests
There are two main ways in which you can run tests.
The Applitools Ultrafast Grid (option 1) Test across all browsers and viewports 30-70 times faster than traditional test clouds. Using this approach, your tests will execute on a local browser once with the DOM, CSS, and additional artifacts captured by the SDK and sent to the Ultrafast Grid for parallel rendering and Visual AI comparison. This is the preferred execution method, however please note that it is a premium feature.
The second approach is to Run Local (option 2). This is the classic test execution in which your tests execute locally and each call to eyes via the SDK will send a screenshot to Eyes for Visual AI comparison.
While running tests using the classic approach is a great start - we recommend running tests on the Ultrafast Grid, the fastest way to ensure quality and consistency across all browsers, viewports, and devices.
Option 1 - Run With The Ultrafast Grid
1. Introduction To the Ultrafast Grid
In real world testing, you'll need to test your app on multiple browsers, viewports and mobile devices. So you'll need to run and re-run ALL the tests in different browsers and multiple viewports to ensure things are fine - which could take hours and hours to complete. You may also encounter browsers not opening properly, or hanging and what not.
What if you could run in just one browser, just once, and still do cross-browser tests across various browsers, viewports and multiple mobile emulators? That's where Ultrafast Grid comes in.
TL;DR
When you execute this tutorial, it'll run the test locally in Chrome browser but will show results in the following 5 combinations:
- Chrome browser (800 X 600 viewport)
- Chrome browser (700 X 500 viewport)
- Firefox browser (1200 X 800 viewport)
- Firefox browser (1600 X 1200 viewport)
- An iPhone4 emulator
... all in ~30 seconds!⏰🚀
2. What Is The Ultrafast Grid?
The Ultrafast Grid provides an elegant and an ultra fast way to render pages in different browsers, viewports and mobile simulators and then take screenshots and then send those screenshots to Applitools AI server. With Ultrafast Grid you just need to run your test in a single browser but perform end-to-end and cross-browser tests in various browsers such as Chrome,Firefox, IE, and also numerous combinations of viewports and in mobile simulators -- all in under a minute. Without Ultrafast Grid, you'll be wasting a lot of time and money running those tests in multiple browsers which could take hours and are error-prone.
And since Ultrafast Grid is integrated with our Eyes AI, you'll actually find a lot more functional and visual bugs than you'd normally find.
Further, you can use our Root-Cause-Analysis (RCA) feature to figure out exactly which DOM element or CSS caused difference! So you can go backwards, from the screenshot to the code!
Best part: All you need to do is to simply make eyes.check
calls in your existing Selenium tests after adding our SDK and instantiating Ultrafast Grid. And that's all there is to it!
How it works
Once you add and instantiate the Applitools SDK, simply add eyes.check
calls wherever you want to check if the state of the page looks OK. Then run the Selenium test as usual. And while running the tests, anytime it encounters eyes.check
(or similar methods), Applitools SDK uploads the DOM resource of that page to our Ultrafast Grid behind the scenes and in batches. The Ultrafast Grid then renders every page in different browsers and mobile emulators (in parallel) and takes screenshots as per the configuration you set. Finally, it sends those screenshots to our AI Server.
When you first run the test, our AI server simply stores those 1st set of screenshots as Baseline images. When you run the same test again, the AI server will compare the new set of screenshots, aka Checkpoint images, with the corresponding Baseline images and highlights bugs.

3. Getting Started With The Ultrafast Grid
Pre-requisites:
- Have Java 8 or higher installed
- Install Eclipse or IntelliJ editor
- Chrome Webdriver is on your machine and is in the
PATH
. Here are some resources from the internet that'll help you.
Option 1: Github Repo
Get the ready-to-run code from the Github repo:
git clone https://github.com/applitools/tutorial-selenium-java-visualgrid.git
,- or, download it as a Zip file and unzip it.
Open the folder as a Maven file in Eclipse or IntelliJ.
Change the Applitools API key in the
AppTest.java
. Get the API key by logging into Applitools > Person Icon > My API KeyRun the test
Option 2: Manual
1. Create a Maven project and add the following properties and dependencies to the pom.xml
<dependencies>
<!-- This is the Applitools Selenium Java SDK -->
<dependency>
<groupId>com.applitools</groupId>
<artifactId>eyes-selenium-java3</artifactId>
<version>RELEASE</version>
</dependency>
</dependencies>
2
3
4
5
6
7
8
9
AppTest.java
and add the following code. And replace the APPLITOOLS_API_KEY
with your own.
2. Create a class with name package com.applitools.quickstarts;
import com.applitools.eyes.BatchInfo;
import com.applitools.eyes.RectangleSize;
import com.applitools.eyes.TestResultsSummary;
import com.applitools.eyes.selenium.BrowserType;
import com.applitools.eyes.selenium.Configuration;
import com.applitools.eyes.selenium.Eyes;
import com.applitools.eyes.selenium.fluent.Target;
import com.applitools.eyes.visualgrid.model.DeviceName;
import com.applitools.eyes.visualgrid.model.ScreenOrientation;
import com.applitools.eyes.visualgrid.services.VisualGridRunner;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
/**
* Unit test for simple App.
*/
public class AppTest {
public static void main(String[] args) {
// Create a new chrome web driver
WebDriver webDriver = new ChromeDriver();
// Create a runner with concurrency of 1
VisualGridRunner runner = new VisualGridRunner(1);
// Create Eyes object with the runner, meaning it'll be a Visual Grid eyes.
Eyes eyes = new Eyes(runner);
setUp(eyes);
try {
// ⭐️ Note to see visual bugs, run the test using the above URL for the 1st run.
// but then change the above URL to https://demo.applitools.com/index_v2.html
// (for the 2nd run)
ultraFastTest(webDriver, eyes);
} finally {
tearDown(webDriver, runner);
}
}
public static void setUp(Eyes eyes) {
// Initialize eyes Configuration
Configuration config = new Configuration();
// You can get your api key from the Applitools dashboard
config.setApiKey("APPLITOOLS_API_KEY");
// create a new batch info instance and set it to the configuration
config.setBatch(new BatchInfo("Ultrafast Batch"));
// Add browsers with different viewports
config.addBrowser(800, 600, BrowserType.CHROME);
config.addBrowser(700, 500, BrowserType.FIREFOX);
config.addBrowser(1600, 1200, BrowserType.IE_11);
config.addBrowser(1024, 768, BrowserType.EDGE_CHROMIUM);
config.addBrowser(800, 600, BrowserType.SAFARI);
// Add mobile emulation devices in Portrait mode
config.addDeviceEmulation(DeviceName.iPhone_X, ScreenOrientation.PORTRAIT);
config.addDeviceEmulation(DeviceName.Pixel_2, ScreenOrientation.PORTRAIT);
// Set the configuration object to eyes
eyes.setConfiguration(config);
}
public static void ultraFastTest(WebDriver webDriver, Eyes eyes) {
try {
// Navigate to the url we want to test
webDriver.get("https://demo.applitools.com");
// Call Open on eyes to initialize a test session
eyes.open(webDriver, "Demo App", "Ultrafast grid demo", new RectangleSize(800, 600));
// check the login page with fluent api, see more info here
// https://applitools.com/docs/topics/sdk/the-eyes-sdk-check-fluent-api.html
eyes.check(Target.window().fully().withName("Login page"));
webDriver.findElement(By.id("log-in")).click();
// Check the app page
eyes.check(Target.window().fully().withName("App page"));
// Call Close on eyes to let the server know it should display the results
eyes.closeAsync();
} finally {
eyes.abortAsync();
}
}
private static void tearDown(WebDriver webDriver, VisualGridRunner runner) {
// Close the browser
webDriver.quit();
// we pass false to this method to suppress the exception that is thrown if we
// find visual differences
TestResultsSummary allTestResults = runner.getAllTestResults(false);
System.out.println(allTestResults);
}
}
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
Option 2 - Run Locally
- Get the code:
- Option 1:
git clone https://github.com/applitools/tutorial-selenium-java-basic.git
- Option 2: Download it as a Zip file and unzip it.
- Option 1:
- Import the folder as a Maven file in Eclipse or IntelliJ.
TIP
Make sure to set
APPLITOOLS_API_KEY
(see Prerequisites section).Open Eclipse from Terminal so it can load the environment variables. On Mac, do something like:
open /Applications/Eclipse.app
You may need to force download Maven dependencies
- In Eclipse:
- Right click on the main Project folder > Maven > Update Project
- Select "Force Update of Snapshots/Releases" checkbox
- Press OK
- In Eclipse:
Change the Applitools API key in the
BasicDemo.java
. Get one by logging into Applitools > Person Icon > My API Key// Change the APPLITOOLS_API_KEY API key with yours: eyes.setApiKey("APPLITOOLS_API_KEY");
1
2Run the test
In Eclipse:
- Right click on the Project (or anywhere in the code) > Run As > JUnit Test
In Command line, run the following Maven command:
mvn -Dtest=BasicDemo test
After you run one set of tests, you now have the baselins. Run the same test but with this URL:
https://demo.applitools.com/index_v2.html
. This version of the demo app has some visual bugs so you can see how it all works. And the second set of screenshots are called "Checkpoint images".
Example
1. Create a Maven project and add the following dependencies to the pom.xml
<!-- Applitools SDK -->
<dependency>
<groupId>com.applitools</groupId>
<artifactId>eyes-selenium-java3</artifactId>
<version>RELEASE</version>
</dependency>
2
3
4
5
6
7
2. Example Java Test
package com.applitools.quickstarts;
import static com.google.common.base.Strings.isNullOrEmpty;
import com.applitools.eyes.*;
import com.applitools.eyes.selenium.ClassicRunner;
import com.applitools.eyes.selenium.Eyes;
import org.junit.After;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
/**
* Runs Applitools test for the demo app https://demo.applitools.com
*/
@RunWith(JUnit4.class)
public class BasicDemo {
private EyesRunner runner;
private Eyes eyes;
private static BatchInfo batch;
private WebDriver driver;
@BeforeClass
public static void setBatch() {
// Must be before ALL tests (at Class-level)
batch = new BatchInfo("Demo batch");
}
@Before
public void beforeEach() {
// Initialize the Runner for your test.
runner = new ClassicRunner();
// Initialize the eyes SDK
eyes = new Eyes(runner);
// Raise an error if no API Key has been found.
if(isNullOrEmpty(System.getenv("APPLITOOLS_API_KEY"))) {
throw new RuntimeException("No API Key found; Please set environment variable 'APPLITOOLS_API_KEY'.");
}
// Set your personal Applitols API Key from your environment variables.
eyes.setApiKey(System.getenv("APPLITOOLS_API_KEY"));
// set batch name
eyes.setBatch(batch);
// Use Chrome browser
driver = new ChromeDriver();
}
@Test
public void basicTest() {
// Set AUT's name, test name and viewport size (width X height)
// We have set it to 800 x 600 to accommodate various screens. Feel free to
// change it.
eyes.open(driver, "Demo App", "Smoke Test", new RectangleSize(800, 800));
// Navigate the browser to the "ACME" demo app.
driver.get("https://demo.applitools.com");
// To see visual bugs after the first run, use the commented line below instead.
//driver.get("https://demo.applitools.com/index_v2.html");
// Visual checkpoint #1 - Check the login page.
eyes.checkWindow("Login Window");
// This will create a test with two test steps.
driver.findElement(By.id("log-in")).click();
// Visual checkpoint #2 - Check the app page.
eyes.checkWindow("App Window");
// End the test.
eyes.closeAsync();
}
@After
public void afterEach() {
// Close the browser.
driver.quit();
// If the test was aborted before eyes.close was called, ends the test as
// aborted.
eyes.abortIfNotClosed();
// Wait and collect all test results
TestResultsSummary allTestResults = runner.getAllTestResults();
// Print results
System.out.println(allTestResults);
}
}
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
6. ⚙️ Troubleshooting Common Issues
Forgetting to set your API key (or getting 401 exception).
Applitools is not opening the app (appears to crash).
- In order to make sure the screenshots are consistent across different tests runs. Applitools uses Viewport as an input. A viewport is simply the size of the screenshot you are taking (width and height). Applitools tries to resize the browser to that size before taking the screenshot. If you are running the app on a small laptop and/or have set the Viewport much larger than the laptop’s screen, then you’ll see this error.
- Solution: Reduce the height or the width. For more see here.
Not properly loading the API key from the environment variable into your IDE (like Eclipse).
- After setting the APPLITOOLS_API_KEY in the environment variable to hold your Applitools API key, open the IDE from the command line terminal (and not from the IDE directly). On Mac, it’d look like this: Open a Terminal and then type:
open ~/Applications/Eclipse.app
. This will load Eclipse with all the environment variables.
- After setting the APPLITOOLS_API_KEY in the environment variable to hold your Applitools API key, open the IDE from the command line terminal (and not from the IDE directly). On Mac, it’d look like this: Open a Terminal and then type:
Trying to run the test but nothing happens
- Check if the APPLITOOLS_API_KEY is null
Incompatibility between Chrome browser and Chrome driver.
- Make sure for your version of Google Chrome, you are using the corresponding Chrome driver
Debug logs
- See this article to enable debug logs to help file support ticket
Resources
Terms & Conditions Privacy Policy GDPR© 2020 Applitools. All rights reserved.