Espresso Tutorial
1. 🤖 How it works
Applitools SDKs work with existing test frameworks to take screenshots of pages, elements, regions or iframes and upload them along with DOM snapshots to our Eyes server. Our AI then compares them with previous test executions' screenshots (aka Baselines) and reports if there is a bug or not. It's that simple!

1.1 Baseline vs. Checkpoint images
When you first run the test, our A.I. server stores those first set of screenshots as Baseline images. When you run the same test again (and everytime there after), the A.I. server compares the new set of screenshots, aka Checkpoint images, with the corresponding Baseline images and highlights differences in a pink color.

1.2 Marking the test as "Pass" or "Fail"
When the AI compares the baseline and the checkpoint image, if it finds a legitimate difference, it will mark the test as Unresolved. This is because the AI doesn't know if the difference is because of a new feature or a real bug and will wait for you to manually mark it as a Pass/Fail for the 1st time.
If you mark the Unresolved checkpoint image as "Failed", it'll only mark the current test result as Failed.

Note:
To automatically mark the checkpoint as a "Fail" in the future test runs, you need to do the following:
- Annotate at least one of differences as a "bug region"
- Select the "Fail tests" checkbox in the popup window
- Press "Thumbs Up" (not "Thumbs Down") button in the checkpoint image's toolbar (Note: this is counter-intuitive. But what happens is that, we now create a new Baseline along with this bug and "Failed" metadata. So if the same image with the exact bug appears, it'll fail again)
- Press "Save" in the main toolbar
If you mark the Unresolved checkpoint image as a "Pass", then it means that the difference is due to a new feature so we set the new checkpoint image as the new baseline and mark the current test as Pass. Going forward we'll compare any future tests with this new baseline.

Note:
Applitools AI has been trained with 100s of millions of images. It doesn't do pixel-to-pixel comparison as this can lead to a lot of false positives. It instead simulates human eyes that ignore differences that humans can't detect and highlight differences that humans can detect.
ACCURACY: Our A.I.'s current accuracy rate is 99.9999%! Which means for most applications the odds that you'll see false-positives are 1 in a million!
A powerful test results dashboard
We provide a state-of-the-art dashboard that makes it very easy for you to analyze differences, report bugs and much more. For more information on the Applitools dashboard check out these articles.

2. 🖼 Analyzing differences
The following Gifs show various tools Applitools provides to easily analyze various differences
Highlight differences between the baseline and checkpoint

Zoom into differences

Toggle between baseline and checkpoint

Show both the baseline and checkpoint side-by-side

3. 🐞 Reporting bugs (straight into Jira or Github)
You can select a section of the image and directly file a bug in Jira or Github. No need to manually take screenshots, write steps and explain things! To read more about bug regions check out this article.

4. ✅ Prerequisites
Create a free Applitools account and get your Applitools API KEY
Install git from https://git-scm.com
TIP
Installing
git
is optional. You need this mainly to clone the demo project from the Github repository. Instead of installinggit
, you can simply download the Zip file from the repo. Further, If you are using Mac OSX, you already havegit
.Android Studio
5.1 🚀 - Run the existing demo app
Get the code:
- Option 1:
git clone https://github.com/applitools/eyes-android-hello-world
- Option 2: Download it as a Zip file and unzip it.
- Option 1:
Import the project into Android Studio
Run the test by clicking the Run Button to the left of the simpleTest() method.
TIP
This method is located in the ExampleInstrumentedTest.java file. Be sure to set your API key in the line that says
eyes.setApiKey("YOUR_API_KEY");
5.2 🤓 - Add Applitools to an existing project
build.gradle
file
Include the SDK in your // If using the Support Android Libraries, add the following dependences
androidTestImplementation 'com.applitools:eyes-android-espresso:4.+@aar'
androidTestImplementation 'com.applitools:eyes-android-common:4.+'
androidTestImplementation 'com.applitools:eyes-android-core:4.+'
androidTestImplementation 'com.applitools:eyes-android-components:4.+@aar'
androidTestImplementation 'com.applitools:eyes-android-components-support:4.+@aar'
2
3
4
5
6
// If using the Androidx Libraries, add the following dependencies
androidTestImplementation 'com.applitools:eyes-android-espresso:4.+@aar'
androidTestImplementation 'com.applitools:eyes-android-common:4.+'
androidTestImplementation 'com.applitools:eyes-android-core:4.+'
androidTestImplementation 'com.applitools:eyes-android-components:4.+@aar'
androidTestImplementation 'com.applitools:eyes-android-components-androidx:4.+@aar'
2
3
4
5
6
Example Espresso Instrumented Test
package com.applitools.helloworld.android;
import android.support.test.espresso.matcher.ViewMatchers;
import android.support.test.rule.ActivityTestRule;
import android.support.test.runner.AndroidJUnit4;
import android.util.Log;
import android.view.View;
import com.applitools.eyes.android.common.BatchInfo;
import com.applitools.eyes.android.common.EyesRunner;
import com.applitools.eyes.android.common.Feature;
import com.applitools.eyes.android.common.Region;
import com.applitools.eyes.android.common.TestResultContainer;
import com.applitools.eyes.android.common.TestResults;
import com.applitools.eyes.android.common.TestResultsSummary;
import com.applitools.eyes.android.common.config.Configuration;
import com.applitools.eyes.android.espresso.ClassicRunner;
import com.applitools.eyes.android.espresso.Eyes;
import com.applitools.eyes.android.espresso.fluent.Target;
import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
import static android.support.test.espresso.Espresso.onView;
import static android.support.test.espresso.action.ViewActions.click;
import static android.support.test.espresso.matcher.ViewMatchers.withId;
@RunWith(AndroidJUnit4.class)
public class DocumentationExampleTest {
private static final String TAG = "DocumentationExampleTest";
@Rule
public ActivityTestRule<MainActivity> mMainActivityRule = new ActivityTestRule(MainActivity.class, false, false);
@Rule
public ActivityTestRule<DialogActivity> mDialogActivityRule = new ActivityTestRule(DialogActivity.class, false, false);
@Rule
public ActivityTestRule<GoogleMapsActivity> mGoogleMapActivityRule = new ActivityTestRule(GoogleMapsActivity.class, false, false);
private static String eyesServerUrl = "https://eyesapi.applitools.com";
private static String appName = "EKB Example : classic app";
private static String batchName = "EKB Example : classic";
private static String apiKey = "YOUR_API_KEY";
private static String testName = "Hello World test";
private static EyesRunner runner = null;
private static Configuration suiteConfig;
private Eyes eyes;
@BeforeClass
public static void beforeTestSuite() {
runner = new ClassicRunner();
suiteConfig = new Configuration();
suiteConfig.setHideCaret(true)
.setAppName(appName)
.setApiKey(apiKey)
.setServerUrl(eyesServerUrl)
//Add the following line to force use of Android PixelCopy to obtain screenshots
//This can improve the quality of the screenshot, for example to ensure rendering of the shadow layer.
.setFeatures(Feature.PIXEL_COPY_SCREENSHOT)
.setBatch(new BatchInfo(batchName));
}
@Before
public void beforeEachTest() {
eyes = new Eyes(runner);
/*
Uncomment the call to 'eyes.setComponentsProvider' if you use AndroidX components such as
NestedScrollView, RecyclerView and ViewPager2
*/
//eyes.setComponentsProvider(new AndroidXComponentsProvider());
eyes.setConfiguration(suiteConfig);
}
@Test
public void testStartScreen() {
mMainActivityRule.launchActivity(null);
eyes.open(testName);
eyes.check("Click me button",Target.region(ViewMatchers.withId(R.id.click_me_btn)));
View helloLabel = mMainActivityRule.getActivity().findViewById(R.id.hello_text_view);
eyes.check("HelloWorld label", Target.region(helloLabel));
Region region = new Region(200, 300, 0, 0);
eyes.check("Region",Target.region(region));
eyes.check("Before button click", Target.window());
onView(withId(R.id.click_me_btn)).perform(click());
eyes.check("After button click", Target.window());
}
@Test
public void testDialog() {
mDialogActivityRule.launchActivity(null);
eyes.open("Dialog test");
//TBD - can we add examples with a popup or dialog and then show the 3 possibilities of
eyes.check("main viewport only",Target.window());
eyes.check("dialog only",Target.window().dialog());
eyes.check("Both main viewport and dialog",Target.window().includeAllLayers());
}
@Test
public void testGoogleMap() {
mGoogleMapActivityRule.launchActivity(null);
eyes.open("GoogleMaps test");
eyes.check("A googleMap", Target.googleMap().id(R.id.map));
// eyes.check("Not a SupportMapFragment", Target.googleMap().id(R.id.map).isNotSupportGoogleMap());
}
@After
public void afterEachTest() {
try {
eyes.close();
} finally {
eyes.abortIfNotClosed();
}
}
@AfterClass
public static void afterTestSuite() {
TestResultsSummary allTestResults = runner.getAllTestResults(false);
for (TestResultContainer result : allTestResults) {
handleTestResults(result);
}
}
private static void handleTestResults(TestResultContainer summary) {
Throwable ex = summary.getException();
if (ex != null) {
Log.e(TAG, "System error occurred while checking target.");
}
TestResults result = summary.getTestResults();
if (result == null) {
Log.e(TAG, "No test results information available.");
} else {
Log.d(TAG, String.format("URL = %s, AppName = %s, testName = %s, matched = %d, mismatched = %d, missing = %d, aborted = %s",
result.getUrl(),
result.getAppName(),
result.getName(),
result.getMatches(),
result.getMismatches(),
result.getMissing(),
(result.isAborted() ? "aborted" : "no")));
}
}
}
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
⚙️ 6. Troubleshooting Common Issues
Forgetting to set your API key (or getting 401 exception):
Debug logs:
- See this article to enable debug logs to help file support ticket
If you're using the AndroidX Components library, ensure you're configuring this in your Eyes test with the following code:
eyes.setComponentsProvider(new AndroidXComponentsProvider());
Resources
Terms & Conditions Privacy Policy GDPR© 2021 Applitools. All rights reserved.