Selenium JavaScript tutorial

1. 🤖 How it works

Applitools SDKs work with existing test frameworks to take screenshots of pages, elements, regions or iframes and upload them along with DOM snapshots to our Eyes server. Our AI then compares them with previous test executions' screenshots (aka Baselines) and reports if there is a bug or not. It's that simple!

Applitools AI with RCA picture

1.1 Baseline vs. Checkpoint images

When you first run the test, our A.I. server stores those first set of screenshots as Baseline images. When you run the same test again (and everytime there after), the A.I. server compares the new set of screenshots, aka Checkpoint images, with the corresponding Baseline images and highlights differences in a pink color.

Baseline vs Checkpoint
The picture above is showing the Side-by-Side view of the baseline and checkpoint images

1.2 Marking the test as "Pass" or "Fail"

When the AI compares the baseline and the checkpoint image, if it finds a legitimate difference, it will mark the test as Unresolved. This is because the AI doesn't know if the difference is because of a new feature or a real bug and will wait for you to manually mark it as a Pass/Fail for the 1st time.

If you mark the Unresolved checkpoint image as a "Fail", any further runs with similar difference will be automatically marked as "Failed".

Mark the checkpoint as a fail
The picture above is showing how to mark the checkpoint image as Failed

If you mark the Unresolved checkpoint image as a "Pass", then it means that the difference is due to a new feature so we set the new checkpoint image as the new baseline and mark the current test as Pass. Going forward we'll compare any future tests with this new baseline.

Mark the checkpoint as a Pass
The picture above is showing how to mark the checkpoint image as Passed

Note:

  • Applitools AI has been trained with 100s of millions of images. It doesn't do pixel-to-pixel comparison as this can lead to a lot of false positives. It instead simulates human eyes that ignore differences that humans can't detect and highlight differences that humans can detect.

  • ACCURACY: Our A.I.'s current accuracy rate is 99.9999%! Which means for most applications the odds that you'll see false-positives are 1 in a million!

A powerful test results dashboard

We provide a state-of-the-art dashboard that makes it very easy for you to analyze differences, report bugs and much more. For more information on the Applitools dashboard check out these articles.

Seeing test result summary
The picture above is showing the summary view
2. 🖼 Analyzing differences

The following Gifs show various tools Applitools provides to easily analyze various differences

Highlight differences between the baseline and checkpoint

Highlight diffs

Zoom into differences

Zoom into diffs

Toggle between baseline and checkpoint

Toggle differences between baseline and checkpoint

Show both the baseline and checkpoint side-by-side

Show both baseline and checkpoint side-by-side
3. 🐞 Reporting bugs (straight into Jira or Github)

You can select a section of the image and directly file a bug in Jira or Github. No need to manually take screenshots, write steps and explain things! To read more about bug regions check out this article.

Reporting bugs to Jira or Github
4. ✅ Prerequisites
  1. Create a free Applitools account and get the Applitools API KEY API-key

  2. Please set the APPLITOOLS_API_KEY environment variable

    • Mac: export APPLITOOLS_API_KEY='YOUR_API_KEY'

    • Windows: set APPLITOOLS_API_KEY='YOUR_API_KEY'

    TIP

    1. You may skip this step if you want to hard code the API KEY inside the tutorial project's code.
    2. It's better to store APPLITOOLS_API_KEY in the system variables (in Windows) or in the ~/.bash_profile (in Mac) so that it is accessible from all Terminal shells
  3. Install node.js from https://nodejs.org​

  4. Install git from https://git-scm.com​

    TIP

    Installing git is optional. You need this mainly to clone the demo project from the Github repository. Instead of installing git, you can simply download the Zip file from the repo. Further, If you are Mac, you already have git.

  5. Install Google Chrome browser from https://www.google.com/chrome/

  6. Install ChromeDriver on your machine and make sure it's is in the PATH.

Below are some resources from the internet that'll help you

5. 🚀 Run Tests

There are two main ways in which you can run tests.

The Applitools Ultrafast Grid (option 1) Test across all browsers and viewports 30-70 times faster than traditional test clouds. Using this approach, your tests will execute on a local browser once with the DOM, CSS, and additional artifacts captured by the SDK and sent to the Ultrafast Grid for parallel rendering and Visual AI comparison. This is the preferred execution method, however please note that it is a premium feature.

The second approach is to Run Local (option 2). This is the classic test execution in which your tests execute locally and each call to eyes via the SDK will send a screenshot to Eyes for Visual AI comparison.

While running tests using the classic approach is a great start - we recommend running tests on the Ultrafast Grid, the fastest way to ensure quality and consistency across all browsers, viewports, and devices.

Option 1 - Run With The Ultrafast Grid

Free Accounts:

Free accounts are encouraged to try the Ultrafast Grid, but should be aware that their monthly consumption of checkpoints could be depleted quickly, depending on the number of browsers and viewport being checked.

Current Customers:

For customers who do not yet have Ultrafast Grid enabled on their account - or are not sure, please contact your CSM or AE to schedule a demo and see about upgrading your account. You may also schedule a demo here.

1. Introduction To the Ultrafast Grid

In real world testing, you'll need to test your app on multiple browsers, viewports and mobile devices. So you'll need to run and re-run ALL the tests in different browsers and multiple viewports to ensure things are fine - which could take hours and hours to complete. You may also encounter browsers not opening properly, or hanging and what not.

What if you could run in just one browser, just once, and still do cross-browser tests across various browsers, viewports and multiple mobile emulators? That's where Ultrafast Grid comes in.

TL;DR

When you execute this tutorial, it'll run the test locally in Chrome browser but will show results in the following 5 combinations:

  1. Chrome browser (800 X 600 viewport)
  2. Chrome browser (700 X 500 viewport)
  3. Firefox browser (1200 X 800 viewport)
  4. Firefox browser (1600 X 1200 viewport)
  5. An iPhone4 emulator

... all in ~30 seconds!⏰🚀

2. What Is The Ultrafast Grid?

The Ultrafast Grid provides an elegant and an ultra fast way to render pages in different browsers, viewports and mobile simulators and then take screenshots and then send those screenshots to Applitools AI server. With Ultrafast Grid you just need to run your test in a single browser but perform end-to-end and cross-browser tests in various browsers such as Chrome,Firefox, IE, and also numerous combinations of viewports and in mobile simulators -- all in under a minute. Without Ultrafast Grid, you'll be wasting a lot of time and money running those tests in multiple browsers which could take hours and are error-prone.

And since Ultrafast Grid is integrated with our Eyes AI, you'll actually find a lot more functional and visual bugs than you'd normally find.

Further, you can use our Root-Cause-Analysis (RCA) feature to figure out exactly which DOM element or CSS caused difference! So you can go backwards, from the screenshot to the code!

Best part: All you need to do is to simply make eyes.check calls in your existing Selenium tests after adding our SDK and instantiating Ultrafast Grid. And that's all there is to it!

How it works

Once you add and instantiate the Applitools SDK, simply add eyes.check calls wherever you want to check if the state of the page looks OK. Then run the Selenium test as usual. And while running the tests, anytime it encounters eyes.check (or similar methods), Applitools SDK uploads the DOM resource of that page to our Ultrafast Grid behind the scenes and in batches. The Ultrafast Grid then renders every page in different browsers and mobile emulators (in parallel) and takes screenshots as per the configuration you set. Finally, it sends those screenshots to our AI Server.

When you first run the test, our AI server simply stores those 1st set of screenshots as Baseline images. When you run the same test again, the AI server will compare the new set of screenshots, aka Checkpoint images, with the corresponding Baseline images and highlights bugs.

Ultrafast Grid architecture picture

3. Getting Started With The Ultrafast Grid

Pre-requisites

  1. Install Node.js from here
  2. Register to Applitools and have the APPlitools API Key
    • Get the API key by logging into Applitools > Person Icon > My API Key
  3. Chrome Webdriver is on your machine and is in the PATH. Here are some resources from the internet that'll help you.

Download the demo project

  1. Get the code:
    • git clone https://github.com/applitools/tutorial-selenium-javascript-ultrafastgrid.git,
    • or, download it as a Zip file and unzip it.
  2. cd tutorial-selenium-javascript-ultrafastgrid folder
  3. npm install
  4. npm run test

Add to existing project

Install the SDK

npm install "@applitools/eyes-selenium"
1

Example test

'use strict';

// Import chromedriver
require('chromedriver');

// Import Selenium Webdriver
const { Builder, Capabilities } = require('selenium-webdriver');

// Import Applitools SDK and relevant methods
const { Eyes, VisualGridRunner, Target, ConsoleLogHandler, Configuration, BrowserType, DeviceName, ScreenOrientation, BatchInfo } = require('@applitools/eyes-selenium'); // should be replaced to '@applitools/eyes-selenium'

function initializeEyes(runner) {
  // Create Eyes object with the runner, meaning it'll be a Visual Grid eyes.
  const eyes = new Eyes(runner);
  // Set logger
  eyes.setLogHandler(new ConsoleLogHandler(false));


  // Create Configuration
  const configuration = new Configuration();

  // Set API key
  configuration.setApiKey(process.env.APPLITOOLS_API_KEY);

  // If dedicated or on-prem cloud, uncomment and enter the cloud url
  // Default: https://eyes.applitools.com
  // configuration.setServerUrl("https://testeyes.applitools.com");

  // Set a proxy if required
  // configuration.setProxy('http://localhost:8888');

  // Set the AUT name
  configuration.setAppName('Eyes Examples');

  // Set a test name
  configuration.setTestName('My first Javascript VisualGridRunner test!');

  // Set a batch name so all the different browser and mobile combinations are part of the same batch
  configuration.setBatch(new BatchInfo("VIP Browser combo batch"));

  // Add Chrome browsers with different Viewports
  configuration.addBrowser(800, 600, BrowserType.CHROME);
  configuration.addBrowser(700, 500, BrowserType.CHROME);

  // Add Firefox browser with different Viewports
  configuration.addBrowser(1200, 800, BrowserType.FIREFOX);
  configuration.addBrowser(1600, 1200, BrowserType.FIREFOX);

  // Add iPhone 4 device emulation
  configuration.addDeviceEmulation(DeviceName.iPhone_4, ScreenOrientation.PORTRAIT);

  // Set the configuration object to eyes
  eyes.setConfiguration(configuration);
  return eyes;
}

async function runTest(url, runner) {
  //Initialize Eyes with Visual Grid Runner
  const eyes = initializeEyes(runner);

  // Create a new Webdriver
  const webDriver = new Builder()
      .withCapabilities(Capabilities.chrome())
      // .setChromeOptions(new ChromeOptions().headless())
      .build();

  try {
    // Navigate to the URL we want to test
    await webDriver.get(url);

    // Call Open on eyes to initialize a test session
    await eyes.open(webDriver);

    // Check the page
    await eyes.check('Main Page ' + url, Target.window());

    // Close eyes asynchronously
    await eyes.closeAsync();
  } catch (e) {
    console.log('Error', e); // eslint-disable-line
  } finally {
    // Close the browser
    await webDriver.quit();
  }
}

(async () => {
  // Create a runner with concurrency of 10
  const runner = new VisualGridRunner(10);

  try {
    // Define links to process
    const urlsToTest = [
      'https://applitools.com/helloworld',
      'http://applitools-dom-capture-origin-1.surge.sh/testWithIframe.html',
      'http://applitools.github.io/demo/TestPages/FramesTestPage/',
    ];

    // Run test for each link
    for (const url of urlsToTest) {
      await runTest(url, runner);
    }

    // Get all results at once
    const results = await runner.getAllTestResults(false);
    // Print results
    console.log(results); // eslint-disable-line
  } catch (e) {
    // if results failed, it goes here
    console.log('Error', e); // eslint-disable-line
  }
})();

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
Option 2 - Run Locally

Run the demo app

  1. Get the code:
    • Option 1: git clone https://github.com/applitools/tutorial-selenium-javascript-basic.git
    • Option 2: Download it as a Zip file and unzip it.
  2. cd tutorial-selenium-javascript-basic folder
  3. Note: Remember to set the APPLITOOLS_API_KEY env variable.
  4. npm install
  5. npm run test
  6. After you run one set of tests, you now have the baseline. Run the same test but with this URL: https://demo.applitools.com/index_v2.html. This version of the demo app has some visual bugs so you can see how it all works. And the second set of screenshots are called "Checkpoint images". You can change the URL by opening the /test/classic-runner-example.test.js file.

Add Applitools to an existing project

Installing the Eyes SDK

npm install @applitools/eyes-selenium --save-dev
1

Example code


'use strict';

require('chromedriver');
const { Builder, By } = require('selenium-webdriver');
const { Eyes, ClassicRunner, Target, RectangleSize } = require('@applitools/eyes-selenium');

describe('DemoApp - ClassicRunner', function () {
  let runner, eyes, driver;

  beforeEach(async () => {
    // Initialize the Runner for your test.
    runner = new ClassicRunner();

    // Initialize the eyes SDK (IMPORTANT: make sure your API key is set in the APPLITOOLS_API_KEY env variable).
    eyes = new Eyes(runner);

    // Use Chrome browser
    driver = await new Builder()
      .forBrowser('chrome')
      // .setChromeOptions(new ChromeOptions().headless())
      .build();
  });

  it('Smoke Test', async () => {
    // Start the test by setting AUT's name, test name and viewport size (width X height)
    await eyes.open(driver, 'DemoApp - ClassicRunner', 'Smoke Test', new RectangleSize(600, 800));

    // Navigate the browser to the "ACME" demo app.
    await driver.get("https://demo.applitools.com");
    // await driver.get("https://demo.applitools.com/index_v2.html");
    // To see visual bugs after the first run, use the commented line above instead.

    // Visual checkpoint #1 - Check the login page.
    await eyes.check("Login Window", Target.window());

    // This will create a test with two test steps.
    await driver.findElement(By.id("log-in")).click();

    // Visual checkpoint #2 - Check the app page.
    await eyes.check("App Window", Target.window().fully());

    // End the test.
    await eyes.closeAsync();
  });

  afterEach(async () => {
    // Close the browser.
    await driver.quit();

    // If the test was aborted before eyes.close was called, ends the test as aborted.
    await eyes.abortIfNotClosed();

    // Wait and collect all test results
    const allTestResults = await runner.getAllTestResults();
    console.log(allTestResults);
  });
});
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
6. ⚙️ Troubleshooting Common Issues
  1. Forgetting to set your API key (or getting 401 exception).

  2. Applitools is not opening the app (appears to crash).

    • In order to make sure the screenshots are consistent across different tests runs. Applitools uses Viewport as an input. A viewport is simply the size of the screenshot you are taking (width and height). Applitools tries to resize the browser to that size before taking the screenshot. If you are running the app on a small laptop and/or have set the Viewport much larger than the laptop’s screen, then you’ll see this error.
    • Solution: Reduce the height or the width. For more see here.
  3. Not properly loading the API key from the environment variable into your IDE (like Eclipse).

    • After setting the APPLITOOLS_API_KEY in the environment variable to hold your Applitools API key, open the IDE from the command line terminal (and not from the IDE directly). On Mac, it’d look like this: Open a Terminal and then type: open ~/Applications/Eclipse.app. This will load Eclipse with all the environment variables.
  4. Trying to run the test but nothing happens

    • Check if you the APPLITOOLS_API_KEY is null
  5. Incompatibility between Chrome browser and Chrome driver.

  6. Debug logs

    • See this article to enable debug logs to help file support ticket

Resources




Terms & Conditions Privacy Policy GDPR© 2019 Applitools. All rights reserved.

Last Updated: 3/4/2020, 5:01:45 PM