WebdriverIO Tutorial

Please select a Tutorial

1. Introduction In real world testing, you'll need to test your app on multiple browsers, viewports and mobile devices. So you'll need to run and re-run ALL the tests in different browsers and multiple viewports to ensure things are fine - which could take hours and hours to complete. You may also encounter browsers not opening properly, or hanging and what not.

What if you could run in just one browser, just once, and still do cross-browser tests across various browsers, viewports and multiple mobile emulators? That's where Visual Grid [BETA] comes in.

TL;DR

When you execute this tutorial, it'll run the test locally in Chrome browser but will show results in the following 5 combinations:

  1. Chrome browser (800 X 600 viewport)
  2. Chrome browser (700 X 500 viewport)
  3. Firefox browser (1200 X 800 viewport)
  4. Firefox browser (1600 X 1200 viewport)
  5. An iPhone4 emulator

... all in ~30 seconds!⏰🚀

2. The Visual Grid

The Visual Grid[BETA] provides an elegant and an ultra fast way to render pages in different browsers, viewports and mobile simulators and then take screenshots and then send those screenshots to Applitools AI server. With Visual Grid you just need to run your test in a single browser but perform end-to-end and cross-browser tests in various browsers such as Chrome,Firefox, IE, and also numerous combinations of viewports and in mobile simulators -- all in under a minute. Without Visual Grid, you'll be wasting a lot of time and money running those tests in multiple browsers which could take hours and are error-prone.

And since Visual Grid is integrated with our Eyes AI, you'll actually find a lot more functional and visual bugs than you'd normally find.

Further, you can use our Root-Cause-Analysis (RCA) feature to figure out exactly which DOM element or CSS caused difference! So you can go backwards, from the screenshot to the code!

Best part: All you need to do is to simply make eyes.check calls in your existing Selenium tests after adding our SDK and instantiating Visual Grid. And that's all there is to it!

How it works

Once you add and instantiate the Applitools SDK, simply add eyes.check calls wherever you want to check if the state of the page looks OK. Then run the Selenium test as usual. And while running the tests, anytime it encounters eyes.check (or similar methods), Applitools SDK uploads the DOM resource of that page to our Visual Grid behind the scenes and in batches. The Visual Grid then renders every page in different browsers and mobile emulators (in parallel) and takes screenshots as per the configuration you set. Finally, it sends those screenshots to our AI Server.

When you first run the test, our AI server simply stores those 1st set of screenshots as Baseline images. When you run the same test again, the AI server will compare the new set of screenshots, aka Checkpoint images, with the corresponding Baseline images and higlights bugs.

Visual Grid architecture picture
3. Marking tests as pass or fail

When the AI compares the baseline and checkpoint image, if it finds a legitimate difference, it'll mark the test as "Undecided". You need to then login to our dashboard and then either mark it as "Fail" or "Pass" manually. If you mark it as a "Fail", any further runs with similar difference will be automatically marked as "Failed" tests. If you mark it as a "Pass", then we update the new checkpoint image as the new "Baseline" and if you run tests again we'll consider those as "Passed" tests.

Note:

  • Visual Grid is currently in Early access.

  • Visual Grid does not run tests, it just opens/renders webpages in different browsers, in different viewports, and in mobile emulators, in landscape and portrait modes. You still need to run a single set of tests either on a local machine or in the cloud (like AWS, GCloud, Azure or in commercial testing vendor clouds). But you don't need to run all the tests on each browsers and each mobile devices and emulators.

  • Applitools AI has been trained with 100s of millions of images. It doesn't do a pixel-by-pixel comparison because it leads to a lot of false positives, but instead simulates real human eyes and ignore normal differences that humans would ignore and only highlight those that humans would highlight as bugs.

  • ACCURACY: A.I's current accuracy rate is 99.999%! Which means for most applications that odds that you'll see false-positives is 1 in a million!

4. Get Started 🚀

Pre-requisites:

  1. Download Selenium Standalone server from here
    • Note that this is a jar file. So you should also have Java installed on your machine.
  2. Run the standalone server jar file - it should look something like below:
    • java -jar selenium-server-standalone-3.141.59.jar (Replace the jar file name with your jar file name)
    • This will run Selenium on localhost and on port 4444
  3. Install Node.js from here

Running the example:

  1. Download the example
    • Option 1: git clone https://github.com/applitools/tutorial-webdriverio-visualgrid.git
    • Option 2: Download it as a Zip file and extract it
  2. CD into the tutorial-webdriverio-visualgrid folder
  3. run npm install
  4. run node index.js

Adding to an existing Node.js project

npm install "@applitools/eyes-selenium" "@applitools/eyes.webdriverio" --save-dev
1

Source code

The following code imports the Applitools SDK to a regular WebDriverIO project. Then it configures VisualGrid options to render the app in 1. Chrome browser (two viewports), 2. Firefox browser(two viewports) and 3. iPhone4 Emulator (Portrait mode). And when you run the test in Chrome browser.

Note

Everytime we call eyes.check, we upload the resources(HTML, CSS, fonts), esseentially the state of the DOM so that we can render the app in different browsers inside Visual Grid.


"use strict";

const chromedriver = require('chromedriver');
const webdriverio = require('webdriverio');
const {
  By,
  Eyes,
  Target,
  VisualGridRunner
} = require('@applitools/eyes.webdriverio');
const {
  BrowserType,
  Configuration,
  DeviceName,
  ScreenOrientation
} = require('@applitools/eyes-selenium');

(async () => {
  chromedriver.start();

  // Open a Chrome browser.
  const chrome = {
    desiredCapabilities: {
      browserName: 'chrome'
    }
  };
  let driver = webdriverio.remote(chrome);
  await driver.init();

  // Initialize the eyes SDK with 3 concurrent runners
  const eyes = new Eyes(new VisualGridRunner(3));

  try {
    const configuration = new Configuration();
    configuration.setAppName('Demo app');
    configuration.setTestName('WebdriverIO Visual Grid test!');

    //Add Chrome browser with two different viewports
    configuration.addBrowser(800, 600, BrowserType.CHROME);
    configuration.addBrowser(700, 500, BrowserType.CHROME);

    //Add Firefox browser with two different viewports
    configuration.addBrowser(1200, 800, BrowserType.FIREFOX);
    configuration.addBrowser(1600, 1200, BrowserType.FIREFOX);

    //Add iPhone 4 with Portrait mode
    configuration.addDeviceEmulation(DeviceName.iPhone_4, ScreenOrientation.PORTRAIT);

    // Set your private API key here or in the "APPLITOOLS_API_KEY" environment variable
    configuration.setApiKey(process.env.APPLITOOLS_API_KEY);
    eyes.setConfiguration(configuration);

    driver = await eyes.open(driver);

    // Navigate the browser to the "hello world!" web-site.
    await driver.url("https://demo.applitools.com");

    //⭐️To see visual bugs, change the above URL to:
    //  https://demo.applitools.com/index_v2.html and run the test again

    // Visual checkpoint #1.
    await eyes.check("Login Page", Target.window());

    // Click the "Log in" button.
    await driver.click(By.id("log-in"));

    // Visual checkpoint #2.
    await eyes.check("App page", Target.window());

    console.log(
      `Please wait, we are now..
      1. Uploading the app's resources (html, css, images)
      2. Rendering them in different browsers, emulators
      3. Analyzing them using our A.I. 

      ...you should see the result within 15 - 60 seconds depending on your internet speed, # combinations and how heavy your app is.
    `
    );

    // End the test.
    // const results = await eyes.close(); // will return only first TestResults, but as we have two browsers, we need more results
    const results = await eyes.getRunner().getAllResults(false);
    console.log(results);
  } finally {
    // Close the browser.
    await driver.end();

    // If the test was aborted before eyes.close was called ends the test as aborted.
    await eyes.abortIfNotClosed();

    chromedriver.stop();
  }
  
})();

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96

Note:

To see visual bugs: After you run one set of tests, you now have the baselines. Run the same test but with this URL: https://demo.applitools.com/index_v2.html. This version of the demo app has some visual bugs so you can see how it all works. And the second set of screenshots are called "Checkpoint images".


Resources




Terms & Conditions Privacy Policy GDPR© 2019 Applitools. All rights reserved.

Last Updated: 4/9/2019, 2:11:57 PM