Testcafe Tutorial

1. 🤖 How it works

Applitools SDKs work with existing test frameworks to take screenshots of pages, elements, regions or iframes and upload them along with DOM snapshots to our Eyes server. Our AI then compares them with previous test executions' screenshots (aka Baselines) and reports if there is a bug or not. It's that simple!

Applitools AI with RCA picture

1.1 Baseline vs. Checkpoint images

When you first run the test, our A.I. server stores those first set of screenshots as Baseline images. When you run the same test again (and everytime there after), the A.I. server compares the new set of screenshots, aka Checkpoint images, with the corresponding Baseline images and highlights differences in a pink color.

Baseline vs Checkpoint
The picture above is showing the Side-by-Side view of the baseline and checkpoint images

1.2 Marking the test as "Pass" or "Fail"

When the AI compares the baseline and the checkpoint image, if it finds a legitimate difference, it will mark the test as Unresolved. This is because the AI doesn't know if the difference is because of a new feature or a real bug and will wait for you to manually mark it as a Pass/Fail for the 1st time.

If you mark the Unresolved checkpoint image as "Failed", it'll only mark the current test result as Failed.

Mark the checkpoint as a fail
The picture above is showing how to mark the checkpoint image as Failed

Note:

To automatically mark the checkpoint as a "Fail" in the future test runs, you need to do the following:

  1. Annotate at least one of differences as a "bug region"
  2. Select the "Fail tests" checkbox in the popup window
  3. Press "Thumbs Up" (not "Thumbs Down") button in the checkpoint image's toolbar (Note: this is counter-intuitive. But what happens is that, we now create a new Baseline along with this bug and "Failed" metadata. So if the same image with the exact bug appears, it'll fail again)
  4. Press "Save" in the main toolbar

If you mark the Unresolved checkpoint image as a "Pass", then it means that the difference is due to a new feature so we set the new checkpoint image as the new baseline and mark the current test as Pass. Going forward we'll compare any future tests with this new baseline.

Mark the checkpoint as a Pass
The picture above is showing how to mark the checkpoint image as Passed

Note:

  • Applitools AI has been trained with 100s of millions of images. It doesn't do pixel-to-pixel comparison as this can lead to a lot of false positives. It instead simulates human eyes that ignore differences that humans can't detect and highlight differences that humans can detect.

  • ACCURACY: Our A.I.'s current accuracy rate is 99.9999%! Which means for most applications the odds that you'll see false-positives are 1 in a million!

A powerful test results dashboard

We provide a state-of-the-art dashboard that makes it very easy for you to analyze differences, report bugs and much more. For more information on the Applitools dashboard check out these articles.

Seeing test result summary
The picture above is showing the summary view
2. 🖼 Analyzing differences

The following Gifs show various tools Applitools provides to easily analyze various differences

Highlight differences between the baseline and checkpoint

Highlight diffs

Zoom into differences

Zoom into diffs

Toggle between baseline and checkpoint

Toggle differences between baseline and checkpoint

Show both the baseline and checkpoint side-by-side

Show both baseline and checkpoint side-by-side
3. 🐞 Reporting bugs (straight into Jira or Github)

You can select a section of the image and directly file a bug in Jira or Github. No need to manually take screenshots, write steps and explain things! To read more about bug regions check out this article.

Reporting bugs to Jira or Github
🚀4. Run the demo app ### Pre-requisites: 1. Install Node.js from [here](https://nodejs.org/en/)

Running the example:

  1. Download the example
    • Option 1: git clone https://github.com/applitools/tutorial-testcafe.git
    • Option 2: Download it as a Zip file and extract it
  2. CD into the tutorial-testcafe folder
  3. Change the APPLITOOLS_API_KEY with your own (apiKey configuration property of the applitools.config.jsfile).
    • Login to Applitools > Click on the Person icon > My API Key
  4. run npm install
  5. run npm test

Example code

//play.testcafe.js

// next line is optional, just for IDEs autocomplete :
/// <reference types="@applitools/eyes-testcafe" />

import Eyes from '@applitools/eyes-testcafe';

// Initialize the eyes
const eyes = new Eyes();

// Set page used in the test
fixture`Testcafe Demo App`.page`https://demo.applitools.com`
    // Call Close on eyes to let the server know it should display the results
    .afterEach(async () => eyes.close())
    .after(async () => {
        // Wait and collect all test results
        // we pass false to this method to suppress the exception that is thrown if we
        // find visual differences
        let allTestResults = await eyes.waitForResults(false)
        // Print results
        console.log(allTestResults)
    });

test('ultraFastTest', async t => {
    // Call Open on eyes to initialize a test session
    await eyes.open({
        t, // pass testcafe contorller
        appName: 'Demo App',
        testName: 'Ultrafast grid demo',
    });

    // check the login page with fluent api, see more info here
    // https://applitools.com/docs/topics/sdk/the-eyes-sdk-check-fluent-api.html
    await eyes.checkWindow({
        tag: "Login Window",
        target: 'window',
        fully: true
    });

    // This will create a test with two test steps.
    await t.click('#log-in')

    // Check the app page
    await eyes.checkWindow({
        tag: "App Window",
        target: 'window',
        fully: true
    })
});
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

Note that you may set the API key, concurrency (number of parallel screenshots), browsers X viewports, etc in the applitools.config.js file in the root directory.

//applitools.config.js
module.exports = {
    apiKey: 'APPLITOOLS_API_KEY',
    //
    browser: [
        // Add browsers with different viewports
        {width: 800, height: 600, name: 'chrome'},
        {width: 700, height: 500, name: 'firefox'},
        {width: 1600, height: 1200, name: 'ie11'},
        {width: 1024, height: 768, name: 'edgechromium'},
        {width: 800, height: 600, name: 'safari'},
        // Add mobile emulation devices in Portrait mode
        {deviceName: 'iPhone X', screenOrientation: 'portrait'},
        {deviceName: 'Pixel 2', screenOrientation: 'portrait'}
    ],
    // set batch name to the configuration
    batchName: 'Ultrafast Batch'
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

TIP

For full documentation of the Testcafe SDK, please head over to the npm page

Resources




Terms & Conditions Privacy Policy GDPR© 2020 Applitools. All rights reserved.

Last Updated: 5/22/2020, 8:21:31 PM