Introduction to Applitools Eyes for Selenium IDE

The Selenium IDE platform allows you to easily create and run tests on websites and browser applications. Some of the key features offered by Selenium IDE are:

  • Monitors your mouse and keyboard use as you exercise different parts of your browser application UI.
  • Stores the mouse and keyboard events to a script.
  • Allows you to edit the script, to add assertions as to what to expect in the state or value of screen elements at various points in the flow.
  • Allows you to playback a script, simulating the mouse and keyboard events, while testing your assertions.

The Applitools Eyes Selenium IDE extension, (SIDE Eyes) extends this functionality by allowing you to add checkpoints, i.e., commands that capture the content of the browser at any point and send it to the Eyes server to be compared against a baseline. A checkpoint is similar to a Selenium verify command in that it is used to test an expected value of an element in the UI. However, verify commands and checkpoints work differently. A verify command accesses the value of a UI element and checks that it has the expected value. A checkpoint, on the other hand, checks all the content of the page visually, it uses Image processing and AI to compare the entire page image to the reference image, So a single checkpoint can verify an entire page in one step and can replace multiple verify commands, each of which checks the value or state of a single element on the page. Another advantage of using checkpoint instead of verify commands is that checkpoints also validate the way the information looks - its position, color, font and so on.

The use of Selenium IDE, together with the SIDE Eyes extension makes it very simple and quick to create test scenarios, add checkpoints in key UI states, run the test and view the results - all without having to write any code. This makes it a useful tool for programmers and non-programmers alike.

The rest of this article focuses on what SIDE Eyes adds to Selenium IDE. To learn about Selenium IDE see theSelenium IDE documentation.

Visual Testing with SIDE Eyes

The diagram below illustrates the process of testing an application or website using Selenium IDE and SIDE Eyes to create and run a visual test. Looking at the diagram, we can identify three top-level steps:
  • Create the test
  • Run the visual test
  • View test progress and results

The figure highlights various key features and actions. Some of these are supported by standard Selenium IDE functionality (in blue) and others are supported by SIDE Eyes (in green) functionality. The rest of this article and its referenced articles focus on the SIDE Eyes features and options.

The sections that follow introduce the key features in each of the steps of the workflow. For a full overview of all the available features in SIDE Eyes see the article Overview of Selenium IDE with Eyes.

Create the test

Design Test and test structure

Both Selenium IDE and SIDE Eyes provide a way to organize your tests:
  1. You can define a group of related tests, and then choose to run all the tests or a subset of the tests.
  2. You can define a test hierarchically so that a single test is composed of multiple steps. You can then view the results both at a summary level ("did all my tests pass?") as well as at a detailed level ("which steps failed?").

You can read how Selenium IDE and SIDE Eyes support these capabilities in the article Organizing tests with Selenium IDE.

Create script Selenium & Eyes commands

A visual test script typically contains:
  • Selenium IDE commands to navigate to a web page.
  • Selenium IDE commands to simulate mouse and keyboard events.
  • SIDE Eyes eyes set commands to do run time configuration.
  • SIDE Eyes eyes check commands to capture the browser content and send it to the Eyes server for processing.
  • You can also use any of the available Selenium IDE commands, including assert and verify commands to implement a hybrid logical/visual test.

Selenium IDE provides two ways to create the script content. You can use the test application using the mouse and keyboard while Selenium IDE is in recording mode. Selenium IDE saves your mouse and keyboard actions as script commands. You can then use the script editor to add details or change a recorded script. You can also create a script from scratch without a recording phase at all.

SIDE Eyes supports both script recording and script editing by allowing you to add SIDE Eyes commands during a recording and by allowing you to add and edit its commands in the script editor.

Run the visual test

The figure above illustrates a visual testing paradigm where the test drives the application UI through a series of application UI states. After a test sets up each UI state, it executes a checkpoint, i.e., it captures an image of the browser content and sends it to the Eyes server. The Eyes server compares the sequence of checkpoints that it receives to a baseline that defines the expected image sequence. Eyes reports the results of comparing the checkpoint sequence and the baseline sequence as a series of steps. A step signifies if a checkpoint matches or mismatches its corresponding baseline image. A step can also indicate that there is a new checkpoint with no corresponding baseline image, or that a checkpoint is missing. For a more detailed description, see How Eyes compares checkpoints and baseline images.

Setup Eyes static and run-time options

SIDE Eyes allows you to set up various options. You set up some options statically using dialogs and menus, and you set other options by including SIDE Eyes specific Selenium IDE commands in the test script. All of the available options are described in the article Setting up tests with SIDE Eyes.

Typical test structure

Some of the SIDE Eyes eyes set commands, such as eyes set baseline env name, must be called before the first call to eyes check, and apply to the entire test. After setting up any necessary run time properties, the test script typically includes the following types of commands for every UI state tested:

  • Selenium IDE commands are used to navigate the website/application pages and to simulate mouse and keyboard events.
  • SIDE Eyes eyes check commands capture screen images and send them to Eyes for processing.
  • Some of the eyes set commands may be changed on the fly during a test. For example, the eyes set match level command can be called to change the match sensitivity for a series of checkpoints.
All the SIDE Eyes commands are listed in the article Creating tests using SIDE Eyes.

View test progress and results

As the test runs, Selenium IDE updates the script and log panes to indicate the progress of the test. The SIDE Eyes check commands are marked up to show that the capture/processing has started. When the result is known, they are marked up to indicate if the result of the step was OK or Failed. When the test run completes, by default Selenium IDE opens the Eyes Test Manager loaded with the test results of the test that completed. Selenium IDE also writes a hyperlink to the log. You can open the Eyes Test Manager at any point, for any test, by clicking on this link. For more details see Reviewing results with Selenium IDE.