Visual locators

In some types of application, using DOM locators to generate keyboard or mouse events is inconvenient or even impossible. Eyes provides visual locators that allow you to use Eyes technology to analyze an application page and discover the coordinates of predefined graphic entities. You can then use these coordinates to simulate mouse and keyboard events using the standard browser device handler methods.

Overview of using visual locators

A visual locator has two attributes - a rectangular image and a name. Two visual locators can have the same name. This means you can implement one "logical entity" which looks different in different circumstances or states by simply creating two visual locators with the same name.

Using visual locators is a two-step workflow. The first step is to define the set of visual locator images and their names. Once Eyes knows what the locator images and names are, the second step is to use an Eyes query to search for instances of the locator images on the current application page view port.

Visual locators are organized by test application name (i.e. the application name that is passed as a string when calling eyes.open( ).

The following sections describe how to use visual locators in more detail:

• Defining visual locators in Eyes
• Searching for visual locators
• Managing visual locators

Defining visual locators in Eyes

Defining visual locators is a one-time action. You do this by uploading the images using the Eyes SDK and then use the Eyes Test Manager to identify the locator to name it. Once locators have been created, they are available for query in any Eyes test run. If necessary (for example, if you need to search for new visual locators), you can repeat the process described below to define new visual locators.

To define visual locators:

1. Run a program that uses the Eyes SDK to execute checkpoints on pages that include UI elements you would like to interact with.

   Some points to note:

   • The purpose of this program is to upload the images to Eyes and not to validate checkpoints. We are simply using an API that is familiar as a way to import images into the Eyes system.

   • The application name passed to eyes.open( ) in the program that uploads the images to Eyes must be identical to the application name passed to eyes.open( ) in the test that will search for the visual locators associated with those images.

   • A checkpoint can contain more than one visual locator. This is convenient when identifying the visual locators in the Eyes Test Manager.

     It might be helpful if the test name contains some standard text such as "VISUAL-LOC". This makes it easy to find and delete the test runs and baselines used to import the visual locators.

2. Once the test has run, use the Eyes Test Manager Step editor or Test editor to mark rectangular regions around the UI elements you wish to define visual locators for and give them a name. This process consists of the following steps:

• In the Test results page, click on the thumbnail of a step to enter the Step editor for a step that contains one or more UI elements.

• Click to open the visual locator gallery.

• Click to add a new visual locator.

• Use the cross-hair cursor to mark a rectangular region around the UI element. The image should have a significant amount of unique detail so that it can be distingushed from other parts of the image and other visual locators. The image should not include a background that might change. For example, if you are marking a rectangle with rounded corners, it is better to create a smaller image that does not include the area on the outer side of the rounded corners.

• In the dialog box that opens, enter a name for the visual locator. If the name is already in use in the application, a dialog will appear and you can choose from the following options:

  1. Replace any existing images with this locator name and associated this image for that locator name.
  2. Cancel the rename option so that you can enter a different name.
  3. Add a new image to the visual locator name.

  4. Click to cancel creation of a new visual locator

Searching for visual locators

The API to locate a visual locator has the following signature:

Map<String, Array<Rectangle>> elementsList =
   eyes.locate(VisualLocatorSettings locator);

The method takes as a parameter a VisualLocatorSettings type, which is a Fluent interface, meaning that it can be a chain of method calls.

The following example calls the name() method three times, passing it the name of a visual locator in each call. You can call this method once for each visual locator. The call to the first() method, indicates that we only wish to get a single result for each visual locator.

In the next example, the method names() passes a list of visual locator names.

This is an alternative way to pass a number of visual locators and is especially useful when the list of visual locator names is dynamic (e.g. it is read from a file or depends on user input). At the end of the chain, call the all() method instead of the first() method to get all instances of each locator and not just the first.

You can call either first() or all() but not both. If neither is called, the default is first(). Note that using all() requires an exhaustive test of the entire image and takes more time so only use it if you know that there can be multiple instances of a locator and you want to find all of them.

The value returned by the call to eyes.locate is a Map, which maps each of a set of String values to an Array of Rectangles, where each String is one of the locator's names passed as a parameter to the name() or names() methods and the list of Rectangles represents all the graphic elements that Eyes found with that locator name.

The Rectangle class exposes methods getLeft( ) and getTop( ), which give the position in pixels of the top-left corner of the locator graphic relative to the top-left corner of the viewport. It exposes getWidth() and getHeight(), which return the dimensions of the locator graphic.

If the image for a particular locator name is not found in the viewport, the array returned for that name will be a zero length array. If a locator name is passed, and a locator of that name has not been defined on the Eyes server, a value of null is returned instead of an array.

The following code is an example of how to search for a coordinate and then use it to generate an event:

Managing visual locators

You can see and manage visual locators in the Eyes Test Manager in the following ways:

• View visual locators of the application of a given step. See View visual locators in the Step and Test editors.
• View visual locators of a given application. See The global visual locator gallery.

Each of these is described in more detail below, but first we will describe functionality that is common to all types of visual locator gallery.

Visual locator operations

In all types of gallery, you will see a window with one or more visual locator cards - one per visual locator.

Each card contains a thumbnail of the image, the locator name, and the date it was created. A toolbar appears when you hover over a particular card, allowing you to take the following actions:

Copying visual locators

You can copy a visual locator to a different app.

• In the Visual locators panel, click Manage to open the Visual Locators management page.

  

• On a thumbnail, click > Copy to another app.

  

• Select the required app and click Copy.

View visual locators in the Step and Test editors

The visual locator pane in the Step editor and Test editor provides access to all the visual locators defined for the application of the current step.

You can view the locator gallery pane as follows:

1. Enter the Step editor by clicking on a thumbnail in the Test results page.

2. Click to open a panel to the left of the checkpoint image.

   

In addition to the actions you can take on each card, you can also take the following actions:

The global visual locator gallery

The global visual locator gallery allows access to all visual locators defined for your team.

To access the global visual locator gallery:

1. Select the Apps & tests page.

2. Select the application you require from the list of applications on the left.

3. Click .

4. The display will change to show all the visual locators for that application.

   

5. You can change the currently shown application using the application menu in the top-right corner.

6. You can also reach this page from any step by clicking on in the Step editor or Test editor.

In addition to the actions you can take on each card, you can also take the following actions:

	As you type into the filter box, Eyes will update the display, showing only visual locator cards for visual locators with names including the typed text.
	Select or deselect all the visual locators visible. If you have applied a filter, then only the visual locators that are displayed are selected or deselected.
	Delete all selected visual locators.

For further information, see our video, Using Visual Locators Inside Automated Functional Tests with Applitools Eyes: