The EyesLibrary configuration file
The Robot Eyes configuration file allows you to control various aspects of Eyes testing. The file is in YAML format.
The configuration file content
If you have installed
EyesLibrary you can initialize the config file with the command:
python -m EyesLibrary init-config
applitools.yaml has several top sections:
web – Web settings
mobile_native – Mobile native settings
web_ufg – Ultrafast Grid settings
The shared settings provide defaults for all run types. The other sections are for specific types of run and define their own configuration values. By default, they inherit the shared settings, but they can override any configuration if necessary.
The following fields can appear top-level of applitools.yaml conf. Unless noted otherwise, you can set these values under the other top-level fields (web, mobile_native and selenium_ufg) to define the value when working in that specific mode:
Use this example to define the URL of your Eyes server. If you have an on-prem server or private cloud, you must specify this. If you use the public Eyes server (https://eyesapi.applitools.com), then setting this key is optional since this is the default value. If you set the environment variable APPLITOOLS_SERVER_URL to the server URL, then Eyes uses that value as the default.
You must set this value to your Eyes API key. See How to retrieve your API key for details.
app_name: my app name
Set the value of the application name property. The application name is one of the five properties that define the baseline. The application name provides the scope to the test name, and tests can be grouped and managed on a per-application basis. See The Apps & tests page for details.
We recommend that you use the same application name for all tests in the same app; Applitools features rely on a shared app name across tests.
viewport_size:width: 1920height: 1080
Set the viewport size to be used by the browser for the test. The viewport size sets the browser window size, and it is also one of the five values that defines the baseline – the set of images against which this test is checked. See Working with baselines for more details.
batch:#optional id: YOUR_BATCH_ID name: YOUR_BATCH_NAME batch_sequence_name: YOUR_BATCH_SEQUENCE_NAME properties: - name: BatchPropery 1 value: some value 1 - name: BatchPropery 2 value: some value 2
Set the name of the batch. In the Test Manager, all of the results of tests that were executed as a single batch are displayed together and can be managed and operated on as a group. See How to organize your tests with batches for more details.
id: Specify the ID explicitly (optional) – see Batching tests in a distributed environment for more details.
batch_sequence_name: For detailed analysis across batches. See Insights batch statistics for more details.
properties: The key/value pairs you define can be viewed as part of the batch properties in the Test Manager and can be used for filtering and grouping. See Using user-defined batch properties for more details.
Set the branch to be used as the baseline of the run. The use of a branch in Eyes corresponds to the use of a branch in the source code version control repository. The branch stores one or more baselines. Typically, every version of your software that runs on a source control branch has a corresponding Eyes branch that stores the Eyes baselines that represent the expected results of tests running on that software branch.
If the branch exists, and a matching baseline exists in the branch, then it is used as the basis for comparing the checkpoint images. If the branch does not exist yet and a parent branch, defined by the
parent_branch_name value, exists and has a matching baseline, then that baseline is used. Otherwise, the default branch is used. The branch named by the
branch_name value is actually created when you view the tests results of a run on this branch, make some changes (i.e. accept a new, changed, or deleted step), and save the changes.
Set the parent branch from which newly created branches get their initial baseline. See the explanation of the
branch_name for more details.
Set the name of the branch that the baseline reference is taken from and to which new and accepted steps are saved to.The baseline must exist. For details how to create a branch from an existing branch, see Copying baselines between branches. Do not use this option if you are using the
Set this value to specify the name of the environment used to determine the baseline.
Eyes stores a set of mappings from an environment name to an environment E, where E is defined as a triplet <OS, Browser, ViewportSize>. You can specify that Eyes uses the baseline defined by the environment name instead of the test environment by assigning this value to the name of the environment.
true, steps that have mismatches are automatically saved to the baseline.
In the usual workflow, if Eyes finds mismatches, you use the Test Manager to view the mismatches, accept or reject the steps with mismatches, and then update the baseline with the images captured in the accepted steps. This method allows you to instruct Eyes so that where steps have mismatches, or where there are new or missing steps, the corresponding steps in the baseline should be updated with the images captured in the current run of the test.
Setting this value to
true, completely overwrites your baseline. It is usually preferable to see the results in the Test Manager and, if necessary, to accept all differences there. If you use this method in a particular circumstance, remember to disable this setting for every day use.
Set this value to specify the maximum amount of time Eyes should try to perform a match on the fully captured image.
Since a browser can take time to render a page (because it is complex, or because of slow network speeds), if Eyes detects mismatches, it initially assumes that the mismatch is because the render has not completed yet, and it retries the match after a short wait. You can set this value to determine how much time Eyes spends retrying the matching before declaring a mismatch.
proxy: url: "http://someproxy-url.com" host: myhostid port: 7200 username: my name password: this is my secret password
If you run tests behind a firewall that can’t access the Eyes server directly, then you can define a proxy server, and the commands are sent to the Eyes server via the proxy server. You must specify the URL. You only need the host, port, username, and password if they are required by the proxy server.
false, new tests will not be automatically saved to the baseline by default.
This option is enabled by default (i.e. new tests are saved automatically to the baseline), so use this method to disable the default behavior.
properties:- name: YOUR_PROPERTY_NAMEvalue: YOUR_PROPERTY_VALUE
Set a list of user-defined properties, each of which is a key/value pair. These properties are defined as properties of the test as opposed to batch properties.
Desktop and mobile browsers with Selenium and Appium.
By default, Eyes only captures the image visible in the browser viewport. If you set this value to true, Eyes captures and checks all of the content on the web page (anything accessible by scrolling the top-level element).
Set this value to specify the amount of time (in milliseconds) that Eyes should wait before capturing a screenshot. This can be used if the image takes time to stabilize if, for example, there is an animation, meaning there is no synchronous way to wait for the image to be stable before calling the checkpoint command.
When a large image is captured with multiple sub-images using scrolling and stitching, Eyes waits the amount of time specified by this method before capturing each sub-image. Setting a value less than or equal to zero sets the default wait time.
web:stitch_mode: CSS# Scroll | CSS
When you set
force_full_page_screenshot to true, Eyes captures the entire page, element, or frame by capturing multiple images across the page and stitching them together. The stitch mode defines the method for doing this. The default value is
Scroll, but the recommended value is
Set this value to
true so that Eyes hides the scrollbars before capturing screenshots, or set it to
false to disable hiding the scrollbars. Hiding the scrollbars is recommended to avoid false mismatches caused by differences in the scrollbar position each time the checkpoint is captured and checked.
Set this value to true so that Eyes hides the cursor before the screenshot is captured. This is recommended to avoid mismatch artifacts caused by a blinking cursor.
Mobile apps with Appium.
Set this value to
true when the device under test is a simulator and not a real device. Eyes needs this information in order to operate correctly.
Set the maximum number of Eyes tests that can run concurrently when using the Ultrafast Grid.
web_ufg: visual_grid_options: - key:YOUR_VISUAL_GRID_OPTION value: YOUR_VISUAL_GRID_OPTION_VALUE
Set this value to pass an option of the Ultrafast Grid. For a list of possible values, see visual_grid_options property.
Under some circumstances, rendering of some resources may be missing. Set this value to
true to eliminate these problems.
Normally, when a webpage contains content from multiple sites, rendering of the page succeeds. If there are rendering errors due to incorrect rendering of content from other sites, set this value to
false. When you do this, iframes that originate from other websites are rendered as blank pages.
By default, cookie information in the browser is sent to the Ultrafast Grid. Set this value to
true to disable sending cookie information when sending checkpoint resources to the Ultrafast Grid.
Configure the SDK to capture multiple DOM images for multiple viewport sizes.
Setting this field to
Setting this YAML value to
true enables this feature and extracts a DOM for every distinct viewport size configured. Alternatively, you can pass a list of distinct viewport widths, in which case the DOM is extracted for those particular viewport widths.
true, Ultrafast Grid test results are added to
robot report.html only when the test suite is finished,
As Ultrafast Grid tests are asynchronous, Eyes keywords do not wait for checks to actually finish, to speed up suite execution. The checks are finished in the background and if a particular check has failed, you can see the result in the eyes.applitools.com dashboard. If this option is
false, these failures will not be seen in
report.html which will show all tests as passed.
If this option is set to
true (default), the EyesLibrary waits for all test results at the end of the suite and then edits
report.html marking failed tests as failed, and adds links to eyes dashboard.
Setting the Ultrafast grid execution environments
When using the Ultrafast grid, you must specify one or more execution environments. There are three types of environment, each with a list of supported devices. The three types are defined by the YAML fields
web_ufg:browsers:desktop for a list of desktop browsers,
web_ufg:browsers:ios IOS device simulation, and
web_ufg:browsers:chrome_emulation for Chrome emulation.
web_ufg: browsers: desktop: - browser_type: CHROME# values from BrowserType width: 1900 height: 1800
For each browser, specify the browser type and the viewport width and height.
web_ufg: browsers: ios: -device_name: iPhone_12_Pro# names from IosDeviceName screen_orientation: PORTRAIT# PORTRAIT | LANDSCAPE ios_version: LATEST# LATEST | ONE_VERSION_BACK
For each IOS device to be simulated, specify the device name, the screen orientation (
LANDSCAPE), and the ios version (
web_ufg: browsers: chrome_emulation: - device_name: iPhone_4 # names from DeviceName screen_orientation: PORTRAIT # PORTRAIT | LANDSCAPE