This article describes how to adapt an Eyes visual test that was created before the availability of the Ultrafast Grid to enable use of the Ultrafast Grid. The adapted code is based on the Runner API. Once your test uses this API, you can easily switch between running with or without the Ultrafast Grid as required. The required changes are limited to the setup and wrap up code, but do not impact the test code where images are captured and checked.
The article assumes that you have a general knowledge of working with the Eyes SDK and what the Ultrafast Grid is. It focuses on introducing new concepts and the changes you need to make to existing code so that it can use the Ultrafast Grid.It includes a concise example, designed to get you up and running converting existing code to use the Ultrafast Grid as quickly as possible. For an overview of what the Ultrafast Grid can do and how it does it see the article Introduction to the Ultrafast Grid. The article Ultrafast Grid Configuration is more expansive than the current article, describing in more detail the various options that are available.
- Runner interface: This object manages access by multiple Eyes instances to the Ultrafast Grid and the Eyes Server. There are two types of runners, one for working with the Ultrafast Grid and one for working without it.
- Configuration class: An object that supports configuration of the Ultrafast Grid and is also useful for applying a common configuration on multiple Eyes instances.
Converting a legacy test suite to use these new concepts includes the following steps:
- Before any test is run:
- Create a runner object using the
visualgridrunnerconstructor to run with the Ultrafast Grid or with the
classicrunnerconstructor to run without the Ultrafast Grid.
- If using the Ultrafast Grid, define one or more browser configuration(s) using the
- Set other global configurations that you want to apply to all tests.
- For each test:
- Create the Eyes instance, passing the runner object.
- If you previously set the server URL by passing it to the Eyes constructor, remove this parameter and set it when setting the global configurations.
- Replace calls to the Eyes
eyes$closemethod with the method
- After all the tests have been executed, use the method
visualgridrunner$getAllTestResultsto wait for all the tests to complete and to return all of the test results.
The sections below explain the above steps in more detail and illustrate them with a simple example.
Example test and how to migrate to it
- runner This object coordinates multiple tests, each running with it's own Eyes instance. The object also collects the test results and makes them available when all the tests are complete. You need one runner instance for the entire test suite.
- suiteconfig This object represents all the Ultrafast Grid and Eyes configuration values. Before any tests starts, you set it up with default values and it is then applied to each test before the test starts.
- Create an instance of the class
- Define which browser configurations the Ultrafast Grid should render.
- Set up any global configuration values that apply to all the tests.
The type of runner you pass to the Eyes constructor when creating it determines if its checkpoints are rendered using the Ultrafast Grid or not.
object passed to the
constructor is used to configure the runner. The example shows the method
being passed. This option is used to limit the maximum number of Eyes tests that the runner will run simultaneously.
Increasing the concurrency value can allow your test suite to run faster, but the maximum number of tests that can run simultaneously depends on your Eyes plan, if concurrency has been allocated at a team level, and what other tests are being run on your account at the same time.
The integer value passed to the constructor limits the maximum number of Eyes tests that interact concurrently with the and the . Increasing the concurrency value can make your test suite run faster, but the maximum possible concurrency depends on your Eyes plan.
The Configuration object
configuration object stores the required values of all the configuration values. The following snippet shows this object being created and configured with Ultrafast Grid browser configurations and Eyes configuration values that are typically common for all tests.
When each test is set up, as demonstrated in the
The example demonstrates the use of the method
to configure for which desktop browsers, emulated and simulated devices the Ultrafast Grid should generate images for each of the checkpoints. You can read more about these methods and their variations in the article Ultrafast Grid Configuration..
You can also use the
object to set up
configuration values that need to be applied to the entire test suite.The
object supports most of the
setXXX methods exposed by the
object. For an overview of common test suite level configuration values see
Test suite configuration.
The snippet below shows how to set up the Eyes instance and web driver objects that are required for each test. For conciseness, this example is not multithreaded and uses class fields for the Eyes and web driver objects. These are set with new instances for each test. If your test is multithreaded, then each thread must have its own instance of the Eyes and web driver objects.
Create the Eyes instance by calling the constructor, passing it the runner object that was created in the
beforetestsuitemethod. Passing a runner of type visualgridrunner indicates to Eyes that this test should run using the Ultrafast Grid. If a different type of runner is passed, then the test will run without the Ultrafast Grid.
In previous versions of the SDK, if you used a private cloud or an on-premise server, you could pass the URL of the cloud or server when instantiating the Eyes object. This is no longer supported, and you should set it using the
configuration$setserverurlmethod on the suiteconfig object in the
beforetestsuitemethod as shown in the example above.
- Use the Eyes
eyes$setconfigurationmethod, passing it the suiteconfig object to apply the test suite default configuration values to the Eyes instance. Note that this method sets all the configuration values, either to the values defined explicitly or to the SDK default values. We will see in the next section how you can change only specific configuration values.
- If in your code you use one or more eyes.setXXX commands for the initial configuration of the Eyes instance, then it is recommended that you replace these calls with the equivalent calls to the global configuration object as shown in the method
beforetestsuite, since the eyes.setXXX might be deprecated at some point.
The following snippet shows some typical test code, which first does some test specific configuration and then executes the test:
- A test may have some test-specific configuration. For example, we configured the application name in the global configuration since it is common to all the tests. Now in the test we need to configure the test name.
- Open the Eyes session by calling the method
In legacy code, this method took as a parameter the application name, test name and viewport size. If these are now set in the Configuration object, then you don't need to pass them as parameters, although you can do so if you want to override the default set in the configuration.
The snippet below shows the typical content for a method that wraps up after every eyes test.
- Call the eyes method
eyes$closeasyncinstead of the
eyes$closemethod. Calling the
eyes$closeasyncmethod signifies the end of the checkpoints, but unlike calling
eyes$close, the method does not wait for results, so execution can proceed immediately to the next test.
If your test infrastructure indicates that the test terminated abnormally, then you should call
After all the tests have been closed, you wait for all the test processing to complete and obtain all of the tests results using the method
visualgridrunner$getalltestresults as shown in the snippet below.
The call to
visualgridrunner$getalltestresultsreturns an iterator that you can use to get the test results of each browser configuration of each test. For example, if the test is configured with five browser configurations and the test method is called 10 times, you will get 50 test results.
- The object returned by the iterator,
testResultContainer, supports the following methods or properties:
testResultContainer$getexception$morp: Returns a non null status value if a failure prevented the test from completion.
testResultContainer$gettestresults$morp. Returns a
testresultsobject which contains the results of each test or a null if an exception occurred and test results are not available.
getAllTestResults() method takes an optional Boolean parameter that determines what happens if Eyes finds mismatches in the test. If the parameter is
true, or no parameter is passed, then an exception is raised is any test had a mismatch, was missing or if there were new checkpoints. Typically, if you want your test framework to capture and handle the exceptions in the same way as any other assertion failure in your tests then pass "
true. If you want to generate a custom report or otherwise process the test result programmatically upon test completion then pass