Migrating code to use the Ultrafast Grid

This article describes how to adapt an Eyes visual test that was created before the availability of the Ultrafast Grid to enable use of the Ultrafast Grid. The adapted code is based on the Runner API. Once your test uses this API, you can easily switch between running with or without the Ultrafast Grid as required. The required changes are limited to the setup and wrap up code, but do not impact the test code where images are captured and checked.

The article assumes that you have a general knowledge of working with the Eyes SDK and what the Ultrafast Grid is. It focuses on introducing new concepts and the changes you need to make to existing code so that it can use the Ultrafast Grid.It includes a concise example, designed to get you up and running converting existing code to use the Ultrafast Grid as quickly as possible. For an overview of what the Ultrafast Grid can do and how it does it see the article Introduction to the Ultrafast Grid. The article Ultrafast Grid configuration is more expansive than the current article, describing in more detail the various options that are available.

Overview

With the introduction of the Ultrafast Grid, two new concepts are added to the SDK:
  • Runner interface: This object manages access by multiple Eyes instances to the Ultrafast Grid and the Eyes Server. There are two types of runners, one for working with the Ultrafast Grid and one for working without it.
  • Configuration class: An object that supports configuration of the Ultrafast Grid and is also useful for applying a common configuration on multiple Eyes instances.

Converting a legacy test suite to use these new concepts includes the following steps:

  • Before any test is run:
    • Create a runner object using the visualgridrunner constructor to run with the Ultrafast Grid or with the classicrunner constructor to run without the Ultrafast Grid.
    • If using the Ultrafast Grid, define one or more browser configuration(s) using the configuration class methods configuration$addbrowser, configuration$adddeviceemulation or configuration$addbrowsers.
    • Set other global configurations that you want to apply to all tests.
  • For each test:
    • Create the Eyes instance, passing the runner object.
    • If you previously set the server URL by passing it to the Eyes constructor, remove this parameter and set it when setting the global configurations.
    • Replace calls to the Eyes eyes$close method with the method eyes$closeasync.
  • After all the tests have been executed, use the method visualgridrunner$getAllTestResults to wait for all the tests to complete and to return all of the test results.

The sections below explain the above steps in more detail and illustrate them with a simple example.

Example test and how to migrate to it

Each of these are discussed in the sections that follow., the full example which appears at the end of the article are based on TestNG (Java), NUnit(C#) and Mocha (JavaScript).

The Runner and Configuration Objects

The new objects managed in the setup and wrap up code are:
  • runner This object coordinates multiple tests, each running with it's own Eyes instance. The object also collects the test results and makes them available when all the tests are complete. You need one runner instance for the entire test suite.
  • suiteconfig This object represents all the Ultrafast Grid and Eyes configuration values. Before any tests starts, you set it up with default values and it is then applied to each test before the test starts.

Set up the test suite

The one-time test suite setup includes the following steps:
  • Create an instance of the class VisualGridRunner.
  • Define which browser configurations the Ultrafast Grid should render.
  • Set up any global configuration values that apply to all the tests.

Creating the VisualGridRunner object


                                

                                
 The runner object can be an instance of one of these classes: 
                                    
    
                                        
  • 
                                             visualgridrunner : To use the Ultrafast Grid.
    • 
                                        
  •   classicrunner: When not using the Ultrafast Grid. 
    • 
                                    

                                

                                
The type of runner you pass to the Eyes constructor when creating it determines if its checkpoints are rendered using the Ultrafast Grid or not.

                                
The 
			RunnerOptions object passed to the VisualGridRunner constructor is used to configure the runner. The example shows the method RunnerOptions.testConcurrency being passed. This option is used to limit the maximum number of Eyes tests that the runner will run simultaneously.
		Increasing the concurrency value allows your test suite to run faster, but the maximum number of tests that can run simultaneously depends on your Eyes plan, if concurrency has been allocated at a team level, and what other tests are being run on your account at the same time.

                                

		The  integer value passed to the  constructor  limits the maximum number of Eyes tests that interact concurrently. Increasing the concurrency value can make your test suite run faster, but the maximum possible concurrency depends on your Eyes plan.

                                
The Configuration object

                                
The  configuration object stores the required values of all the configuration values. The following snippet shows this  object being created and configured with Ultrafast Grid browser configurations and Eyes configuration values that are typically common for all tests.		

                                

                                

                                
When each test is set up, as demonstrated in the 
				  
				method below, the Configuration  object is used to initialize the Eyes object. 
				Note the use of the fluent API style to simplify setting up  multiple configuration values.

                                
The example demonstrates the use of the method  
		addbrowser
					 to configure for which desktop browsers, emulated and simulated devices  the Ultrafast Grid should  generate images for each of the checkpoints. You can read more about these methods and their variations in the article Ultrafast Grid configuration..

                                
	You can also use the 
					Configuration
					object to set up 
					Eyes 
					configuration values that need to be applied to  the entire test suite.The 
					Configuration 
					object supports most of the 
					setXXX methods exposed by the
					Eyes 
					object. For an overview of common test suite level configuration values see 
					Test suite configuration.

                                
Set up before each test

                                
The snippet below shows how to set up the Eyes instance and web driver objects that are required for each test. For conciseness, this example is not multithreaded and uses class fields for the Eyes and web driver objects. These are set with new instances for each test. If your test is multithreaded, then each thread must have its own instance of the Eyes and web driver objects. 

                                
    
                                    
  • 
                                        
    Create the Eyes instance by calling the constructor, passing it the runner object that was created in the
					 beforetestsuite
						method. Passing a runner of type  visualgridrunner indicates to Eyes that this test should run using the Ultrafast Grid. If a different type of runner is passed,  then the test will run without the Ultrafast Grid.
					
    
                                        
    In previous versions of the SDK, if you used a private cloud or an on-premise server, you could pass the URL of the cloud or server when instantiating the Eyes object. This is no longer supported, and you should set it using the  configuration$setserverurl 
						method on the
						 suiteconfig object
					in the 
		 beforetestsuite
					method as shown in the example above.
    
                                    
    • 
                                    
  • Use the Eyes  eyes$setconfiguration method, passing it the
  suiteconfig
					object to apply the test suite default configuration values to the Eyes instance. Note that this method sets all the configuration values, either to the values defined explicitly or to the SDK default values. We will see in the next section how you can change only specific configuration values.
    • 
                                    
  • If in your code  you use one or more eyes.setXXX commands for the initial configuration of the  Eyes instance, then it is recommended that you replace these calls with the equivalent calls to the global configuration object as shown in the method 
				 beforetestsuite, since the eyes.setXXX might be deprecated at some point.
				
    • 
                                

                                
The test

                                
The following snippet shows some typical test code, which first does some test specific configuration and then executes the test:

                                
    
                                    
  • A test may have some test-specific configuration. For example, we configured the application name in the global configuration since it is common to all the tests. Now in the test we need to configure the test name.

    • 
                                    
    
Note the sequence to change only specific configuration values:
					
                                        
      
                                            
    • Call  eyes$getconfiguration to get the current configuration values.
      • 
                                            
    • Make changes to the current configuration.
      • 
                                            
    • Set the updated configuration using the method  eyes$setconfiguration. 
      • 
                                        
    
                                    
    
                                    
  • Open the Eyes session by calling  the method  eyes$open. 
    In legacy code, this method took as a parameter the application name, test name and viewport size. If these are now set in the Configuration object, then you don't need to pass them as parameters, although you can do so if you want to override the default set in the configuration.
    • 
                                

                                
Wrap up after each test

                                
The snippet below shows the typical content for a method that wraps up after every eyes test.

                                
    
                                    
  • Call the eyes method   eyes$closeasync instead of the  eyes$close method. Calling the  eyes$closeasync method signifies the end of the checkpoints, but unlike calling  eyes$close, the method does not wait for results, so execution can proceed immediately to the next test.
    • 
                                    
  • 
			If your test infrastructure indicates that the test terminated abnormally, then you should call    eyes$abortifnotclosed instead of  
			 eyes$closeAsync.
		
    • 
                                

                                
Wrap up the test suite and obtain the test results

                                
After all the tests have been closed, you wait for all the test processing to complete and obtain all of the tests results using the method  visualgridrunner$getalltestresults as shown in the snippet below.

                                
    
                                    
  • 
				The call to 
			 visualgridrunner$getalltestresults 
				returns an iterator that you can use to get the test results of each browser configuration of each test. For example, if the test is configured with five browser configurations and the test method is called 10 times, you will get 50 test results. 
    • 
                                    
  • The object returned by the iterator,  	
		 testResultContainer, supports the following methods or properties:
    • 
                                    
      
                                        
    • 
                                             testResultContainer$getexception$morp: Returns a non null status value if a failure prevented the test from completion. 
      • 
                                        
    •  testResultContainer$gettestresults$morp. Returns a  testresults object which contains the results of each test or a null if an exception occurred and test results are not available.

      • 
                                    
    
                                

                                
The getAllTestResults() method takes an optional Boolean parameter that determines what happens if Eyes finds mismatches in the test. If the parameter is true, or no parameter is passed, then an exception is raised is any test had a mismatch, was missing or if there were new checkpoints. Typically, if you want your test framework to capture and handle the exceptions in the same way as any other assertion failure in your tests then pass "true. If you want to generate a custom report or otherwise process the test result programmatically upon test completion then pass false. 

                                
Full Example

                                
The code below shows an example of a visual test using the  visualgridrunner , and  configuration  object to use Eyes with the Ultrafast Grid.