Writing visual tests that use the Native Mobile Grid

In a previous article, we described how to write an Eyes visual test using the ClassicRunner. When using the ClassicRunner, the Eyes SDK uses a real mobile device, Android emulator, or iOS simulator to capture screenshots for the checkpoints. The current article explains how to adapt this code so that it uses the Native Mobile Grid. When using the Native Mobile Grid, the Eyes SDK captures a view hierarchy snapshot and other resources and sends them to the Grid server, where the checkpoint image is rendered concurrently for all relevant configurations, along with other checkpoints in your test suite.

Using the Native Mobile Grid considerably reduces test time and also enables you to test multiple mobile devices in a single test run. For a full description of the Native Mobile Grid and what it can do for you, see the article Introduction to the Native Mobile Grid.

Overview

Most aspects of coding Eyes visual tests are the same irrespective of whether you use the Native Mobile Grid or not. The only difference is in how you set up the test suite. The set-up process involves several steps:

The first step is typically done as part of the instrumentation process.

The instrumentation process is available for both iOS and Android platforms, to ease the integration of the test application with Applitools.

Applitools collects the application resources by instrumenting a dedicated library (“Grid Library”) into the test application, in order to render the image across the other configured devices.

Specifying the use of the Native Mobile Grid

The Java snippet below illustrates how you can easily switch to using the Native Mobile Grid by assigning the runner an instance of a VisualGridRunner class instead of a ClassicRunner class.

Copy

public void beforeTestSuite() {
      /* REMOVE
      runner = new ClassicRunner();
      */
      runner = new VisualGridRunner(new RunnerOptions().testConcurrency(10));
      // continued below....
  }

The type of runner you pass to the Eyes constructor when you create it for each test determines whether its checkpoints are rendered using the Native Mobile Grid or not.

The RunnerOptions passed to the VisualGridRunner constructor is used to configure the runner. The example shows the method $runnerOptions.testConcurrency being passed. This option is used to limit the maximum number of Eyes tests that the runner will run simultaneously. Increasing the concurrency value can allow your test suite to run faster, but the maximum number of tests that can run simultaneously depends on your Eyes plan, if concurrency has been allocated at a team level, and what other tests are being run on your account at the same time.

Baselines

By default, when running tests using the Native Mobile Grid, every mobile device configuration uses the baseline implied by the execution environment of that configuration, i.e. the Device’s viewport size and its operating system:

When adding a mobile device (e.g. using the Configuration.addMobileDevice method), the Device and Viewport size baseline attributes are defined by the parameters of the method. The Operating system attributes will depend on the device type.

Example

The code below shows a complete example of a test that runs using the Native Mobile Grid.

The examples are for Android and iOS platforms, based on Java Appium and Espresso with Kotlin for Android and XCUI with Swift for iOS.

The code examples below are for demonstration purposes only. In order to be able to execute tests with the Native Mobile Grid, additional prerequisites are required.

Android

Framework: Appium, Language: Java

Add the following dependency to the pom.xml file:

Copy

<dependency>
    <groupId>com.applitools</groupId>
    <artifactId>eyes-appium-java5</artifactId>   
    <version>5.16.0</version>
</dependency>

Write your test code:

Framework: Espresso, Language: Kotlin

Copy

package com.applitools.android.ufg.espresso

import androidx.test.core.app.ActivityScenario
import com.applitools.android.ufg.espresso.ui.login.LoginActivity
import com.applitools.eyes.android.common.AndroidDeviceName
import com.applitools.eyes.android.common.logger.Logger
import com.applitools.eyes.android.common.logger.StdoutLogHandler
import com.applitools.eyes.android.components.androidx.AndroidXComponentsProvider
import com.applitools.eyes.android.espresso.Eyes
import com.applitools.eyes.android.espresso.fluent.Target
import com.applitools.eyes.android.espresso.visualgrid.RunnerOptions
import com.applitools.eyes.android.espresso.visualgrid.VisualGridRunner
import org.junit.Before
import org.junit.Test

class KotlinTest {

    @Before
    fun beforeTest() {
        ActivityScenario.launch(LoginActivity::class.java)
    }

    @Test
    fun testLogin() {
        val logger = Logger()
        logger.logHandler = StdoutLogHandler(true)
        val options = RunnerOptions()
        options.apiKey("<API_KEY_HERE>")
        val runner = VisualGridRunner(options)
        val eyes = Eyes(runner)
        eyes.componentsProvider = AndroidXComponentsProvider()
        eyes.configuration =
            eyes.configuration
                .addMobileDevice(AndroidDeviceInfo(AndroidDeviceName.Pixel_4))
                .addMobileDevice(AndroidDeviceInfo(AndroidDeviceName.Pixel_4_XL))
                .addMobileDevice(AndroidDeviceInfo(AndroidDeviceName.Pixel_3_XL))
        yes.logHandler = logger.logHandler
        try {
            eyes.open("AndroidX NMG test app", "Test Login page NMG Kotlin")
            eyes.check(Target.window().withName("Check"))
            eyes.closeAsync()
        } finally {
            runner.allTestResults
            eyes.abortIfNotClosed()
        }
    }
}

iOS

Framework: XCUI, Language: Swift

Copy

import EyesXCUI
import XCTest

class NMGDemoTests: XCTestCase {

  static var visualGridRunner = NMGDemoTests.configureRunner()
  static var eyes = NMGDemoTests.configureEyes(runner: visualGridRunner)

  private static func configureRunner() -> VisualGridRunner {
    let options =
      RunnerOptions().testConcurrency(5).apiKey("API_KEY_HERE")
    return VisualGridRunner(options: options)
  }

  private static func configureEyes(runner: VisualGridRunner) -> Eyes {
    let eyes = Eyes(runner: runner)
    let config = Configuration()
    let iPhone8 = IosDeviceInfo(deviceName: .deviceiPhone8)
    let iPhoneProMax = IosDeviceInfo(deviceName: .deviceiPhone12ProMax)
    let iPhone13 = IosDeviceInfo(deviceName: .deviceiPhone13)
    config.addMobileDevices([iPhone8, iPhoneProMax, iPhone13])
    eyes.configuration = config
    return eyes
  }

  override class func setUp() {
    let app =
      XCUIApplication(bundleIdentifier: "com.mycompany.myapp")
    app.launchEnvironment = ["UFG_LIB_FRAMEWORK":"", "NML_API_KEY":"API_KEY_HERE"]
    app.launch()

    visualGridRunner = NMGDemoTests.configureRunner()
    eyes = NMGDemoTests.configureEyes(
      runner:
        visualGridRunner)
    eyes.testedApplication = app
    eyes.open(
      withApplicationName: "My iOS App",
      testName:
        "eyesXCUI NMG Demo")
  }

  func testMyCheck() throws {
    Self.eyes.check(
      withTag: "Check",
      andSettings:
        Target.window())
  }

  static override func tearDown() {
    eyes.closeAsync()

    let allTestResults =
      visualGridRunner.getAllTestResultsShouldThrowException(false)
    print(allTestResults)
  }
}