- Guillermo Rauch – Founder & CEO, Zeit (@rauchg, @zeithq)
- Gleb Bahmutov – VP of Engineering, Cypress (@bahmutov, @cypress_io)
- Simen Bekkhus – Developer, Folio (@SBekkhus, @folio, @fbjest)
- Adam Carmi – Co-Founder and CTO, Applitools (@carmiadam, @applitools)
- Kevin Lamping – Developer, Test Automation (@klamping)
- Vikram Subramanian – Googler, Angular Universal (@vikerman)
- Tracy Lee – Founder, This Dot & RxJS Core Team (@ladyleet , @thisdotmedia)
The panel covered a broad range of test issues. Adam began his segment by zeroing in on a simple problem: how small changes can have huge impacts on front end behavior. Specifically, how can developers account for changes in their CSS?
Also published on Medium.
CSS changes can affect look and feel of a set of pages – leaving them visually problematic for users. With an existing CSS, developers can create functional tests around expected behaviors and visualizations. But, CSS changes can have unknown or unexpected behavior changes that impact functional behavior.
How Do Front End Developers Test?
Adam started out by asking how most front end developers handle visual testing. He acknowledged the reality that most frontend developers don’t do testing, and those that do usually run unit tests. Some developers create functional UI testing with tools like Cypress, Webdriver.IO, Selenium, and other tools.
Adam gave an example of a typical login page test. The page requires that a user provide both an email and a password. If either or both are missing, the page should respond with “Required” to indicate that content is missing.
This kind of behavior is easy to test in Cypress. The test below instructs Cypress to run the test behavior and ensure that the screen shows the “Required” error for each box.
Functional Tests Miss UI Changes
After walking through the functional test approach, Adam asked,
“What happens if another developer on our team, or one on a different team, changes the CSS… “
What happens when the change unintentionally modifies pages with an existing functional test? The test would still pass because the functional elements haven’t changed. If we don’t have other means to verify how the UI looks like, we just might this change might go unnoticed and push this back to production.
Why Automated Visual UI Testing Matters
Then, Adam discussed the range of problems that result in visual UI issues. He said that the typical web page,
“… can have hundreds of elements and each element has dozens of attributes that affect the way that it is rendered.”
“How can we make sure that the application looks right renders correctly on all different form factors and viewport sizes that affect its layout? Especially when each browser uses a different layout engine, and each has a slightly different interpretation for the same domain CSS? Even if we don’t change a line of code a browser version update might be incompatible with the application.”
Then, Adam talked about visual UI testing and how Applitools addresses these issues.
“The answer,” he said, “is visual testing and the way that you do it with Applitools. Basically, you just add a visual assertion, or a visual checkpoint, as we call it, which validates the entire window with one line of code. It captures a screenshot of the page, or a component (if you want to test a specific component) and compares that with a baseline image that defines the expected appearance of the page, and if there is any difference it the test.”
Next, Adam described how Applitools new Visual Grid lets engineers automatically run their tests on a range of target environments and devices.
“What we do is basically capture all the resources that are used by the browser to render the page locally. We upload the differences to our Visual Grid, which is where we host our own devices and browsers. And then we simultaneously render the same resources on all these different combinations of environment details. You can run hundreds of visual tests in less than a minute – super powerful and super cool.”
Adam then demonstrated Applitools Eyes in action.
He noted that when Applitools Eyes detects a difference, you can see the difference highlighted on the screen. The dashboard lets you compare the checkpoint and baseline image. You can toggle between the two images and clearly see what changed. Applitools Eyes is simple enough to use that anyone on your team who knows your application can run these tests.
Root Cause Analysis
Adam then showed the Applitools Eyes Root Cause Analysis option.
“Beyond just seeing what’s changed, we can show you what caused the change,” he said. “For example, if we click on a difference, Root Cause Analysis highlights the code difference between the checkpoint and the baseline. Root Cause analysis highlights changes in CSS that affect color, text case, and even indicate misbehavior on a specific platform. Root Cause Analysis saves a lot of time by presenting the code differences, rather than requiring the developer to dig into the code.”
Ease of Use
Next, Adam spoke about ease of use.
“We have plenty of features in the product that allow you to scale up your tests without increasing your maintenance overhead,” he said. Maintaining a baseline is very simple – there is a workflow for accepting new screenshots. And, Applitools Eyes can isolate errors to a single root cause, so an error touching hundreds of pages will show up as one error.
Finally, Adam spoke about Applitools integration with development and continuous development – continuous integration tools.
“The last thing I’ll mention is that visual testing needs to integrate with your day-to-day workflow. We integrate with bug trackers and with continuous integration systems like Jenkins, Travis CI, and many others. We also integrate with code repositories like Bitbucket and GitHub. You can quickly view, initiate, and maintain visual test results directly from your pull requests.”
See The Whole Webinar
We have linked the entire webinar here. Enjoy!
Also published on Medium.