Case study

CloudShare: provides a secure, self-service public cloud computing services & cloud test labs that extend internal IT capabilities

Cloud-computing Provider Reduces UI Automation Code by Over 50% with Applitools Eyes

Highlight

This is the story about how CloudShare reduced UI Automation code by over 50% while greatly increasing overall test coverage with Applitools Eyes automated visual testing.

About CloudShare's Test Automation Infrastructure

CloudShare uses Production environments to host and run all test environments (for testing both Production and local development labs). This includes machines from most modern Windows operating systems, and with all browsers combinations supported (Chrome, Firefox and Internet Explorer 9+).

CloudShare's automated tests are executed with each local deployment (which happens several times each day), with each Production deployment (every Sunday). In addition, full regression cycles executed nightly and throughout the weekend.

The Challenge
  • Testing on multiple browsers, screen resolutions, operating systems and devices
  • Manual UI regression testing slows down deployment
  • Automation code requires a lot of maintenance
Solution

With the previous testing tool used, CloudShare was limited to Internet Explorer, which caused them to miss a large number of bugs that only showed up on other browsers. With Applitools Eyes, this is no longer an issue, as the automated visual testing solution validates the app visual & functional correctness across all browsers, devices, screen resolutions and operating systems. CloudShare now tests in a variety of screen resolutions, which allows them to ensure the UI looks right – across the board.

An important capability CloudShare finds extremely useful is the ability to test pages with dynamic content and layout, and to easily maintain those tests. Dynamic Region Labelling allows to label any regions to ignore (useful where animated gifs are running, or where dynamic data is placed), or define floating regions on a page; these capabilities allow CloudShare to ignore dynamic regions on any page, and build robust tests that effectively compare pages with dynamic content and layout, without failing due to insignificant pixel variations.

Another capability that allowed CloudShare to further optimize its testing efforts is change propagation (i.e. perceiving the app as a whole, and not as random unrelated pages): when a new change affects multiple app pages in a similar fashion (e.g. change in header or navigation bar), CloudShare only needs to review and accept this change for a single occurrence, and Applitools Eyes automatically propagates and approves similar changes in related pages.

Results
  • Over 50% reduction of UI automation code
  • Improvement of overall number of bugs caught pre-release
  • Greater automated UI coverage
  • Easier, shorter maintenance
  • Peace of mind with each new release and update, knowing the all of the app's visual elements across multiple browsers and various resolutions – are validated and regression-free

Following the addition of Applitools Eyes Automated Visual Testing Solution to CloudShare's Test Automation infrastructure, the majority of UI validation is now handled using screenshot comparison. This substantially reduced the amount of automation code, while increasing browser & resolution coverage, overall test reliability and significantly lowering time spent on code maintenance.

Summary

With Applitools Eyes, CloudShare managed to save quite a bit of time and effort on writing and maintaining automation code, and the implementation of Automated Visual Testing has allowed them to achieve greater automated UI coverage. Applitools Eyes helps CloudShare to quickly and automatically detect many UI appearance & layout issues, and functional issues – and address them pre-release. As a result, CloudShare reduced the overall number of missed bugs and substantially improved product quality and user experience.