This is the story about how CloudShare reduced UI Automation code by over 50% while greatly increasing overall test coverage with Applitools Eyes automated visual testing.
CloudShare uses Production environments to host and run all test environments (for testing both Production and local development labs). This includes machines from most modern Windows operating systems, and with all browsers combinations supported (Chrome, Firefox and Internet Explorer 9+).
CloudShare's automated tests are executed with each local deployment (which happens several times each day), with each Production deployment (every Sunday). In addition, full regression cycles executed nightly and throughout the weekend.
With the previous testing tool used, CloudShare was limited to Internet Explorer, which caused them to miss a large number of bugs that only showed up on other browsers. With Applitools Eyes, this is no longer an issue, as the automated visual testing solution validates the app visual & functional correctness across all browsers, devices, screen resolutions and operating systems. CloudShare now tests in a variety of screen resolutions, which allows them to ensure the UI looks right – across the board.
An important capability CloudShare finds extremely useful is the ability to test pages with dynamic content and layout, and to easily maintain those tests. Dynamic Region Labelling allows to label any regions to ignore (useful where animated gifs are running, or where dynamic data is placed), or define floating regions on a page; these capabilities allow CloudShare to ignore dynamic regions on any page, and build robust tests that effectively compare pages with dynamic content and layout, without failing due to insignificant pixel variations.
Another capability that allowed CloudShare to further optimize its testing efforts is change propagation (i.e. perceiving the app as a whole, and not as random unrelated pages): when a new change affects multiple app pages in a similar fashion (e.g. change in header or navigation bar), CloudShare only needs to review and accept this change for a single occurrence, and Applitools Eyes automatically propagates and approves similar changes in related pages.
Following the addition of Applitools Eyes Automated Visual Testing Solution to CloudShare's Test Automation infrastructure, the majority of UI validation is now handled using screenshot comparison. This substantially reduced the amount of automation code, while increasing browser & resolution coverage, overall test reliability and significantly lowering time spent on code maintenance.
With Applitools Eyes, CloudShare managed to save quite a bit of time and effort on writing and maintaining automation code, and the implementation of Automated Visual Testing has allowed them to achieve greater automated UI coverage. Applitools Eyes helps CloudShare to quickly and automatically detect many UI appearance & layout issues, and functional issues – and address them pre-release. As a result, CloudShare reduced the overall number of missed bugs and substantially improved product quality and user experience.