Manually testing took days – actual days – of mind-numbing work. Now it can be done in 10-20 minutes. Using Applitools has given us the wherewithal to make other parts of our platform better.
Kenji Crosland — SincroSoftware Engineer
Sincro, a global leader in digital marketing solutions for car dealerships, manages thousands of websites. Ensuring quality across so many sites and dynamically generated pages was incredibly challenging and time-consuming for the Sincro team.
Using Applitools, validation of these pages could be done in a fraction of the time it took using manual or alternative solutions.
With Applitools in place, developers are now free to focus on adding additional value and improving the Sincro platform rather than manually triaging bugs.
Sincro is a leading provider of digital marketing solutions to car dealerships across the globe. They manage and provide services for websites, advertising, and many other intelligent technologies that help car dealers reach new users more effectively and decrease costs. Sincro works with most of the major OEM car manufacturers and thousands of local dealerships worldwide.
For Sincro, it’s critical that their digital services are easy to use and of the highest quality.
Providing top-notch websites for thousands of individual dealerships and their manufacturers, across numerous brands and regions that each have their own distinct identity, is no small feat. Sincro is committed to delivering amazing websites for each and every partner. To do this, they provide a platform that makes it easy for individual site owners to choose from a large number of carefully designed customizable content blocks aligned to each parent brand’s specifications. This allows them to change the look and feel of their sites to stay on brand and adapt for seasonal changes or promotions.
However, a challenge with this approach is validation. Changes to parent designs have a cascading effect across numerous child designs – a single design issue can affect thousands of websites.
It’s crucial that no design issues slip through with so many websites relying on each parent design. Yet validating across so many brands and designs was time-consuming and difficult.
Sincro releases updates every week, and initially developers were tasked with manually validating a large number of pages across 5 or more browsers. This tedious process required time from 5 developers who would methodically validate step by step against a checklist, repeating for each browser that was supported.
“Manual validation of a feature across several browsers with 50 plus designs took days – actual days,” said Kenji Crosland, a software engineer at Sincro. “I was spending 3 hours every Friday on a single representative site per brand and it was mind-numbing work.” After doing this for a while, Kenji knew he had to find a better way. “I eventually decided to invest some of the time I had been spending manually validating towards finding an automated solution. I figured it would be a good gamble.”
Kenji and his team initially assembled a hybrid solution to address this. They combined automated test scripts with a legacy cloud-based platform for cross-browser execution alongside Applitools for their automated visual tests. While this was faster than manual testing, there were concerns around the quality coming from the cloud-based, cross-browser portion of the solution. “The biggest issue for me was the test fidelity,” noted Kenji, who worried about “extremely costly” regressions that he’s sure were missed.
Despite being faster than pure manual testing it also created a new bottleneck. Anytime more than 50 tests were queued up, it overwhelmed the 50 VMs they had access to in the legacy solution which resulted in time-outs, test re-runs and frustration. Mobile emulation was also a struggle. Despite having access to mobile VM’s through the legacy provider, the reality was that the VMs were slow to launch and prone to failure. Each mobile VM would take nearly 2 minutes to spin-up only to result in a 90-95% timeout rate.
Given these struggles with the legacy solution, Kenji kept looking for an alternative cross-browser, automated testing platform – until he realized he already had one in Applitools. After switching to the Applitools Ultrafast Test Cloud for cross-browser emulation, Kenji quickly found the time he was spending on each suite of tests dropped dramatically – from 15 minutes each to 3 minutes. “The Ultrafast Test Cloud was a breath of fresh air,” added Kenji, noting that his team members no longer had to meticulously schedule their turns and wait in line to run their web and mobile tests.
Even more importantly, the test fidelity was vastly improved thanks to the accuracy of Visual AI. This meant his team could rely on the results and spend less time manually reviewing issues.
“My favorite feature is the AI that removes differences caused by element displacement,” said Kenji, “It gets rid of a ton of false diffs so that we can actually focus on real ones – it cuts the time we spend on triaging bugs in half.” The number of pages to check for a single car brand can easily reach into the hundreds, and checking all these with Applitools now takes 10-20 minutes rather than 3-4 hours.
What’s the end result of this increased productivity? More time to spend on high priority issues. To top it all off, the cost to Sincro is 40% less than they were spending on their previous solution. That means they are now spending less money for faster and higher quality testing results.
“Using Applitools has given us the wherewithal and time to make other parts of our platform better,” noted Kenji. “Test quality is exceptional now. We’re very happy.”
Kenji said he plans to think about how he can move his team towards more prevalidation and apply visual testing earlier in the process. He’d also like to begin testing individual components or groups of components as a representative page, to further streamline the review process.
Most of all, Kenji shared that he was excited that the pain of manual testing and even automated functional testing with the legacy solution is behind him. “Testing is stable now, and that’s a good thing,” he added, and developers are free to focus on adding additional value.