Dealertrack Moves From Zero UI Automation to Expansive Test Coverage with Applitools
There is no other tool that gives us this peace of mind. And as a QA, peace of mind is my job.
Jonathan Gray — DealertrackQA Engineer
Dealertrack
Highlights
Dealertrack, an all-in-one platform for automotive dealers, lenders and creditors, was looking for a visual UI testing tool that would reduce manual testing time and increase automated UI coverage.
They tested other visual UI testing tools, but all fell short except for Applitools. In a few short months, Dealertrack has gone from zero automated UI verifications to being on their way to implementing full test coverage of critical workflows.
As Dealertrack moves towards Continuous Delivery in 2016, Applitools will play an essential role in providing UI test automation, as well as empowering product owners and scrum masters to verify the user experience in real time.
The Challenge
Dealertrack is the leading provider of software, marketing and e-commerce for automotive dealers. Founded in 2001, its on-demand platform links buyers to car retailers, manufacturers and lenders online, while seamlessly handling the credit checks and transactions for buying a new car online. Dealertrack was recently acquired by Cox Automotive in a $4 billion dollar deal, the largest in Cox Enterprises’ history.
When a potential buyer lands on a dealer’s site and starts looking around at various cars, it’s likely that he or she is actually landing on the Dealertrack platform, since they work with more than 80% of all franchised dealers. Given the volume and the nature of the business — with users passing their sensitive credit card information through the platform for credit checks; a stellar end-user experience is incredibly important to the success of Dealertrack.
To ensure quality on the site, the Quality Assurance team was spending 10 hours every sprint manually testing their various user workflows. “We had Selenium automation for the functional testing of the forms, but there was a huge gap for automated UI testing,” says Jonathan Gray, a QA Automation Engineer at Dealertrack. “With manual testing, after a while, you’re going to start to skip over UI issues and get tunnel vision.”
Dealertrack needed to reduce the amount of time spent manually testing while increasing their UI coverage and they knew automation was the way to go. After meeting Applitools at the 2015 Selenium Conference in Portland, they got to work integrating it into the Dealertrack testing process.
The Solution
Jonathan says he and the team had investigated automated UI tools in the past, but they all fell short except for Applitools. It quickly became clear that Applitools had everything they were looking for, from image aliasing to capturing full screenshots, not to mention leveraging Selenium behind the scenes.
There are about 20 team members responsible for UI performance at Dealertrack, from engineers to product owners to scrum masters. Initially, Jonathan was the only person using Applitools as he piloted the tool by testing some of their existing workflows. Within days, he felt a difference in having Applitools integrated in their process. “Every morning I check the previous regression runs. It only takes 10 minutes now and it’s great peace of mind,” Jonathan says.
Now he’s expanding Applitools within his team and getting others on board. “We don’t have very many automation engineers to write the workflow code, so if we can get our black box testers to manage this once we get a test case going, we’ll free up our test engineers,” Jonathan says. “The test engineers will manage the test results through the Applitools website, and then we’ll focus more on the code.” It’s this change in process, he says, that’s “really helped us sell this to other people within the organization.”
The Results
In just a few short months since integrating Applitools, manual testing time at Dealertrack has been cut in half. They’ve gone from zero automated UI testing to being on their way to full test coverage of their critical user workflows that directly impact company revenue. The team is only anticipating more time savings and more developer productivity as more engineers get up and running with it.
Jonathan is also excited about the impact this will have on the product and scrum teams, now that they have access to the exact UI experience their users are seeing through the Applitools user dashboard. “Before Applitools, at the end of every sprint, we had a demo to go over the features that the team completed,” Jonathan says. “That would be the only time the product owner got to see the UI and plenty of times we didn’t show the full application.” Now, with each new build, everyone involved in the sprint has access to how the UI is doing and how users are experiencing the app.
Things have been exciting for Dealertrack. Not only did they get acquired by the largest automotive ecosystem in the world, but they evolved into a Continuous Deployment delivery system. Applitools, Jonathan says, played an important role in their CD process. Applitools helped them cut their regression testing from days to hours. The freed time allowed Dealertrack to deliver features and improvements that everyone, from the product managers to end users, wanted to see.
And for the QA team, Applitools continues to provide that extra layer of security in catching bugs and UI issues as the tool is rolled out to more engineers and team. Says Jonathan, “There is no other tool that gives us this peace of mind. And as a QA, peace of mind is my job.”