Case study

Dealertrack: the leading provider of digital solutions to the automotive retail industry

Dealertrack Moves From Zero UI Automation to Expansive Test Coverage with Applitools

Highlight
  • Dealertrack, an all-in-one platform for automotive dealers, lenders and creditors, was looking for a visual testing tool that would reduce manual testing time and increase automated UI coverage.
  • They tested other visual testing tools, but all fell short except for Applitools. In a few short months, Dealertrack has gone from zero automated UI verifications to being on their way to implementing full test coverage of critical workflows.
  • As Dealertrack moves towards Continuous Delivery in 2016, Applitools will play an essential role in providing UI test automation, as well as empowering product owners and scrum masters to verify the user experience in real time.
"There is no other tool that gives us this peace of mind. And as a QA, peace of mind is my job." — Jonathan Gray, QA Engineer at Dealertrack.
The Challenge

Dealertrack is the leading provider of software, marketing and e-commerce for automotive dealers. Founded in 2001, its on-demand platform links buyers to car retailers, manufacturers and lenders online, while seamlessly handling the credit checks and transactions for buying a new car online. Dealertrack was recently acquired by Cox Automotive in a $4 billion dollar deal, the largest in Cox Enterprises' history.

When a potential buyer lands on a dealer's site and starts looking around at various cars, it's likely that he or she is actually landing on the Dealertrack platform, since they work with more than 80% of all franchised dealers. Given the volume and the nature of the business — with users passing their sensitive credit card information through the platform for credit checks; a stellar end-user experience is incredibly important to the success of Dealertrack.

To ensure quality on the site, the Quality Assurance team was spending 10 hours every sprint manually testing their various user workflows. "We had Selenium automation for the functional testing of the forms, but there was a huge gap for automated UI testing," says Jonathan Gray, a QA Automation Engineer at Dealertrack. "With manual testing, after a while, you're going to start to skip over UI issues and get tunnel vision."

Dealertrack needed to reduce the amount of time spent manually testing while increasing their UI coverage and they knew automation was the way to go. After meeting Applitools at the 2015 Selenium Conference in Portland, they got to work integrating it into the Dealertrack testing process.

The Solution

Jonathan says he and the team had investigated automated UI tools in the past, but they all fell short except for Applitools. It quickly became clear that Applitools had everything they were looking for, from image aliasing to capturing full screenshots, not to mention leveraging Selenium behind the scenes.

There are about 20 team members responsible for UI performance at Dealertrack, from engineers to product owners to scrum masters. Initially, Jonathan was the only person using Applitools as he piloted the tool by testing some of their existing workflows. Within days, he felt a difference in having Applitools integrated in their process. "Every morning I check the previous regression runs. It only takes 10 minutes now and it's great peace of mind," Jonathan says.

Now he's expanding Applitools within his team and getting others on board. "We don't have very many automation engineers to write the workflow code, so if we can get our black box testers to manage this once we get a test case going, we'll free up our test engineers," Jonathan says. "The test engineers will manage the test results through the Applitools website, and then we'll focus more on the code." It's this change in process, he says, that's "really helped us sell this to other people within the organization."

The Results

In just a few short months since integrating Applitools, manual testing time at Dealertrack has been cut in half. They've gone from zero automated UI testing to being on their way to full test coverage of their critical user workflows that directly impact company revenue. The team is only anticipating more time savings and more developer productivity as more engineers get up and running with it.

Jonathan is also excited about the impact this will have on the product and scrum teams, now that they have access to the exact UI experience their users are seeing through the Applitools user dashboard. "Before Applitools, at the end of every sprint, we had a demo to go over the features that the team completed," Jonathan says. "That would be the only time the product owner got to see the UI and plenty of times we didn't show the full application." Now, with each new build, everyone involved in the sprint has access to how the UI is doing and how users are experiencing the app.

This next year will be an exciting one for Dealertrack. Not only did they recently get acquired by the largest automotive ecosystem in the world, they are moving towards Continuous Deployment. Applitools, Jonathan says, will be an important piece of the puzzle in achieving that goal. The team currently spend three days every sprint on regression testing and expects that to be two days soon. By the end of the year, their goal is to reduce this to a matter of hours with their Continuous Deployment process in place. This will free up developers to spend even more time on the features and improvements that everyone, from the product managers to the end user, wants to see.

And for the QA team, Applitools will continue to provide that extra layer of security in catching bugs and UI issues as the tool is rolled out to more engineers and team. Says Jonathan, "There is no other tool that gives us this peace of mind. And as a QA, peace of mind is my job."