Affirm

Highlights

Who is Affirm?

Affirm is a financial services company that specializes in the buy now, pay later space. Their goal is to make it easier for people to access the items they need with an honest financial product with no fees or surprises. With over 17 million purchases and counting Affirm was growing quickly, and they knew they needed to focus time and energy on increasing testing resilience so that bugs didn’t slip into production.

Rapid Growth and the Challenge of Scaling Up

Affirm is growing rapidly, and existing methods of testing were not catching the bugs they needed. “We always knew that we needed better visual regression testing,” said Peter Tan, Software Engineer on the Web Platform Team at Affirm. Not only were they growing fast and in need of a process to appropriately vet an increasing number of highly-visited pages, but when the team reviewed specific “post-mortems” on bugs that escaped they noticed that most were visual in nature and not functional.

All these visual bugs, they realized, would have been caught with better visual regression testing.

They had an existing solution capable of comparing pixel diffs, but the workflow was challenging to use and it was difficult to adopt widely. Results were also poor when it came to analyzing pages with dynamic content. Peter and his team knew they needed an automated solution that was capable of scaling up with them and which was also easy to use.

The Immediate Results of Automated Visual Testing

After incorporating Applitools Eyes into their testing workflow, two critical visual bugs on a page with a crucial partner were caught almost immediately. These pages had otherwise passed their standard suite of functional tests and would have been deployed to millions of visitors. Since then the team has continued to expand the use of Applitools to ensure visual consistency across an increasing number of pages.

“Customers are used to things working a certain way,” noted Peter. “Visual consistency definitely has an effect on our brand and our reputation.” 

Dynamic Pages, AI-Powered Match Levels and Ease of Use

When comparing Applitools to other solutions, a few additional important features stood out to the Affirm team.

A crucial differentiator was the AI-powered match levels thresholds. “Other solutions only had a pixel-by-pixel comparison, and a lot of our dynamic pages would fail all the time and result in false-positives that waste people’s time,” Peter said. With Applitools, dynamic content can be labeled and only the areas that need to be tested for visual regressions will be analyzed.

Ease of use was also a critical factor. Their previous visual testing solution was not only less reliable but more difficult to use, to the point where team members simply weren’t using it. The effectiveness of the match level thresholds along with the speed and ease of use of Applitools led to rapid internal adoption and a lot of time saved. “Now we’re spending less time fixing mistakes and our developers have more time to put out higher quality deployments,” Peter noted.

Peter and the team also appreciated the new time-saving integration with GitHub Actions, and the fact that Applitools is compatible with WCAG (WCAG) standards for accessibility.

What’s Next for Affirm?

Affirm is growing and expanding in many ways. One crucial direction they are expanding in is into new geographies, which means being able to perform robust localization testing – including for visual consistency – is key. As the company grows they are also seeking to increase the adoption of Applitools across more teams and into development, and to eventually integrate it as a standard best practice, so that they can continue to deliver the highest quality experience to their customers.