We run our test spectrum automatically while we do pull requests. If all the tests pass and our build passes, we can deploy. So from two and a half hours per component to five minutes running for all components. If there are unresolved changes, we need to check them and it only takes about three to four minutes. That really saved us tons of time.
Nina Westenbrink — KPNSoftware Engineer
Telecom provider KPN moved from testing on physical devices to testing in the cloud to increase testing efficiency of its design system.
With Applitools, KPN was able to reduce testing time from two and a half hours for one component to five minutes for all components.
KPN is a telecom and IT provider based in the Netherlands, servicing both private customers and business users of any size organization. KPN was originally founded as a postal service called Statenpost in 1752 and started providing telegram services in the 1900s. Today, KPN is a market leader in the Netherlands, offering a portfolio of services to businesses of all sizes.
When Nina Westenbrink first joined KPN as a Software Engineer, the development team consisted of an agile scrum team of eight people. They had many physical devices to test the mobile responsive website. They were developing on local environments, a development environment, an acceptance environment, and production. As the team would develop, they would test each of these environments on devices. Those iterations of testing with devices took the team a lot of time and didn’t even cover all components.
“We would use about 10 devices, which all took about five minutes per component. So it took about two and a half hours for us per component to test.”
Nina Westenbrink, Software Engineer at KPN
KPN then moved on to virtual systems with emulators, which helped them test digitally. This reduced testing time down to two hours per component, but the team was still using many different environments and not all components were tested.
When Nina joined the design system team at KPN, they used acceptance environments and production environments but only used one branch per environment. The team had tests, but they were mostly tests for rendering the DOM. Visual testing would still be manual, but overall it would take about one hour to test each component.
The purpose of a design system is to have a consistent user experience through all the web touch points for your company. It provides components that act like the building blocks of your app. It also acts as a style guide for the UI. KPN uses Storybook to develop its design system Kermit.
Nina learned about Applitools when a member of Applitools gave a talk about visual testing. The takeaways that you can test multiple browsers and multiple components at one time inspired Nina to start a proof of concept. The proof of concept was to make sure that the team could run the tests locally before doing a pull request. The proof of concept ran through about 600 Storybook stories in four or five minutes.
KPN was able to see tremendous ROI by moving from manually testing components to using AI-powered automated visual testing with Applitools. With the new approach, pull requests were ready to merge if there were no unresolved tests. If there are unexpected changes, the team could then share it back with the teammate who made the changes to address.
“We run our test spectrum automatically while we do pull requests. If all the tests pass and our build passes, we can deploy. So from two and a half hours per component to five minutes running for all components. If there are unresolved changes, we need to check them and it only takes about three to four minutes. That really saved us tons of time.”
Nina Westenbrink, Software Engineer at KPN