I find so much value using Applitools in this process. As the applications get built, I have automation to capture the construction of each page. I know when page behaviors match my expectations, and I can immediately uncover unexpected differences. I cannot imagine clicking through all the pages and modals to uncover behavior repeatedly any other way.
Marianne Epstein — EVERFIDirector of UX Design
EVERFI, a leader in e-learning technology is investing in a new common product platform, but quickly realized they’d need to re-think their approach to quality to ensure an optimal user experience.
With 45 engineers and the newly formed UX team now using Applitools, the team is able to catch bugs much earlier in development saving valuable time & money.
Moving forward, EVERFI’s improved ‘test early’ workflow and newly formed UX team ensures the organization is well positioned to continue on its rapid growth trajectory helping to redefine education.
EVERFI is leading education’s evolution and redefining the way learners approach the world. The company empowers teachers, citizens, and students to improve their lives and make an impact in their communities. Their e-learning solutions activate today’s most sought-after skills in dynamic, incentive-driven online environments. EVERFI’s critical skills courses are catalyzing comprehension, retention, and behavior change while fulfilling the need for scalable education in an ever-changing world.
To succeed, EVERFI combines a deep knowledge of educational curricula, proven ability to induce knowledge and behavior change through online courseware, and the ability to develop web-based training products that meet the needs of corporate, professional, community, and K-12 educators. EVERFI specializes in evidence-based online learning products for a range of core topics. These topics include financial literacy, social and emotional learning, STEM and career readiness, conduct and culture, and health and wellness.
EVERFI has developed a range of products since its founding in 2008. Each product has its own software development history, customizability, and state of quality. Over time, EVERFI began to standardize courseware development, but courses retained their own unique code that required maintenance and updates. Top management realized that this collection of individual courses blocked their ability to scale. In response, EVERFI invested in creating a new common product platform to help standardize course development and maintenance.
In developing this platform, EVERFI leaders realized that they would have to rethink their approach to software quality. In their legacy course-by-course development process, they tested with a combination of automated and manual end-to-end tests. By moving to a common platform, EVERFI needed automated tests throughout development to validate behavior and uncover bugs before they rippled through the entire platform.
To guide their approach to quality, EVERFI hired Greg Sypolt as Vice President of Quality Assurance. Greg brought extensive experience in software quality from his prior work. In addition, Greg wrote extensively on software quality and quality maturity. EVERFI realized that Greg could bring the wealth of quality experience needed to drive this platform development process to a successful outcome.
The Challenge of Scaling
Greg saw the scale of his quality challenge immediately and realized that a single development change could ripple across a range of courses with unintended consequences, affecting tens of thousands of web pages. Manual testing would never scale in this situation. So, Greg brought in Applitools. From his experience, Greg knew that Applitools would help capture visual behavior of the finished product, but also the behavior of code at check-in.
Powered by Visual AI, Applitools allows EVERFI to compare rendered pages quickly and efficiently. By abstracting pixels into visual elements, Applitools identifies visually distinct changes in layout, color, and content elements, while ignoring pixel-based rendering differences that plague non-AI based visual test systems. Now, with the ability to focus comparisons on areas of expected difference and expected similarity, Applitools provides huge benefits during application development in exposing unintended changes for both new behavior and untouched code.
With Greg’s direction, the courseware platform team implemented a model-based testing methodology using TestModeller.io combined with Cypress and Applitools. Instead of waiting for finished code, the developers validate their code check-ins. Applitools ensures that new behaviors get captured and validated across web and mobile web viewports, but also that existing elements that should remain untouched have no unexpected changes.
Designing a Consistently Perfect Experience
It’s not only the EVERFI engineering team benefiting from Applitools, but also their newly formed UX team.
To aid in developing a common platform, EVERFI invested in user experience, recently onboarding Marianne Epstein as Director of UX design. Marianne is tasked with a mission to collaborate with engineering to deliver an unparalleled user experience. Early on, it became obvious that the biggest challenge was the opaqueness of the platform – not knowing what users are seeing because of all the ways and places screens can be dynamically generated. “How can we improve the user experience, if we don’t actually know exactly what our users are seeing?” said Epstein. “The number of permutations is overwhelming and was impossible for the UX team to navigate. Applitools provides instant access and validation to the massive inventory of screens and the peace of mind knowing that every regression is immediately spotted across all screen variations ensuring the user experience is exactly as it was intended to be. I feel like it is a transformative tool for our UX team and how we partner with development to drive user experience.”
When situations arise and the UI does not match what has been designed in Figma, Marianne’s team is able to pinpoint the screen, and thus know exactly who from engineering they need to collaborate with on the fix. As the UX team continues on their quest towards UI perfection, they are starting to write their own codeless automation using Selenium IDE along with the native Applitools SDK.
The Benefit of Applitools at EVERFI
Currently, 45 engineers use Applitools in their work on the new courseware platform. The platform development has proceeded smoothly, in large part, because the component teams and the platform teams find visual problems at code check-in time. The engineering teams haven’t been surprised with visual errors during their integration tests because they have exposed these issues during development.
As Marianne said, “I find so much value using Applitools in this process. As the applications get built, I have automation to capture the construction of each page. I know when page behaviors match my expectations, and I can immediately uncover unexpected differences. I cannot imagine clicking through all the pages and modals to uncover behavior repeatedly any other way.”
Applitools provides visibility to the broad customizations built into the new platform. EVERFI can test coursework migration from legacy systems and validate behavior before courses go live. The automation infrastructure lets course testing across the platform run in parallel, and Applitools provides all the visual captures to validate the platform user experience.
“Applitools lets us test almost 5,000 unique platform pages with confidence,” Greg said. “We count on Applitools to help capture the hundreds of thousands of combinations that will exist across all the courses once all the courses get migrated.”
With their new Visual AI powered framework in place, improved ‘test early’ workflow, newly formed UX team, and hiring on overdrive, EVERFI’s engineering team is well positioned to continue on its rapid growth trajectory helping to redefine education.