A data-centric view into the most popular tools for automated testing

Asking “what’s the best test framework?” is a sure way to get into a heated debate. Everyone has their favorite automation tool and wants to let the world know that it’s the best!

Instead of giving you yet another opinion on the matter, I am going to provide the statistics, because after all, numbers don’t lie.

What our data is saying

Many companies are using Applitools for the functional and visual testing of their web and mobile applications. With millions of tests running in our cloud every week, we’ve analyzed how widely used the most-talked about automation testing tools really are.

Most popular web test automation framework

Selenium WebDriver

There’s been a lot of discussion about the future of Selenium WebDriver when there are so many new kids on the block in terms of framework choices. Is Selenium WebDriver losing momentum?

According to our data, only slightly. However, make no mistake, Selenium Webdriver is by far, still the most widely used web automation tool with 75% of our users executing their tests with this framework.

Besides being the tried and true web automation framework for over a decade, Selenium WebDriver also supports all major browsers (Chrome, IE, Edge, Firefox, and Safari) and boasts support for Java, Python, C#, Ruby, JavaScript, and Kotlin whereas most other web automation tools only provide support for a limited number of browsers and a single programming language.

Given this, it’s expected that Selenium WebDriver will generally have more users, as it has a much larger target audience.

So to be fair, let’s look at the JavaScript usage, since most of the newer web automation frameworks are for that demographic.

JavaScript Automation Testing

If we only look at the JavaScript world, we see a much different story.

Cypress

Leading the pack is relative newcomer, Cypress, with just shy of half (47%) of our JavaScript users choosing this tool for their web testing. This is quite impressive, considering up until a couple of months ago (February 2020), Cypress’ browser support was limited to Chrome only.

WebdriverIO

WebdriverIO, which is an independent JavaScript implementation of the WebDriver API, comes in second place with 25% of our JS users choosing this framework for their test automation needs.

WebDriverJS

Selenium WebDriver’s official JavaScript implementation, WebdriverJS, is close behind with 23% of JavaScript tests being executed with this tool.

TestCafe

After the Big Three of JavaScript web automation tools, comes newcomer, TestCafe, with 5%.

Honorable Mention

Playwright

The youngest web automation framework in the race is Playwright by Microsoft. Playwright has only been available for a few months (May 2020) but we’re beginning to see usage. This is definitely one to keep your eye on.

Puppeteer

While Puppeteer is not on our list, we do realize that teams use this framework for their automation needs. However, from our observation, many are utilizing this for automating browsers (i.e. generating PDFs, crawling sites, etc) as opposed to test automation.

So, what’s the most popular test automation tool for web testing?

  1. Selenium WebDriver

    Although technically a web automation framework (not a testing framework), Selenium WebDriver still reigns supreme. With support for a plurality of browsers and languages, WebDriver is by far the most popular choice overall and has seen a 9% increase this year.

  2. Cypress

    Cypress rules the JavaScript web test automation world, and comes in second in overall web automation tools. Yet, it is a far way off from attracting the wide audience that Selenium WebDriver does as a whole. However, its growth is quite impressive, with an 41% increase so far this year.

  3. WebDriverIO
    WebDriverIO is the third most popular web automation tool amongst our users, and saw an increase of 10% this year.

To hear more about the state of web automation tools, their similarities and differences, join the Future of Testing 2020 virtual event on September 17, where a panel of experts from Selenium, Cypress, Playwright, and Applitools will discuss the future of test frameworks.

For More Information

Applitools works with all popular test automation frameworks. We allow you to add accurate visual validation to your existing automated tests with ease. Check out some of these Applitools tutorials.

Applitools lets you deliver visually perfect web and mobile apps with AI-powered end-to-end visual testing & monitoring.

Stay updated by signing up for our newsletter