Selenium IDE is back

16 reasons why to use Selenium IDE in 2019 (and 2 why not) 🤔

Have you tried using the new Selenium IDE for your QA test automation?

People seem to like it…

Still skeptical? I get it.

There’s been plenty of stigma around using record and replay tools like Selenium IDE rather than scripted QA automation tools like Selenium Webdriver, Cypress, and WebdriverIO.

And rightfully so.

Traditionally, record and playback tools suffer from a litany of issues, including:


Also published on Medium.

  1. No cross-browser support
  2. Brittle tests
  3. No easy way to wait for app under test
  4. No conditional logic
  5. No way for one test script to call another
  6. No way to embed code into recorded scripts
  7. No way to edit scripts once recorded
  8. No script debugger
  9. No way to run scripts in parallel
  10. No way to run tests from Continuous Integration build scripts
  11. No way to integrate with source code control systems
  12. No plugins to extend functionality
  13. No way to do visual UI testing
  14. Poor support for responsive web
  15. No way to quickly diagnose front-end bugs
  16. No way to export tests to languages like Java
  17. No way to have data-driven tests

On top of all that, Selenium IDE went end-of-life a couple of years ago. (Minor detail.)

Even so, you should seriously consider using Selenium IDE in 2019 for testing web applications(No, I’m not crazy!)

Here’s a list of 16 reasons why — and two why not. Read them, and let me know what you think.

First a bit of background. I’m not some kind of record-and-replay cheerleader. When I started a cloud testing company (Sauce Labs), I made sure we focused first on Selenium Webdriver, not Selenium IDE. Before that, I got my teeth kicked in by customers (thankfully, only figuratively) as the product manager for Mercury QuickTest Pro, now called Micro Focus UFT.

Those lessons were reinforced when I did research for this post. There’s no shortage of articles on why Selenium IDE is bad. There was this Quora thread comparing Selenium IDE with Selenium Webdriver. And plenty of issues listed in the Selenium IDE questions on Stackoverflow. Plus this top 10 list of issues with record & replay.

Even so, I really think that the new Selenium IDE fixes most (but not yet all) of its traditional problems. Let’s dive in.

#1: Selenium IDE is no longer dead

Selenium IDE is baaaccck!
Credit: Joe Colantonio

Back in 2017, Selenium IDE development stopped. Firefox 55 broke the integration point that Selenium IDE depended on to record and replay tests.

Shortly after, my colleagues here at Applitools approached the Selenium open source community to see how we could help. As a company, our growth has benefitted from this vibrant community, and we wanted to give back.

So, we helped revamp much of the Selenium IDE code, which is now freely available on GitHub under an Apache 2.0 license, managed by the Selenium community, and supported by two full-time engineers, one of whom literally wrote the book on Selenium testing.

Selenium IDE committers @corveo and @tourdedave

As a result, Selenium IDE is back. Don’t believe me? Check out this chart on Selenium IDE usage on Chrome — it’s growing steadily.

pasted image 0 4

#2: Selenium IDE is now cross-browser

Selenium IDE first came out in 2006.

It was a different time. iPhones didn’t exist, the Motorola Razr flip phone was the must-have device, and Borat topped the movie box office. Firefox was the shiny new browser, and Chrome wouldn’t come out for two more years.

So it’s no surprise that Selenium IDE hitched its wagon to Firefox. Unfortunately, it remained that way for over a decade, frustrating the heck out of users with its single-browser support.

No more.

Selenium IDE is now available as a Google Chrome Extension

Selenium IDE for Google Chrome

….and Firefox Add-on:

Selenium IDE for Firefox

Even better, Selenium IDE can run its tests on Selenium WebDriver servers. You can do this using Selenium IDE’s command line test runner, called SIDE Runner.

You can think of SIDE Runner as blending elements of Selenium IDE and Selenium Webdriver. It takes a Selenium IDE script, saved as a .side file, and runs that using browser drivers should as ChromeDriver, EdgeDriver, Firefox’s geckodriver, IEDriver, and SafariDriver.

Selenium IDE does cross-browser testing on Chrome, Firefox, Edge, Safari, and IE

SIDE Runner, and the drivers above, are available as a straightforward npm installs. Here’s what it looks like in action:

pasted image 0 6

#3 Selenium IDE tests are no longer brittle

For years, brittle tests have been an issue for functional tests — whether you record them or code them by hand. A huge contributor to this problem has been object locators. These are how your QA automation tool identifies which field to fill, or which button to click. These can be a button label, an XPath expression, or something else.

Developers are constantly sadistically tormenting QA teams releasing new features, and as a result, their UI code is constantly changing as well. When UI changes, object locators often do as well.

Selenium IDE now fixes that by capturing multiple object locators when you record your script. During playback, if Selenium IDE can’t find one locator, it tries each of the other locators until it finds one that works. Your test will fail only if none of the locators work.

This doesn’t guarantee scripts will always playback but it does insulate scripts against many changes. Here’s a screenshot of how it works. As you can see, Selenium IDE captures linkText, an XPath expression, and CSS-based locators.

pasted image 0 31

Imagine building this functionality in Selenium Webdriver. You’d have to first gather up all potential Xpath locators, then all CSS locators, then iterate through each until you find an object match. It’d be a huge chunk of time to automate just one interaction, and you’d be left with a mess of hard-to-maintain code.

Selenium IDE provides an alternative that is fast, resilient, and easy-to-maintain.

#4 Selenium IDE is now better at waiting for your app

When running tests, it’s essential to give your application time to catch up to your test automation tool. This can include time for backend operations, fetching page elements, and rendering the page. It’s especially necessary when running on staging servers that are under-resourced.

Why does waiting matter? If your test script tries to interact with some page element (field, button, etc.) that hasn’t loaded, it will stop running.

Thankfully, the new Selenium IDE nows automatically wait for your page to load. Also, commands that interact with some element now wait for the element to appear on the page. This should eliminate most, if not all, of your explicit waits.

But, if that’s not enough, the new Selenium IDE gives you other options.

In the new Selenium IDE there’s a global set speed command that you can use to pause after every test step. Even better, you can set this from the toolbar in the new Selenium IDE. Check it out below.

Between automatic waits and global set speed, you should have a lot fewer pause commands. That means your tests will be simpler and easier to maintain.

pasted image 0

If you need more fine-grained control, Selenium IDE lets you insert steps to wait for an element to meet some condition: editable, present, or visible — or the opposite (not editable, not present, or not visible).

Finally, there’s the pause command that you can insert after individual steps. Selenium IDE has had this for a long time; feel free to use if you’re feeling nostalgic.

#5 Selenium IDE now has conditional logic

When testing web applications, your scripts have to handle intermittent user interface elements that can randomly appear in your app. These are those oh-so-helpful cookie notices, as well as popups for special offers, quote requests, newsletter subscriptions, paywall notifications, and adblocker requests.

Cookie notice
You use cookies? Who knew!

Conditional logic is a great way to handle these intermittent UI annoyances features. You want your scripts to say, If X appears, click the link to make it go away.

You can now easily insert conditional logic — also called control flow —  into your Selenium IDE scripts. Here are details, and how it looks:

pasted image 0 32

#6 Selenium IDE now supports modular test scripts

Just like application code, test scripts need to be modular. Why?

Many of your test scripts will have steps to sign into your app, sign up for an account, and sign out of an app. It’s a waste of time to re-create those test steps over and over.

pasted image 0 26

Selenium IDE now lets one script run another. Let’s say you have a login script that all your other scripts call. You can easily insert this step into Selenium IDE. Here’s how it looks:

pasted image 0 21

This way, if your sign in, sign up, or sign out functionality changes, you only have one test script to change. That makes test maintenance a lot easier.

Here’s a quick demo of this in action:

#7 Selenium IDE now supports embedded code

As broad as the Selenium IDE API is, it doesn’t do everything. For this reason, Selenium IDE has execute script and execute async script commands that lets your script call a JavaScript snippet.

This provides you with a tremendous amount of flexibility by being able to take advantage of the flexibility of JavaScript and wide range of JavaScript libraries.

To use it, click on the test step where you want JavaScript to run, choose Insert new command, and type execute script or execute async script in the command field, as shown below:

pasted image 0 8

#8 Selenium IDE scripts now can be edited

In the old Selenium IDE, scripts couldn’t be edited. For this reason, Selenium IDE tests were considered throwaway scripts: if they didn’t work, you’d have to delete them and re-record a test.

With the new Selenium IDE, you can easily modify your tests. Insert, modify, and delete commands, as you can see below. No more throwaway scripts!

pasted image 0 9

#9 Selenium IDE now has a debugger

Pretty much every IDE on the market has combined an editor and a debugger. (That is, after all, what’s meant by Integrated Development Environment.)

But not the old Selenium IDE. It had no debugger. (Whoops.)

The new Selenium IDE lives up to its name, and provides a way for you to set breakpoints in your script. Just click on the left margin of your test.

This way, you can inspect your browser’s state when your script stops due to a breakpoint. Here’s how it looks:

pasted image 0 20

This makes it a lot easier to troubleshoot issues. (Speaking of troubleshooting, check out #16 below.)

#10 Selenium IDE now can run scripts in parallel

With the old Selenium IDE tests could only be run one at a time. This made tests take much longer. Alternatives like Selenium Grid were only available when used with Selenium WebDriver.

Selenium IDE can now run tests in parallel. This lets you get through your test suites much faster.

To run multiple SIDE Runner tests in parallel, just tell it the number of parallel processes you want. Here’s an example of running three tests at once:

No, that’s not a Bandersnatch reference…

Here’s a quick video of this in action (view in full screen since the fonts are small):

#11 Selenium IDE now can be run from CI build scripts

Because SIDE Runner is called from the command line, you can easily fit into your continuous integration build scripts, so long as your CI server can call selenium-ide-runner and upload the .side file (your test script) as a build artifact. For example, here’s how to upload an input file in Jenkins, Travis, and CircleCI.

pasted image 0 28

This means that Selenium IDE can be better integrated into your DevOps toolchain. The scripts created by your less-technical QA team members — including business analysts — can now be run with every build. This helps align QA with the rest of the business and ensures that you have fewer bugs escaped into production.

#12 Selenium IDE scripts can be managed in a code repository

Other record and replay tools store their tests in a range of binary file formats. (For example, here are UFT’s binary file formats.) You could check these into a source code repo, such as GitHub or GitLab, but it wouldn’t be all that useful since you couldn’t inspect test scripts, compare differences, or selectively pull in changes.

In contrast, the new Selenium IDE stores test scripts as JSON files. This makes them easy to inspect, diff, and modify. Here’s a script I recorded, viewed in Sublime text editor. You can easily change the starting URL, window size, and object locators.

pasted image 0 23

If you manage your Selenium Webdriver scripts in GitHub, GitLab, Bitbucket, Microsoft Visual Studio Team Services (now called Azure DevOps), AWS CodeCommit, Google Cloud Source, or some other source code repo, you can now do the same for your Selenium IDE scripts.

pasted image 0 30

#13 Selenium IDE is now extensible with plugins

Unlike the old Selenium IDE, the new Selenium IDE now supports third-party plugins to extend its functionality. Here’s how to build your own Selenium IDE plugin.

This is pretty exciting. You can imagine companies building plugins to have Selenium IDE do all kinds of things — upload scripts to a functional testing cloud, a load testing cloud, or a production application monitoring service like New Relic Synthetics.

Plenty of companies have integrated Selenium Webdriver into their offerings. I bet the same will happen with Selenium IDE as well.

Speaking of new plugins…

#14 Selenium IDE now can do visual UI testing

We here at Applitools have built a Selenium IDE plugin to do AI-powered visual validations on Selenium IDE, called Applitools for Selenium IDE. (Imaginative, right?)

To get it, head to the Chrome and Firefox stores, do the three-second install, plugin your Applitools API key, and you’re ready to go.

Create a Selenium IDE script, choose Insert new command, type eyes (that’s the name of our product), and insert a visual checkpoint into your test script. Like this:

pasted image 0 13

Visual checkpoints are a great way to ensure that your UI renders correctly. Rather than a bunch of assert statements on all your UI elements — which would be a pain to maintain — one visual checkpoint checks all your page elements.

Best of all, Applitools uses visual AI to look at your web app the same way a human does, ignoring minor differences. This means fewer fake bugs to frustrate you and your developers — a problem that often leads simple pixel comparison tools to fail. When Applitools finds a visual bug, it’s worth paying attention to.

Here’s an example of Applitools Visual AI in action, finding a missing logo on a GitHub page. We didn’t have to create an assert statement on the logo; Applitools visual AI figured this out on its own.

pasted image 0 15

#15 Selenium IDE can now visually test responsive web apps

When you’re testing the visual layout of your responsive web apps, it’s a good idea to do it on a wide range screen sizes (also called viewports) to ensure that nothing appears out of whack. It’s all too easy for responsive web bug to creep in.

And when they do, the results can range of merely cosmetic to business-stopping. Here’s Southwest Airlines putting the kibosh on their checkout process with a responsive bug that covers up the Continue button:

Visual bug on Southwest.com
Not good, right?

When you use Applitools for Selenium IDE, you can visually test your webpages on Applitools Ultrafast Grid. This cloud-based testing service has over 100 combinations of browsers, emulated devices, and viewport sizes. This lets you do thorough visual testing on all your web apps.

Here’s how you specify which combinations to test on:

pasted image 0 17

Once your tests run on Ultrafast Grid, you can easily check your test results on all the various combinations, like this:

Applitools Ultrafast Grid
Your responsive web bugs can run but they cannot hide…

#16 Selenium IDE now help pinpoint the cause of front-end bugs

Every Selenium IDE script you run with Ultrafast Grid can be analyzed with our Root Cause Analysis.

This matters because, to bastardize Jerry Seinfeld, it’s not enough to FIND a bug. You have to FIX the bug.

Like the Seinfeld car rental company, every testing tool I know of finds bugs, but doesn’t tell you how to fix them.

Except Applitools.

When you find a visual bug in Applitools, click on it, and view the relevant DOM and CSS diffs, as shown below:

pasted image 0 1

I want to point out that we don’t show all DOM and CSS diffs — just the handful that are likely to have caused a visual bug. This makes debugging visual bugs go much faster.

We covered a ton of different ways Selenium IDE and Applitools work together. Here’s a visual summary:

pasted image 0 24

Back to Selenium IDE itself.

Even with all the things the new Selenium IDE can do, there are still things that it doesn’t yet do. Here are two reasons why not to use Selenium IDE:

#1 Selenium IDE doesn’t export Webdriver scripts (yet)

You can’t yet export Selenium IDE tests as Selenium Webdriver scripts. The old Selenium IDE did this, and it’s being worked on actively. You can track progress here.

Update: Selenium IDE now has code export. As of this writing, it’s only for Java JUnit, but it’s a start.

#2 Selenium IDE doesn’t support data-driven scripts (yet)

You can’t yet use the new Selenium IDE to import a bunch of tabular data, like a CSV file or database table, and then run a parameterized test once for each row of data. This, too, is being worked on, and you can track progress here.

Summary

Here’s how Selenium IDE compares to traditional record & replay:

Capability Traditional Record & Replay New Selenium IDE
Cross-browser support No Yes
Resilient tests No Yes
Automatically wait for app under test No Yes
Conditional logic No Yes
Run one test from another No Yes
Embed code into scripts No Yes
Edit scripts No Yes
Debug scripts No Yes
Run scripts in parallel No Yes
Run scripts during CI builds No Yes
Manage scripts in source code repo No Yes
Plugins to extend functionality No Yes
Visual UI testing No Yes (w/ plugin)
Responsive web support No Yes (w/ plugin)
Diagnose root cause of visual bugs No Yes (w/ plugin)
Export tests to code No Not yet
Data-driven tests No Not yet

 

‘Less is more

To me, Selenium IDE is part of a larger trend of software making life simpler for technical folks. One example: the broad range of codeless tools for developing applications.

Other examples: Serverless offerings like AWS Lambda make it easier to write just the code you need to get a job done. And Schemaless databases like MongoDB provide architects with much more flexibility to innovate versus tightly constricted SQL databases.

Codeless, serverless, schemaless — and now scriptless, with Selenium IDE. We might be seeing a trend here.

Go deeper

If I were to pick one place for you to get started, I’d recommend this tutorial on Selenium IDE. It’s done by my colleague Raja Rao, a former QA manager who’s been using Selenium Webdriver for over a decade. So he knows what he’s talking about.

Beyond that, here’s a fairly complete list of resources to learn the new Selenium IDE in 2019:

Selenium IDE pages

Applitools for Selenium IDE pages

Selenium IDE videos

How do you plan on using Selenium IDE? Let us know!

 

Want to become a test engineering rockstar? What about money and prizes? Sign up for our Applitools Visual AI Rockstar Hackathon. Click here to learn more.

 


Also published on Medium.

Al Sargent profile photo

Al Sargent

VP Product Marketing

Al Sargent on Twitter Al Sargent on LinkedIn

About the author

Al Sargent started coding at age 10, wrote his first computer game at 12, and by 13, got sent to the principal’s office for skipping class to code. Decades later, he’s now Vice President of Product Marketing at Applitools. He loves the process of crafting stories about technology: understanding customers and users, what they care about, and how they use software in their jobs and lives. Prior to Applitools, Al helped create the world’s first software-testing cloud, Sauce Labs; drive the fastest-growing business unit at VMware; advance market-changing open source technologies such as Spring and Cloud Foundry; and build a new software category — Software Analytics — at New Relic.

Applitools lets you deliver visually perfect web and mobile apps with AI-powered end-to-end visual testing & monitoring.

Stay updated by signing up for our newsletter