16 reasons why to use Selenium IDE in 2019 (and 2 why not) 🤔

Have you tried using the new Selenium IDE for your QA test automation?

People seem to like it…

Still skeptical? I get it.

There’s been plenty of stigma around using record and replay tools like Selenium IDE rather than scripted QA automation tools like Selenium Webdriver, Cypress, and WebdriverIO.

And rightfully so.

Traditionally, record and playback tools suffer from a litany of issues, including:

1. No cross-browser support
2. Brittle tests
3. No easy way to wait for app under test
4. No conditional logic
5. No way for one test script to call another
6. No way to embed code into recorded scripts
7. No way to edit scripts once recorded
8. No script debugger
9. No way to run scripts in parallel
10. No way to run tests from Continuous Integration build scripts
11. No way to integrate with source code control systems
12. No plugins to extend functionality
13. No way to do visual UI testing
14. Poor support for responsive web
15. No way to quickly diagnose front-end bugs
16. No way to export tests to languages like Java
17. No way to have data-driven tests

On top of all that, Selenium IDE went end-of-life a couple of years ago. (Minor detail.)

Even so, you should seriously consider using Selenium IDE in 2019 for testing web applications. (No, I’m not crazy!)

Here’s a list of 16 reasons why — and two why not. Read them, and let me know what you think.

First a bit of background. I’m not some kind of record-and-replay cheerleader. When I started a cloud testing company (Sauce Labs), I made sure we focused first on Selenium Webdriver, not Selenium IDE. Before that, I got my teeth kicked in by customers (thankfully, only figuratively) as the product manager for Mercury QuickTest Pro, now called Micro Focus UFT.

Those lessons were reinforced when I did research for this post. There’s no shortage of articles on why Selenium IDE is bad. There was this Quora thread comparing Selenium IDE with Selenium Webdriver. And plenty of issues listed in the Selenium IDE questions on Stackoverflow. Plus this top 10 list of issues with record & replay.

Even so, I really think that the new Selenium IDE fixes most (but not yet all) of its traditional problems. Let’s dive in.

#1: Selenium IDE is no longer dead

Credit: Joe Colantonio

Back in 2017, Selenium IDE development stopped. Firefox 55 broke the integration point that Selenium IDE depended on to record and replay tests.

Shortly after, my colleagues here at Applitools approached the Selenium open source community to see how we could help. As a company, our growth has benefitted from this vibrant community, and we wanted to give back.

So, we helped revamp much of the Selenium IDE code, which is now freely available on GitHub under an Apache 2.0 license, managed by the Selenium community, and supported by two full-time engineers, one of whom literally wrote the book on Selenium testing.

As a result, Selenium IDE is back. Don’t believe me? Check out this chart on Selenium IDE usage on Chrome — it’s growing steadily.

#2: Selenium IDE is now cross-browser

Selenium IDE first came out in 2006.

It was a different time. iPhones didn’t exist, the Motorola Razr flip phone was the must-have device, and Borat topped the movie box office. Firefox was the shiny new browser, and Chrome wouldn’t come out for two more years.

So it’s no surprise that Selenium IDE hitched its wagon to Firefox. Unfortunately, it remained that way for over a decade, frustrating the heck out of users with its single-browser support.

No more.

Selenium IDE is now available as a Google Chrome Extension…

….and Firefox Add-on:

Even better, Selenium IDE can run its tests on Selenium WebDriver servers. You can do this using Selenium IDE’s command-line test runner, called SIDE Runner.

You can think of SIDE Runner as blending elements of Selenium IDE and Selenium Webdriver. It takes a Selenium IDE script, saved as a .side file, and runs that using browser drivers should as ChromeDriver, EdgeDriver, Firefox’s geckodriver, IEDriver, and SafariDriver.

SIDE Runner, and the drivers above, are available as a straightforward npm installs. Here’s what it looks like in action:

#3 Selenium IDE tests are no longer brittle

For years, brittle tests have been an issue for functional tests — whether you record them or code them by hand. A huge contributor to this problem has been object locators. These are how your QA automation tool identifies which field to fill, or which button to click. These can be a button label, an XPath expression, or something else.

Developers are constantly sadistically tormenting QA teams releasing new features, and as a result, their UI code is constantly changing as well. When UI changes, object locators often do as well.

Selenium IDE now fixes that by capturing multiple object locators when you record your script. During playback, if Selenium IDE can’t find one locator, it tries each of the other locators until it finds one that works. Your test will fail only if none of the locators work.

This doesn’t guarantee scripts will always playback but it does insulate scripts against many changes. Here’s a screenshot of how it works. As you can see, Selenium IDE captures linkText, an XPath expression, and CSS-based locators.

Imagine building this functionality in Selenium Webdriver. You’d have to first gather up all potential Xpath locators, then all CSS locators, then iterate through each until you find an object match. It’d be a huge chunk of time to automate just one interaction, and you’d be left with a mess of hard-to-maintain code.

Selenium IDE provides an alternative that is fast, resilient, and easy-to-maintain.

#4 Selenium IDE is now better at waiting for your app

When running tests, it’s essential to give your application time to catch up to your test automation tool. This can include time for backend operations, fetching page elements, and rendering the page. It’s especially necessary when running on staging servers that are under-resourced.

Why does waiting matter? If your test script tries to interact with some page element (field, button, etc.) that hasn’t loaded, it will stop running.

Thankfully, the new Selenium IDE nows automatically wait for your page to load. Also, commands that interact with some element now wait for the element to appear on the page. This should eliminate most, if not all, of your explicit waits.

But, if that’s not enough, the new Selenium IDE gives you other options.

In the new Selenium IDE there’s a global set speed command that you can use to pause after every test step. Even better, you can set this from the toolbar in the new Selenium IDE. Check it out below.

Between automatic waits and global set speed, you should have a lot fewer pause commands. That means your tests will be simpler and easier to maintain.

If you need more fine-grained control, Selenium IDE lets you insert steps to wait for an element to meet some condition: editable, present, or visible — or the opposite (not editable, not present, or not visible).

Finally, there’s the pause command that you can insert after individual steps. Selenium IDE has had this for a long time; feel free to use if you’re feeling nostalgic.

#5 Selenium IDE now has conditional logic

When testing web applications, your scripts have to handle intermittent user interface elements that can randomly appear in your app. These are those oh-so-helpful cookie notices, as well as popups for special offers, quote requests, newsletter subscriptions, paywall notifications, and adblocker requests.

You use cookies? Who knew!

Conditional logic is a great way to handle these intermittent UI annoyances features. You want your scripts to say, If X appears, click the link to make it go away.

You can now easily insert conditional logic — also called control flow — into your Selenium IDE scripts. Here are details, and how it looks:

#6 Selenium IDE now supports modular test scripts

Just like application code, test scripts need to be modular. Why?

Many of your test scripts will have steps to sign in to your app, sign up for an account, and sign out of an app. It’s a waste of time to re-create those test steps over and over.

Selenium IDE now lets one script run another. Let’s say you have a login script that all your other scripts call. You can easily insert this step into Selenium IDE. Here’s how it looks:

This way, if your sign in, sign up, or sign out functionality changes, you only have one test script to change. That makes test maintenance a lot easier.

Here’s a quick demo of this in action:

#7 Selenium IDE now supports embedded code

As broad as the Selenium IDE API is, it doesn’t do everything. For this reason, Selenium IDE has execute script and execute async script commands that lets your script call a JavaScript snippet.

This provides you with a tremendous amount of flexibility by being able to take advantage of the flexibility of JavaScript and wide range of JavaScript libraries.

To use it, click on the test step where you want JavaScript to run, choose Insert new command, and type execute script or execute async script in the command field, as shown below:

#8 Selenium IDE scripts now can be edited

In the old Selenium IDE, scripts couldn’t be edited. For this reason, Selenium IDE tests were considered throwaway scripts: if they didn’t work, you’d have to delete them and re-record a test.

With the new Selenium IDE, you can easily modify your tests. Insert, modify, and delete commands, as you can see below. No more throwaway scripts!

#9 Selenium IDE now has a debugger

Pretty much every IDE on the market has combined an editor and a debugger. (That is, after all, what’s meant by Integrated Development Environment.)

But not the old Selenium IDE. It had no debugger. (Whoops.)

The new Selenium IDE lives up to its name and provides a way for you to set breakpoints in your script. Just click on the left margin of your test.

This way, you can inspect your browser’s state when your script stops due to a breakpoint. Here’s how it looks:

This makes it a lot easier to troubleshoot issues. (Speaking of troubleshooting, check out #16 below.)

#10 Selenium IDE now can run scripts in parallel

With the old Selenium IDE tests could only be run one at a time. This made tests take much longer. Alternatives like Selenium Grid were only available when used with Selenium WebDriver.

Selenium IDE can now run tests in parallel. This lets you get through your test suites much faster.

To run multiple SIDE Runner tests in parallel, just tell it the number of parallel processes you want. Here’s an example of running three tests at once:

No, that’s not a Bandersnatch reference…

Here’s a quick video of this in action (view in full screen since the fonts are small):

#11 Selenium IDE now can be run from CI build scripts

Because SIDE Runner is called from the command line, you can easily fit into your continuous integration build scripts, so long as your CI server can call selenium-ide-runner and upload the .side file (your test script) as a build artifact. For example, here’s how to upload an input file in Jenkins, Travis, and CircleCI.

This means that Selenium IDE can be better integrated into your DevOps toolchain. The scripts created by your less-technical QA team members — including business analysts — can now be run with every build. This helps align QA with the rest of the business and ensures that you have fewer bugs escaped into production.

#12 Selenium IDE scripts can be managed in a code repository

Other record and replay tools store their tests in a range of binary file formats. (For example, here are UFT’s binary file formats.) You could check these into a source code repo, such as GitHub or GitLab, but it wouldn’t be all that useful since you couldn’t inspect test scripts, compare differences, or selectively pull in changes.

In contrast, the new Selenium IDE stores test scripts as JSON files. This makes them easy to inspect, diff, and modify. Here’s a script I recorded, viewed in Sublime text editor. You can easily change the starting URL, window size, and object locators.

If you manage your Selenium Webdriver scripts in GitHub, GitLab, Bitbucket, Microsoft Visual Studio Team Services (now called Azure DevOps), AWS CodeCommit, Google Cloud Source, or some other source code repo, you can now do the same for your Selenium IDE scripts.

#13 Selenium IDE is now extensible with plugins

Unlike the old Selenium IDE, the new Selenium IDE now supports third-party plugins to extend its functionality. Here’s how to build your own Selenium IDE plugin.

This is pretty exciting. You can imagine companies building plugins to have Selenium IDE do all kinds of things — upload scripts to a functional testing cloud, a load testing cloud, or a production application monitoring service like New Relic Synthetics.

Plenty of companies have integrated Selenium Webdriver into their offerings. I bet the same will happen with Selenium IDE as well.

Speaking of new plugins…

#14 Selenium IDE now can do visual UI testing

We here at Applitools have built a Selenium IDE plugin to do AI-powered visual validations on Selenium IDE, called Applitools for Selenium IDE. (Imaginative, right?)

To get it, head to the Chrome and Firefox stores, do the three-second install, plugin your Applitools API key, and you’re ready to go.

Create a Selenium IDE script, choose Insert new command, type eyes (that’s the name of our product), and insert a visual checkpoint into your test script. Like this:

Visual checkpoints are a great way to ensure that your UI renders correctly. Rather than a bunch of assert statements on all your UI elements — which would be a pain to maintain — one visual checkpoint checks all your page elements.

Best of all, Applitools uses visual AI to look at your web app the same way a human does, ignoring minor differences. This means fewer fake bugs to frustrate you and your developers — a problem that often leads to simple pixel comparison tools to fail. When Applitools finds a visual bug, it’s worth paying attention to.

Here’s an example of Applitools Visual AI in action, finding a missing logo on a GitHub page. We didn’t have to create an assert statement on the logo; Applitools visual AI figured this out on its own.

#15 Selenium IDE can now visually test responsive web apps

When you’re testing the visual layout of your responsive web apps, it’s a good idea to do it on a wide range screen sizes (also called viewports) to ensure that nothing appears out of whack. It’s all too easy for responsive web bug to creep in.

And when they do, the results can range from merely cosmetic to business-stopping. Here’s Southwest Airlines putting the kibosh on their checkout process with a responsive bug that covers up the Continue button:

Not good, right?

When you use Applitools for Selenium IDE, you can visually test your webpages on Applitools Visual Grid. This cloud-based testing service has over 100 combinations of browsers, emulated devices, and viewport sizes. This lets you do thorough visual testing on all your web apps.

Here’s how you specify which combinations to test on:

Once your tests run on Visual Grid, you can easily check your test results on all the various combinations, like this:

Your responsive web bugs can run but they cannot hide…

#16 Selenium IDE now help pinpoint the cause of front-end bugs

Every Selenium IDE script you run with Visual Grid can be analyzed with our Root Cause Analysis.

This matters because, to bastardize Jerry Seinfeld, it’s not enough to FIND a bug. You have to FIX the bug.

Like the Seinfeld car rental company, every testing tool I know of finds bugs, but doesn’t tell you how to fix them.

Except Applitools.

When you find a visual bug in Applitools, click on it, and view the relevant DOM and CSS diffs, as shown below:

I want to point out that we don’t show all DOM and CSS diffs — just the handful that are likely to have caused a visual bug. This makes debugging visual bugs go much faster.

We covered a ton of different ways Selenium IDE and Applitools work together. Here’s a visual summary:

Back to Selenium IDE itself.

Even with all the things the new Selenium IDE can do, there are still things that it doesn’t yet do. Here are two reasons why not to use Selenium IDE:

#1 Selenium IDE doesn’t export Webdriver scripts (yet)

You can’t yet export Selenium IDE tests as Selenium Webdriver scripts. The old Selenium IDE did this, and it’s being worked on actively. You can track progress here.

Update: Selenium IDE now has code export. As of this writing, it’s only for Java JUnit, but it’s a start.

#2 Selenium IDE doesn’t support data-driven scripts (yet)

You can’t yet use the new Selenium IDE to import a bunch of tabular data, like a CSV file or database table, and then run a parameterized test once for each row of data. This, too, is being worked on, and you can track progress here.

Summary

Here’s how Selenium IDE compares to traditional record & replay:

Capability Traditional Record & Replay New Selenium IDE Cross-browser support No Yes Resilient tests No Yes Automatically wait for app under test No Yes Conditional logic No Yes Run one test from another No Yes Embed code into scripts No Yes Edit scripts No Yes Debug scripts No Yes Run scripts in parallel No Yes Run scripts during CI builds No Yes Manage scripts in source code repo No Yes Plugins to extend functionality No Yes Visual UI testing No Yes (w/ plugin) Responsive web support No Yes (w/ plugin) Diagnose root cause of visual bugs No Yes (w/ plugin) Export tests to code No Not yet Data-driven tests No Not yet

‘Less is more

To me, Selenium IDE is part of a larger trend of software making life simpler for technical folks. One example: the broad range of codeless tools for developing applications.

Other examples: Serverless offerings like AWS Lambda make it easier to write just the code you need to get a job done. And Schemaless databases like MongoDB provide architects with much more flexibility to innovate versus tightly constricted SQL databases.

Codeless, serverless, schemaless — and now scriptless, with Selenium IDE. We might be seeing a trend here.

Go deeper

If I were to pick one place for you to get started, I’d recommend this tutorial on Selenium IDE. It’s done by my colleague Raja Rao, a former QA manager who’s been using Selenium Webdriver for over a decade. So he knows what he’s talking about.

Beyond that, here’s a fairly complete list of resources to learn the new Selenium IDE in 2019:

Selenium IDE pages

• Selenium IDE documentation
• Selenium IDE tutorial
• Selenium IDE on Google Chrome Store
• Selenium IDE on Firefox Store
• Selenium IDE GitHub page
• Selenium IDE (SIDE Runner) npm page
• Selenium IDE on Stackoverflow
• Selenium IDE on Quora
• Selenium IDE on Reddit
• Selenium IDE on Hackernews

Applitools for Selenium IDE pages

• Applitools for Selenium IDE product page
• Applitools for Selenium IDE documentation
• Applitools for Selenium IDE on Google Chrome Store
• Applitools for Selenium IDE on Firefox Store

Selenium IDE videos

• Selenium IDE tutorial video playlist
• Dave Haeffner presenting Selenium IDE at Selenium Camp
• Dave Haeffner presenting the new Selenium IDE launch webinar
• Tomer Steinfeld on how he built the new Selenium IDE
• Joe Colantonio presentation on Selenum IDE, part one and part two
• Simon Stewart: Selenium IDE — the next generation
• Other Selenium IDE videos

How do you plan on using Selenium IDE? Let us know!

Originally published at Applitools Blog.