A Guide to Automated Testing with Drupal and Applitools

Traditionally, web applications built on Drupal are built using various entities like Content types, blocks, components using Layout Builder, and then the product is made available to the end-user on the front-end using HTML, CSS and JavaScript. The team usually starts with back-end stories related to building various content types, related roles, and permissions, and then the front-end team picks it up to make the site more usable and accessible as per the design requirements.

Of course, with component libraries like Storybook, Fractal, PatternLab, and with designs in place, the front-end team can start implementing them as component libraries in parallel, which are later integrated with Drupal.

Let’s talk about testing the application in the next section.

Automated Testing in Drupal

With these tools, one can automate unit, integration, and acceptance level tests. But what about automating the visual tests? That’s the missing tip of the pyramid which we will address through this blog post.

We have used Cypress to automate the browser and Applitools for AI-based visual validation. Our reasons for using Cypress over other tools are many, including the following:

  1. One can quickly get started with writing actual tests with Cypress as compared to Selenium.
  2. Cypress enables fast-paced test execution.
  3. Our POC with Cypress + Drupal proved that testing the Drupal side of things can also be achieved with Cypress.
  4. Cypress offers harmonious integration with Applitools. Having said that, please note that Applitools does have SDKs for Selenium PHP and NightwatchJS and many more just in case you have your existing automation functional suites written using any of the other testing frameworks.
  5. Since Cypress is a JS-based framework, developers can also contribute to writing automated tests.

The site to demonstrate the concept is Drupal Umami, the major advantage being that the site is already constructed and we can directly focus on writing automated visual tests without having to worry about creating these pages.

NOTE: If you are completely new to the concept of automated visual validation testing, then please refer to the course “Automated Visual Testing: A Fast Path To Test Automation Success” on Test Automation University from Angie Jones.

Applitools and Drupal

The test in the above example launches the homepage and verifies the UI using the “ checkWindow()” command. The checkWindow() command takes in the following parameters:

  1. tag: The name of the test.
  2. target: Shall the window be targeted or a particular element?
  3. fully: Identify the scope, whether it is the current viewport or the entire window.

That’s it! And you are ready to execute the first automated visual test. So, let’s execute it using the command `npx cypress run`, assuming that the baseline image was captured correctly on the first run.

Here’s a small screencast for the same.

Interpreting the automated test execution results

The tests passing is a good thing. However, that’s not the primary reason for having automated tests. You want the tests to correctly detect the discrepancies as close as the point of introduction. For the purpose of this demo, we have intentionally introduced a couple of CSS bugs on the homepage through Cypress’s invoke command. Once the script launches the homepage, CSS changes are made at run-time in the browser and then the screenshots are captured as below:

Let’s re-execute our test to see how the tool catches these bugs. The tool has correctly highlighted the three errors (in pink color) below that we introduced on purpose:

  1. “Search bar” in the header has shifted from its original position.
  2. The font-color for the “View recipe” button on the banner has changed.
  3. “Find out more” link in the footer.

We confirm that these indeed are bugs, and reject the image marking the test as failed and then report bugs in Jira directly from Applitools. Additionally, the root cause analysis feature from Applitools helps us quickly identify the visual (UI) issues, in this case, caused by CSS changes as shown in the images below:

Until now, it was only about one browser. However, if we really want to leverage the automated tests written for validating the UI, then the true benefit lies in having the ability to execute these tests across several browsers, Operating systems, and devices. Verifying that the UI looks correct on one browser/device doesn’t guarantee that it would look exactly the same on all other browsers/devices because the rendering of the UI might be different on other browsers/devices.

Cross-browser/device/OS testing using the Ultrafast Test Cloud

  1. Configure your suite to execute automated tests across several browsers and devices. However, not only the test authoring time increases but also creates a dependency on the technical staff as the logic for tests to run all tests across several browsers and devices need to be coded.
  2. Linear automated test execution increases execution time, thereby resulting in larger build times and delaying the feedback of the application to the entire team.
  3. Maintain an in-house grid for parallel execution or purchase additional subscriptions provided by Cloud-based solutions for parallel execution of automated tests.

This brings us to discussing the Applitools Ultrafast Test Cloud which inherently takes care of the above points.

By using Applitools Ultrafast Test Cloud, you would be able to execute the automated visual validation tests across several browsers, operating systems, and devices of your choice and at lightning speed as although the test runs once on say Chrome (assuming Chromedriver is configured in the tests), the capturing of the pages occurs in parallel, in the background for all the configured browsers and viewports.

So, let’s write some more tests for Articles and Recipes landing and listing pages on the site. Let us also execute these tests in parallel on several browsers/devices as configured below using Applitools Ultrafast grid solution:

Here are the Ultrafast Grid Test Results across several browsers and devices.

To be precise, here are the details:

  1. Number of browsers and devices = 7
  2. Total number of functional tests = 6
  3. Total number of visual tests = 7*6 = 42
  4. The time taken for complete execution on Ultrafast grid — 5 minutes 2 seconds

Execute once and see the results on so many browsers and devices. Now, that’s what I call truly automating your visual tests.

Also, notice that using the Applitools Batch feature, we have logically grouped the Test Results, to make it extremely readable.

Other tools in the market

The Applitools Advantage

  1. The AI-driven image comparison algorithm is incredibly accurate and avoids false positives which otherwise occurs in a pixel-to-pixel based comparison. The amount of time it would take to troubleshoot false positives, especially on full-page screenshots would be time and cost-prohibitive. Pixel-based is ok for verifying small components across a short period of time, otherwise, it breaks down.
  2. Seamless integration with your existing functional automation suite through several Web, mobile, screenshot, desktop, and codeless SDKs available for testing and automation frameworks like Cypress, Selenium, WebdriverIO, Nightwatch, Appium, and also languages like PHP, Java, Javascript, C#, Python only to name a few.
  3. With the help of Ultrafast Test Cloud, your entire web application can be tested for visual accuracy at a fast speed (as the tests run only once whereas the visual rendering validations happen in parallel on several browsers and devices in the background) with guaranteed reliability and security.
  4. Applitools also provides out of the box integration with the Storybook component library for React, Vue and Angular.
  5. Learn more about the following Applitools Eyes integrations on their site:
  6. Integration with GitHub
  7. Integration with Microsoft Azure
  8. Integration with GitLab
  9. Integration with BitBucket
  10. Integration with Jira
  11. Integration with Email and Slack for notifications

What next?

Originally published at https://applitools.com on December 8, 2020.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store

Deliver visually perfect web and mobile apps with AI-powered end-to-end visual testing & monitoring. Get started, free: http://bit.ly/ApplitoolsFree