How Do I Build A Culture of Quality Excellence?

Greg Sypolt, VP of Quality Assurance at EverFi, led an hour-long webinar for Applitools titled “Building A Culture of Quality Excellence — Understanding the Quality Maturity Rubric.” Let’s review his key points and his core slides.

About Greg Sypolt

Greg has been writing blogs about his experiences as a quality advocate inside his company. He wants to learn how to make testing more efficient and share what he learns with others. He loves to geek out about testing and technology, but he’s also a sports fan, he loves doing DIY projects, and he’s a husband and father of three.

Overview

Gregs webinar focuses on you — someone who is looking to improve the quality culture at your organization. No matter where you are on the journey, from just starting out to making significant progress, Greg has useful thoughts and ideas for you. His webinar breaks into several key pieces:

Vision

Greg’s key message starts with a key idea: Quality is a journey, not a destination. Your tools and software constantly evolve, and your processes and procedures evolve as well.

How do you build your culture of quality excellence? To start, Greg says, envision an organization where each member has the mindset that he, she, or they own quality. This vision differs drastically from the legacy idea of a “QA team”. By getting on that path towards broad quality ownership, Greg says, each team member can focus on becoming the voice of quality for the organization.

At the same time, Greg says, understand your long-term goals. You want to build an organization where quality aligns with your organization’s other goals. You want to provide both speed and efficiency, in addition to reliability. Your organization likely needs a higher level of collaboration. And, by speed, he means focusing on the speed of testing — making sure that your actual tests run quickly. Quality should not be your bottleneck.

Quality Journey Objectives

Next, Greg shows the following slide:

Greg describes each of these steps in turn.

Greg’s Journey As An Example

Greg shows some data from his experience at his previous company. Greg had worked there for five years. When he started, the company had no test automation. Each release took weeks to accomplish.

At the end of his journey, the company:

By creating all this test infrastructure to go along with the software development builds, Greg’s team created the infrastructure to validate application behavior through test automation. They also built the tools to automate test generation. Finally, they created a discipline to build tests that would run quickly with comprehensive coverage, so testing would allow those 4000+ weekly builds.

Clearly, the process before and after required an evolution at the company. He is working with his team at Everfi with the goal of achieving a comparable outcome.

The Three Pillars of Application Quality Excellence

Next, Greg speaks about the core pillars of building a culture of quality.

First, Greg describes “Quality over quantity.” The first step here is to test the right things, but not everything. Every line of test code you write becomes test code you need to maintain. New features get added, and your test code will likely have to change. You need your developers to learn to think from an end-user’s perspective — how a user will use the application? Analytic tools can help your team understand the paths that users take through your application to know what’s important. That data can help you get a robust test infrastructure that targets how your users use your products.

Quality over quantity also focuses your team on transparency for quality. If a developer makes a change and kicks off a build, the build result provides immediate feedback to the developer and the team if the added feature causes test failures. Hold everyone accountable for the quality of their code.

Second, Greg describes “Work smarter not harder.” At Greg’s previous company, they had a dedicated developer solutions team providing self-service solutions for testing. That team made it easy for developers to build the app and tests at the same time. They also built a central repository for metrics, so testing results across behavior, visual, and integration tests could be viewed and analyzed in one place. They worked on tools to author tests more quickly and more effectively. The result made it easier to create the right tests.

They also looked at ways to include more than just developers in helping to both understand the tests and to help create tests.

Third, Greg explains what he means by “Set the right expectations.” He makes it clear that you need to set the right expectations with the development team. When developers begin to develop tests, they need to understand what you expect from them — both from a coverage as well as velocity perspective. You need to make sure that everyone understands who can ensure quality and performance of the product — and it’s the people who write the code who have the greatest influence on the quality of that code.

Greg also makes it clear that the you need a clear plan to move to a more mature quality structure. Everyone needs to know what is expected, what will be measured, and what outcomes the team wants to achieve.

Measuring Quality Maturity Rubric

At this point, Greg shares his rubric for analyzing quality maturity.

He mentions that the metrics involve the people on your teams, the technology you use, and the processes you put in place. Your people need to be ready and organized appropriately. You require the right technology to increase your automation. And, you need to put processes in place that allow you to succeed at improving your quality maturity level.

As a preface, Greg describes these metrics and this rubric as based on his experience. He expresses an openness to update these based on the experience of others.

The Quality Maturity Rubric

His full rubric involves 23 different maturity attributes that each have four range values:

Greg goes through several attributes in detail, giving examples of beginner through expert for each. For example:

This table shows the “Culture” attribute.

To use this table, compare your organization to the description of each value that most closely matches yours. Give yourself a “1” if you’re a beginner, “2” if you’re Intermediate, “3” if you’re advanced, and “4” if you achieved expert.

Greg has created tables for each of the attributes. For example:

For the Environment attribute, the Advanced level involves automated builds and staging, with push-button deployment.

Using the rubric, you go through each attribute and assign a numeric value based on how close you are to one of the levels.

As you go through the analysis, you can also use the table to set your goals for improvement over time, as you look to increase the quality maturity of your organization.

Greg posted his grading rubric online — you can find it at:

http://bit.ly/qmc-grading-rubric

Once you use the grading rubric, you can start to figure out the next two steps:

Quality Roles

Now that you have people, processes and technology thought through, along with your approach to maturity, it’s time to think about your people and their skills.

Greg presents a map of QA role clarity that helps you think about your existing technical skills among your team. Greg created this table based on his experiences in quality engineering.

Greg sees this table as another rubric you can use to evaluate your people. You can evaluate where you are today. And, you can start to think about skills you want to add to your team. The people on your team will help you execute your vision, so you need to know where you are and where you want to go.

You can look to hire people with these skills. You likely can find engineers with Level 3 skills of SDET. More than likely, though, you, like Greg, will be building these skills among your team over time.

Building A Quality Platform

Once you have thought through your quality maturity, reviewed your existing processes and technologies and begun to evaluate your team, Greg wants you to think about your current and future states as a “quality platform”.

He reminds you that your quality platform serves key outcomes:

The combination of people, processes, and technologies that make up your quality platform can help you deliver major quality improvements.

At the core of your quality platform is the Developer Solutions team — working to create quality solutions among the software development team.

Next, you need data insights that help create visibility across the organization.

Third you develop turnkey solutions that simplify the deployment of key functions or processes. For example you can easily deploy Selenium or Cypress test automation through a set of well-defined code structures, and. You can easily build new tests and structures in your code. For example, at EverFi, Greg has deployed Applitools to easily add visual validation to tests — simplifying overall test development.

Fourth — your platform team serve as quality ambassadors. They represent quality practices and endorse effective change within the organization.

Fifth — you focus on functional values of the platform, making testing better and easier for everyone.

Lastly, you focus on non-functional behaviors, like monitoring the critical paths inside your application. You make sure to understand the critical paths and ensure they work correctly.

Reaping The Savings

Finally, Greg gets to the bottom line.

How does the quality platform deliver savings to your organization? Greg shows this handy table.

Each element of the quality platform contributes savings.

Greg gives the example of Data Insights providing cost savings because they help you know where you can add tests as well as providing data that can be consumed by the team. Developer Solutions helps you reduce the time to deploy pre-existing or new solutions. Functional tests can help improve your quality by ensuring you run tests on every pull request and get instant feedback, instead of waiting for savings. Or, a turnkey session with an external vendor can help you get up to speed quickly with the vendor’s technology.

Building Your Own Quality Platform

As Greg points out, moving to your own quality platform results in a journey of constant improvement. Your begin with your current state, envision a future state, and move to deliver that future state through people, technology, and process improvements. Each step along the way you end up with measurable savings.

Here’s to your journey!

For More Information

Originally published at https://applitools.com on September 1, 2020.

Deliver visually perfect web and mobile apps with AI-powered end-to-end visual testing & monitoring. Get started, free: http://bit.ly/ApplitoolsFree