Catch ‘Em All: Hunting Accessibility Bugs Like a Champion with Automated Testing
So, you wanna be the very best in software testing, like no one ever was?
Testing for access barriers is part of a comprehensive quality assurance process. But where does one start? While the WCAG are as extensive as the Pokédex, what exactly is the assistive tech equivalent of Bulbasaur? Where do you start to catch ‘em all?
Accessibility is a key part of quality assurance, but navigating the world of accessibility testing tools can feel as overwhelming as choosing your first Pokémon. So, where do you start?
The WCAG are your Pokédex
How many Pokémon are there? Depends on the edition, but let’s go with the 151 from the 1st generation. How many success criteria are there? 85 in the 2.2 version. Just like Pokémon have their evolutions, success criteria have levels: A, Double A, and Triple A.
You can think of them as the same thing: each Pokémon starts out at level A, which is its base form. You can’t battle with an egg (unless you’re Misty), and your website needs to pass at least level A to have basic functionality present. The better your website usability gets, the more high-level success criteria you can pass. Like a training montage!
Not all success criteria are applicable to all websites or apps, but certain criteria are always required, such as:
- 1.1.1 Non-text Content
- 1.3.3 Sensory Characteristics
- 1.4.1 Use of Color
- 2.1.1 Keyboard Operable
- 2.1.2 No Keyboard Trap
- 2.3.1 Three Flashes or Below Threshold
- 2.4.1 Bypass Blocks
- 2.4.2 Page Titled
- 2.4.3 Focus Order
- 2.4.4 Link Purpose (In Context)
- 2.5.3 Label in Name
- 3.1.1 Language of Page
- 3.2.1 On Focus
- 3.2.6 Consistent Help (from WCAG 2.2, aka the 4th gen)
- 4.1.2 Name, Role, Value
These are the very, very basic requirements for your website, the very first Pokémon you catch on your adventure. Others will also be applicable, depending on the type of content you add, but these guys will be your MVP (minimal viable Pokémon) team to challenge the first gym leader to get your Level A compliance score.
While some success criteria build on each other, others are stand alone criteria. Like Ditto, Lapras, and your favorite legendary, they are not part of an evolutionary line, but they could be Level A, AA or AAA in their default form.
So while we can agree that Charmander is cute, ultimately we all want Charizard. In the same way, we shouldn’t be content with simply passing AA. If you want to become an Accessibility Champion, keep evolving and keep striving for AAA.
Pick your starter!
Now that we’ve collected our WCAG Pokédex and encouraging words from Professor Oak, it’s time to pick a starter and get started with accessibility testing!
Automated Testing
Automated testing has come a long way, but it’s not yet good enough for the Champions League. So to become an Accessibility Champion, you need to learn how to pick the right starter testing tools, then, with a few manual testing tips and tricks, plus a training montage, you will be catching accessibility bugs and barriers, like no one ever has! Let’s go meet some starters:
Pikachu = Google Lighthouse
I know Pikachu is as iconic as it can get, all the while, Google Lighthouse has a bad reputation due to a lack of accuracy. But hear me out: Lighthouse isn’t a bad tool. It can teach you a lot, but has its limits, and you won’t be able to challenge the Accessibility Champions League with it alone.
It suffers from the same problems all automated testing tools have: They can only check what they can check, meaning that they may, for example, state that alt-text is used correctly, even if the string value looks something like “asdfgThisIsGibberishTM_randomStringOfNumbers”. Yes, alt text is technically present, but with content like that, the screen reader might as well be freestyling.
But the errors it throws up are correct. You will have a straightforward list of what to fix, which makes for an easy starting point.
Web accessibility is a long road to master, including knowledge of disabilities, assistive tech hardware & software, legal frameworks like the ADA or EAA, stretching all the way to human rights, but every now and then, you will still get stuck on a bridge with niche ARIA labels and the question of what they are compatible with, blocking your way.
Bulbasaur = WAVE
WAVE is a web accessibility evaluation tool by WebAIM:
Rather than providing a complex technical report, WAVE shows your original web content with embedded icons and indicators that reveal the accessibility information within your page. WAVE cannot tell you if your web content is accessible – only a human can determine true accessibility – but WAVE can help you evaluate the accessibility of your web content.
It is a solid choice when you are starting out, but (like the plant starter in each generation) it also has its weaknesses. For example, Many landing pages use hero images at the top. WAVE doesn’t recognize if the color contrast between the background image and the text in an overlay on top of it is sufficient; all it sees is a textbox with a transparent background, so it will assume that
What I like most about it is that it lets you reference the relevant part of the source code for each issue. To become an Accessibility Champion, you have to learn how to handle accessibility in code, too. With WAVE, it’s very much a bite-sized introduction to it.
Both tools will point you toward learning resources for the particular errors they recognize. This is what makes them the perfect starter Pokémon: They are easy to use and show you where to go for more training.
Links
