Thread by @emergetools

Finding accessibility issues in an app is an incredibly manual task that can take hours depending on the size. Which seems like a perfect place for ✨AI ✨

🧵Surfacing accessibility issues in the @Drizly app w/ AI 🧠

A quick background - our Snapshots product turns Previews into snapshots. A recent feature gives the ability to use AccessibilitySnapshot w/ one line

You don't need to write test code to get snapshot tests or to snapshot VoiceOver elements

We can take snapshots for any app leaking Previews into production, which @Drizly (and many other apps) are. We were able to generate 86 snapshots for Drizly

We then fed these images into an AI we're working on to detect accessibility errors. Here's what we found

20 screenshots are using file names and file paths, which can be confusing when read by VoiceOver

Here are examples where the VoiceOver has to read
gifting/guided/close
gifting/address_circle
onboarding_v2/two

VoiceOver reads "underscore" and "slash" aloud, so a user hears "gifting slash guided slash close. Button"

Here's an example of Drizly's home page with these elements read aloud

We found 6 screenshots with insufficient description for interactive elements

Notice VO w/ just "Button", but no accompanying text to read what the button does

We found another 7 screenshots where there was no accompanying VO text

Notice the "X" icon in the top left has no accompanying VO

Some of these might be expected as they look like empty inputs. This is an area where the AI can continue to improve

Wrapping up - here is the count of manually verified issues the AI caught

20 previews w/ non-human text
7 previews w/ insufficient description
6 previews w/ empty VO text

It's still early days of this work, and it's by no means perfect, but we're very encouraged by the results we've seen so far. We'll continue to refine our accuracy and expand what we can help catch 🔬

If you're interested in learning more about this work, please get in touch!