Finding accessibility issues in an app is an incredibly manual task that can take hours depending on the size. Which seems like a perfect place for ✨AI ✨
🧵Surfacing accessibility issues in the @Drizly app w/ AI 🧠
A quick background - our Snapshots product turns Previews into snapshots. A recent feature gives the ability to use AccessibilitySnapshot w/ one line
You don't need to write test code to get snapshot tests or to snapshot VoiceOver elements
We can take snapshots for any app leaking Previews into production, which @Drizly (and many other apps) are. We were able to generate 86 snapshots for Drizly
We then fed these images into an AI we're working on to detect accessibility errors. Here's what we found
20 screenshots are using file names and file paths, which can be confusing when read by VoiceOver
Here are examples where the VoiceOver has to read
gifting/guided/close
gifting/address_circle
onboarding_v2/two
VoiceOver reads "underscore" and "slash" aloud, so a user hears "gifting slash guided slash close. Button"
Here's an example of Drizly's home page with these elements read aloud
We found 6 screenshots with insufficient description for interactive elements
Notice VO w/ just "Button", but no accompanying text to read what the button does
We found another 7 screenshots where there was no accompanying VO text
Notice the "X" icon in the top left has no accompanying VO
Some of these might be expected as they look like empty inputs. This is an area where the AI can continue to improve
Wrapping up - here is the count of manually verified issues the AI caught
20 previews w/ non-human text
7 previews w/ insufficient description
6 previews w/ empty VO text
It's still early days of this work, and it's by no means perfect, but we're very encouraged by the results we've seen so far. We'll continue to refine our accuracy and expand what we can help catch 🔬
If you're interested in learning more about this work, please get in touch!
• • •
Missing some Tweet in this thread? You can try to
force a refresh
Another day, another "How is this app so big tweet" 🤓
This time its the @DJIGlobal iOS app, which comes in at 1 GB
A lot of the comments speculated that the app was large due to assets & while there are 155 MB of .mp4 video files, the bulk of the size is coming from ML models
There's 264 MB of .mlmodelc files - these are compile Core ML Models which are designed to run on Apple devices. Some of the biggest models 👇
Here's a top level view of the latest version of LinkedIn
300 MB for just dynamically linked frameworks & Plugins is...a lot. In fact, just the Dylibs & Plugins today are bigger than the entire app was back in November 2022 🫠
And the Plugins and Frameworks seems to be where the greatest inefficiencies lie. Here is a more detailed look at the LinkedIn Plugins + the dynamically linked VoyagerLibs.framework
How did the @Twitter iOS app change over the last 6 months? Why did the latest release of Google Translate Android reduce app size by 1/3rd?
We try to make these questions easier to answer, which is why we're very excited about our newest feature, AI summaries of build diffs
Jumping right in - Twitter v9.59.1 vs. v9.34.6 (iOS)
Overall size decreased by 34 MB, but how quickly can we identify what changed?
The summary is pointing to removals of plugins and bundles, which is easy to see in the X-Ray diff
T1Twitter.framework is also highlighted in the summary. Searching through the diff, we can see that many assets were modified or removed to reduce its size