Thread by @ddkilzer

A thread on thread safety crashes in Apple OSes. Yesterday I helped both a third-party developer and an Apple engineer analyze crashes that turned out to be thread safety issues. (Yes, that's me as Buzz Lightyear in the Woody-Buzz-Everywhere meme.) #WWDC19

NOTE: This is a mini "field guide" to thread safety issues with objects (data structures) in languages like C, C++, Objective-C & Swift. It is not meant to be comprehensive. Thread safety bugs also occur in plain old data (POD) types (int), but those usually don't cause crashes.

Thread safety bugs (race conditions) occur when memory is read-after-written, written-after-read, or written-after-written on two threads at the same time. Read-after-read is generally safe as long as all writing is completed during initialization (before objects are used)!

Now that you know what causes a thread safety issue, you can simply review all your source code and fix all the bugs!

Unfortunately, most folks don't have time to do that, so let's talk about what thread safety crashes look like on Apple OSes.

SIDEBAR: Enabling Thread Sanitizer, then Build & Run in your Xcode project for iOS/iPadOS/tvOS/watchOS Simulator or macOS is the best way to find and debug thread safety issues short of code inspection. Come to clang/crash labs for help with setup/analyzing/fixing issues! #WWDC19

Thread safety crashes tend to manifest as multiple signatures that occur in the same area of code, usually reading or writing the same object. Ranking crashes from most to least, an exponential decay curve is usually observed. This is a "smell" that a thread safety issue exists.

Another crash "smell" is one crash signature that happens on different threads (main & background thread, or dispatch queue "A" & "B"). Look at all crashes (not just one!) to observe this pattern. Sometimes you can find a crash log that shows the thread safety issue "in action"!

Once you've identified an object or method that's crashing, analyze the code to determine if it can be called on two threads at the same time with the same object, or try to reproduce with Thread Sanitizer enabled in Xcode Build & Run. Try writing a standalone test, too.

Assuming you've identified the thread safety issue, the general fix is to serialize access to the object. Common solutions are using a serial dispatch queue or fine-grained locking with NSLock, @synchronize, etc. Each scenario is unique, so research serialization techniques.

Now that I've discussed the root cause and the general solution, I'll note a common anti-pattern in code, some crash log patterns for Objective-C and a debug mode for CoreData.

A common anti-pattern in Objective-C and Swift code is to assume mutable collection objects (arrays, dictionaries, sets) are thread safe. They are NOT! Adding, removing and updating objects in collections must be serialized in both Objective-C and Swift.

Thread safety crashes in mutable Objective-C collection objects may look like a crash due to an over-released object at first glance, but with enough crash data, there are four different patterns seen in the crash logs:

Patterns for thread safety crashes at one signature in mutable Objective-C collection objects:
1. SIGSEGV on near-NULL ptr (0x10, 0x20).
2. SIGSEGV on valid-ish heap address.
3. SIGABRT (NSException) for -doesNotRecognizeSelector:.
4. SIGSEGV on "random"/garbage address.

If you see the above patterns ⬆️ in the crash logs for a single crash signature for your app, and you're crashing when reading from or writing to a mutable NS collection object (array, dictionary, set), you probably have a thread safety bug.

If you see a SIGABRT (NSException) crash with this frame near the top of the crashing stack, then you have a thread safety bug accessing the NS mutable collection object (or you're modifying the object inside the loop itself!):

CoreFoundation: __NSFastEnumerationMutationHandler

Note that the __NSFastEnumerationMutationHandler exception ⬆️ does NOT fire with CFMutableDictionary objects prior to iOS 13 or macOS Catalina, and will only fire if Foundation.framework was also loaded in process.

If you see a SIGTRAP with this frame near the top of the crashing stack, then you have a thread safety bug accessing an SQLite database using the same connection object on two threads at the same time:

libsqlite3.dylib: sqlite3MutexMisuseAssert

Finally, if you're seeing a long tail of (mysterious) CoreData crashes, try enabling the com.apple.CoreData.ConcurrencyDebug mode on your app and running it from Xcode or try living on it. Fix all the crashes you find in this mode.
stackoverflow.com/questions/3139…

I hope you find this information to be useful!