Thread by @AndrewBennet

Today I released v2 of @ReadingListApp which *finally* includes iCloud Sync. I've been promising this for years; it turns out two-way cloud sync is hard 😅

Here are my learnings, which might be useful to those interested in #iosdev

https://twitter.com/ReadingListApp/status/1379690919050555396?s=20

I'm using Core Data and CloudKit - you can get the two syncing relatively quickly with NSPersistentCloudKitContainer, but it has drawbacks:

❌ It's a black box. if sync isn’t happening quickly, or behaving weirdly, there’s not a lot you can do to work out why 🤷‍♂️

❌ Merges are resolved via last-writer wins across entities, not attributes. In @ReadingListApp, a Book entity has lots of fields. This merge behaviour means that some local changes could be erased by the app receiving remote changes of other fields 😢

If these issues don't matter to you, then consider using NSPersistentCloudKitContainer as it will be much less work than developing your own solution! developer.apple.com/documentation/…

If you want full control, you have to roll your own sync engine, as I did:

The general approach:

• Watch for changes on a background managed object context
• Use CKModifyRecordsOperations to upload local changes
• Use CKFetchRecordZoneChangesOperation to receive remote changes

See WWDC2016 231 for info: developer.apple.com/videos/play/ww…
@_inside also has a great demo project and blog post: rambo.codes/posts/2020-02-…

Here's a few Tips and Gotchas I've found useful during developing:

💡 Tip:
Run your operations on an OperationQueue with maxConcurrentOperationCount set to 1. Running one operation at a time helps keep things (reasonably) understandable. It also ensures you process local changes in order they occurred.

💡Tip:
To avoid pushing stale data, use chained operations to attach CKRecords to a CKModifyRecordsOperation. This ensures that you generate the CKRecords at the “correct” point in the operation queue. Perform context merges in the queue too, to keep change processing consistent

(Tip continued): without the above approach, processing two local changes made in quick succession would cause the second upload to fail when the original CKRecord change tag is used. Deferring CKRecord creation until the operation is processed means the correct tag is used 👍

🪤 Gotcha:
When fetching record changes, you might get a CKRecord containing a CKReference to another record that you’ve not seen yet. You’ll have to store the referenced record ID locally somewhere, to resolve into a Core Data relationship when the referenced record arrives

💡Tip:
Keep track of all of the local modifications which have not yet been pushed to iCloud. When remote changes are received, partially apply the CKRecords to the corresponding local object, skipping the fields with local changes so as to not overwrite new local data.

💡Tip:
Modelling ordered relationships in a CKRecord is tricky, since a CKReference cannot have an order. Instead, I modified my Core Data schema to use unordered relationships, but with a bridging entity containing 2 references and an index, synced as standalone entities.

💡 Tip:
Use NSPersistentHistory gather local changes. These are persisted, so don't worry about what might happen if the app terminates before you process the change! Store the last processed token in some persistent storage like UserDefaults (or github.com/AndrewBennet/P…)

🪤 Gotcha:
There is a horrible bug somewhere in Core Data model migration (or Xcode) when you use the Preserve After Deletion attribute: Mapping Models end up with incorrect hashes, which means that loading a mapping model by providing a source and destination model fails...

(Gotcha continued):
To resolve this, I had to add in some ugly code to recalculate and replace the hashes of the entity mappings in a manual loaded NSMappingModel 🤮 (it works though)

💡Tip:
If there is a meaningful external identifier of your entities, use them as the basis of your remote CKRecord IDs. For @ReadingListApp, I'm using external book identifiers (Google Books IDs) as a basis, which prevents duplicates ending up in iCloud

💡 Tip:
To plan for future model changes, include an integer field on all CKRecords you upload, based on app version. If you ever receive a record with a larger version than currently running - stop syncing until upgraded!
[caveat that this hasn’t been tested in the wild yet...]

There’s a load of stuff I could go into: data mapping, merge strategies, error handling, change processing, etc, but probably best suited to a blog post one day. All in all there’s a heck of a lot of code required.

The end result is working rather nicely though!