Thread by @jamonholmgren

Tonight's "coding for fun" task is to figure out why we're running into a bit of a perf problem with MobX-State-Tree and MobX-React-Lite in a specific large data set situation.

I'm first going to try to replicate the conditions in a bare React Native app.

Live-tweet it?

First, I'm spinning up a new RN app.

`npx react-native init MSTPerf`

I could have used the TypeScript template project, but decided to add it myself using the instructions here.

reactnative.dev/docs/typescript

Vanilla app is working on iOS, of course. (But I am the type of programmer who checks anyway.)

tf is going on with Android

Um, so, resizing and rotating the Android emulator apparently doesn't work very well?

ok android emulator stays full size ... fine

Adding MST and co

Going a little slow here because, well, not really in a hurry.

Anyway, starting to put together a root store now. This seems to happen most often with large, complex, nested lists of models, when replaced all at once.

Data showing up in logs now

console.time requires a polyfill (I just grabbed this one: github.com/MaxGraey/react…) but I now have some basic perf times.

I need a large dataset. My favorite one is a list of NBA players and teams, along with their ratings and statistics, located here: raw.githubusercontent.com/alexnoob/Baske…

I'm starting by defining a player model and letting most of these be "frozen" data types (aka treated as if they are immutable, and thus not observed).

I will make them observable next, so we have deep & complex MST objects.

Pretty simple api service

My RootStore now can fetch the players from the API and apply that snapshot. I think. Haven't, you know, actually tried it yet. lol

we got objects for days ... it's loading!

So, I want to be testing in production mode, since MST does a lot of optimizations in prod that it doesn't do in dev.

Rebuilding the app...

One fun thing about MST is that you can just copy real data into its models and it'll figure out the type (both TypeScript as well as runtime type checks).

I'm gradually replacing these `types.frozen` properties with real MST models.

Lots of runtime type errors because this API data is kinda dirty. But I'm getting there.

Now we're going to make the app an observer, to capture changes to the store or models that are used within.

Having some problems with making the app run in prod, for some reason, so I'm just going to force it to load what I want.

Objective-C baby

Oops, ignore the syntax error there ... I fixed it 😅

So logs don't show up in terminal anymore when I'm running in prod mode, but I can find them in Xcode. Fine.

Okay, so it's loading 872 quite complex "players" into the system in 1.18s, which isn't amazing but also isn't too bad, considering they're fully "observable" objects.

So now I'm going to render them on the screen, and then trigger re-hydrations to see what happens. Maybe even rearrange them while I'm at it.

It's hydrating. Noice.

Accessing some sub-objects now, just to do it.

Perf seems decent so far.

Okay, so I think I"m starting to see the hints of the issue we're noticing.

Notice that the first log is 1179ms and the second is 1737ms?

So, the first log is *inside* the second's scope, meaning that ideally they would be the same number.

`applySnapshot` is all that's running in `replacePlayers`. So why is `replacePlayers` taking so much longer? 🤔

Proving it. See?

Getting my kids sent to bed and settling back in for a bit longer.

I need to make this even more complex to see where this issue is really coming from.

I'll make "teams" and assign players to each team, too. With references back and forth maybe?

Okay, so before I even get that far, I'm starting to see the behavior exhibited before.

This is without displaying any data, so no observers are being harmed here.

The initial load is just over 1.1s. But the reload is taking like 5 seconds. 5.4, to be exact.

Why...

It also happens when I do `applySnapshot` instead of `.replace()`.

Okay, I'm going to see if I can replicate this in a simple React web app.

I'm impressed with myself at how fast I spun up a new completely bare-bones React app. Not using CRA or anything, just Parcel, index.html, and a few tsx files.

Anyway, self-congrats aside, yes, this happens on web too.

So apparently it's kinda a pain to use @brave with the Chrome Debugger + VS Code.

Downloading Chrome *just* for this purpose. Brave has been entirely fine otherwise as a Chrome replacement.

@brave Debugger running!

@brave It's interesting stepping through the code and seeing what's running.

When the action runs, MST doesn't update any views or observers until the end of the transaction, which is cool

Okay, it's past midnight, so I'm going to call it a night.

HOWEVER

I figured out something *very* interesting.

If I run `.clear()` on the large dataset before repopulating it, there is no longer the long lag.

This leads me to believe MST is doing a bunch of checks against the existing dataset that is just taking forever.

Perhaps @mweststrate can help me figure this out. I'll share the code.

@mweststrate Github: github.com/jamonholmgren/…

1. Clone it down
2. `yarn`
3. `yarn parcel index.html`
4. Open localhost:1234
5. Open the console & observe

And here’s the answer:

https://twitter.com/mweststrate/status/1415947375525474304

Going to work on solutions next. Perhaps a new way to update MST models without doing the merge?