Thread by @dan_abramov

Love this thread. Someday I’d like to write (or contribute to) a five year retrospective of this project. So many stories! For now I’ll do a small thread of random fun moments I remember from the last couple of years.

https://twitter.com/rickhanlonii/status/1402771549808214016

When we started, we wrote our own scheduler with a plan that eventually the browser would provide a native one. Our work on React took long enough that by this year, the native scheduler already exists behind a flag and we could even experiment with it! chromestatus.com/feature/603116…

https://twitter.com/rickhanlonii/status/1402771556716236801

Our first thinking about how Suspense should work didn’t make sense. We found out the hard way in the middle of the FB website rewrite in 2019. A simple change could make UI stall for 5 seconds and engineers couldn’t say why. We couldn’t either. We were confused by our own API!

https://twitter.com/rickhanlonii/status/1402771554220584961

To understand those issues, I’ve met with around ten product engineers, and documented every single problem they’re aware of. These are notes that I took. I categorised the problems and why fixing one led to the other with our flawed APIs. (“Blue” is old website, “Comet” is new)

Based on the information we gathered, @sebmarkbage went back to the drawing board. Informed by all the product use cases and conversations with colleagues outside our team (special shout-out to @_jstejada @catchingash @frankyan @en_JS @jingc @steveluscher) he redesigned the API.

In 2020, we had a different set of issues. The redesigned Suspense-related APIs composed well but the implementation had a lot of bugs. To give you an idea, here’s a Thing(tm) we saw when an engineer tried to add data refetching. Pretty scary. We methodically fixed each bug.

Reproducing these bugs and then figuring out the reason was difficult. My favorite strategy was to delete 99% of the component tree until the buggy part was isolated. Then I could see the exact order that causes issues, and try recreating it in a sandbox. Some bugs took a week.

When I isolated a bug, I would try to write a failing test or make a sandbox. @acdlite or @sebmarkbage would come in and fix them. We added a regression test for every single bug — so that it never happens again. This is a rule: we don’t fix things unless we know why they broke.

With time we’ve noticed many bugs had common causes. Some parts of our code were overly “clever” and fragile. In some places the whole model was flawed and fixing one thing broke the other. This knowledge led to @acdlite, @lunaruan and @brian_d_vaughn doing two huge refactors.

At some point after the refactors rolled out, these bugs just… stopped happening. We haven’t had a big one for months by now. But rolling out those refactors was very hard because they could break everything. So we kept two versions of most files, and shipped both as AB test.

One of these refactors was risky because we replaced a “clever” fast mechanism with a more naïve one. This fixed bugs but AB test showed a significant perf regression. We couldn’t ship it because we didn’t know the cause. We can’t slow down all React apps due to some refactor.

We tried many strategies and tried a bunch of optimizations. We ran many experiments. It was slow. We couldn’t guess where the problem is but we were hoping it’s a bug and not a fatal perf flaw. So we reverted the entire refactor and then split it into small individual commits.

In retrospect, we should’ve done that from the beginning. It seemed gruelling to redo the work as small atomic changes and AB test each change separately (for at least a week) but finally we found the bad commit. In a small change, the team guessed the issue and the fix worked.

This refactor unblocked the remaining work which was needed to fix the last significant set of issues with Suspense API that we’ve learned over the two years of using it extensively in the product code. We switched to working on incremental adoption right after in early 2021.

I know people have mixed feelings about FB getting “early access” to features. But I am 100% confident that shipping what we had in 2018, 2019, or 2020 would have been a disaster. Open source releases have a high bar. We still support design mistakes from 2014 for stability.

I felt in the dark many times. I felt we’ve let down the community by sharing our vision too early and now having to focus on putting out the fires while people think we forgot or don’t care. There’s one thing @sebmarkbage told me many years ago that kept me going.

“Our work is rooted in theory. In theory, some approach is better. So we pursue it. Many people don’t trust the theory. But we keep on going. While it may take many practical compromises and pragmatic sacrifices to get there. If we keep on going, eventually the theory wins.”

Now, back to work.

Of course I missed someone. @rhagigi deserves a double-special shoutout for all his feedback and help. I probably missed many more.

I want to clarify one thing. Deploying at FB is how we prove a feature works and scales. But we’ve said no to many features that FB engineers wanted. We’re driven by our vision (based on our shared experience of what makes UI programming hard). FB product code is a stress-test.

https://twitter.com/dan_abramov/status/1402949268877582336

I know it’s hard to believe. But for example we started work on concurrency way before FB embarked on the new website. In fact we were told many times that a React rewrite can’t happen. The concurrency work spawned from a theoretical 2014 prototype—because it made sense.

Early on, many FB engineers were extremely skeptical about basing the new website on our research. (In a way, they were right—it was nowhere near ready!—but many also didn’t believe the model would ever work or bring benefits). If we didn’t have our vision, we’d give up long ago.

Our team is lucky to have had enough trust from our colleagues that we were allowed to take our wild ideas and put them into the pressure cooker of a website rewrite. It was very hard but we both got what we wanted. FB got the perf, and we got the API and model proven to work.

Now we have refocused our work entirely on the community rollout. That’s what React 18 Alpha and Working Group are about. From research to production to open source. We think these features will already benefit many React apps, and we’re excited re: what we can build on top.

I’d like to think in the future we could make the dogfooding process less FB-centric. It’s hard because fixing early issues requires full access to debugging environment. But I can imagine the team expanding to have non-FB engineers working full time and testing on their apps.

We’ve already had informal partnerships with big frameworks like Next and Gatsby that added non-FB input at early stages. React 18 Working Group is the first attempt to formalise this. If this effort is successful we can think about expanding it further.

Let me clarify this point real quick. It’s not about me investigating a FB-specific bug. Think of FB as a giant fuzz tester. The codebase is so large that every possible bug shows up somewhere. You wouldn’t want this one in your app! When bugs like this exist, we can’t release.

https://twitter.com/dan_abramov/status/1402935426596364292

When “scary” bugs show up, it’s a sign that there’s a flaw in the model. The solution isn’t just to fix them one by one (although that’s helpful) but to fix the whole model. Once we fixed the model, a whole class of bugs disappeared. That gave us confidence to prepare the Alpha.

I’m sharing this because I think it’s important to be honest about the process. Some people will try to spin it as React being buggy. But that’s just what engineering looks like. We make mistakes, we fix them, we learn from them, we redesign, we verify, and we release.

Closing thought. Yes, React is complex inside. You know what else is complex? Node.js is complex. Postgres is complex. I don’t care because they make my code simpler. You know what’s not complex inside? Redux. With all respect, its internal simplicity doesn’t help the app code.

I’m hearing this sounds like a dig on Redux. I’m sorry, I should’ve worded this better. What I’m comparing is where does the complexity go. Redux doesn’t absorb the complexity. This doesn’t make it good or bad—it was never intended to do that.

https://twitter.com/dan_abramov/status/1403054633317191683

This is similar to how a Promise doesn’t absorb the complexity. Are Promises a useful pattern? Absolutely! Do they write your asynchronous logic for you? No, they let *you* write your asynchronous logic. Or someone else.