Thread by @SarahJamieLewis

I received quite a few requests for "how to get started" building something like this. So I'm going to dump some history and resources in this thread.

This isn't difficult. You can do it to. Getting to the level where it is peaceful instead of frustrating might take a while.

https://twitter.com/SarahJamieLewis/status/1459296130387177472

I started programming simple real mode operating systems in my teens and it's fluctuated as a hobby for me over the course of the last 2 decades.

In between I've built hobby emulators, (dis)assemblers, fuzzers, compilers, and uncountable weird hybrids.

I'm going to assume you know at least one high level programming language. If you don't then you should learn one. Any one will do. People may tell you the choice matters, it doesn't.

The basic principles you will learn in one are transferable to others.

Awesome you know a language and you want to dive into operating systems/emulators/assemblers etc.

First, I recommend building a CHIP-8 emulator. The spec is simple. Covers many core concepts. And there is a fair catalogue of simple games & test programs.

en.wikipedia.org/wiki/CHIP-8

I did not start by building a CHIP-8 emulator but I have built a couple over the years. My latest one featured a taint tracker that displayed which parts of memory had been tainted by external input.

I recommend that as a fun side project.

Alternative: Find a CHIP-8 VM (there are probably thousands of open source ones) and write an program (+ assembler if you want to ramp the difficulty factor up a smidgen) for it.

Same principles learned. Different direction.

The CHIP-8 exposes you to all the necessary basics - reading and interpreting a specification, understanding registers and stacks, the difference between an assembly instruction, an instruction, and and opcode and, dealing within control flows.

Probably a good moment to point out that the large portion of lower level operating systems/emulator dev comes down to either reading specifications to engineer desired behaviour, or reverse engineering specifications from observed behaviour.

At this point you have a few directions you might like to go. Think of it more as a collection of winding national trails than a single path.

There is no single journey, nor one single destination.

At this point I'm going to list a bunch of projects that I did over the years. In a fairly random order. This is not a "do this, and then do that" list.

Your interests will determine how you respond to each project. Feel free to ignore them. Combine them. Bring in new ones etc.

The first time I seriously got back into operating systems after university was playing with xv6 - github.com/mit-pdos/xv6-p…

xv6 is a teaching operating system originally for x86 architectures but now maintained for RISC-V github.com/mit-pdos/xv6-r…

xv6 is very well documented (there is a whole, free, book about it) and you can start wherever you feel comfortable.

My main goal was to learn how the filesystem worked and interacted with the rest of the kernel - so I had a small side project extending it with file permissions.

Over the years I also built several hobby emulators for older systems (NES, Gamboy) - there are entire wikis and sites out there dedicated to documenting every last weird thing about these systems - and it is always a thrill watching games come alive.

A bit more recently my main learning focus was how to write device drivers. So with the help of all the new fun rust osdev resources (os.phil-opp.com) - I spun up a small kernel, and set about adding networking support:

https://twitter.com/SarahJamieLewis/status/1223128935300382720

I also recently wrote a small multi-hart bootloader and kernel for risc-v mostly in assembly because I wanted to get a better sense of the architectural differences (I also came to really like risc-v)

In between all of that I also wrote a custom multi-core safe memory allocator (rust osdev makes this pretty fun), process schedulers, and before I had to abandon my last apartment because it started raining indoors I did get a basic cut of userland processes done.

Venturing further away, If you are into retro games I also recommend diving into some of the more ridiculous speed running strategies that involve in-game stack corruption.

I spent a good vacation week zoning out and perfecting execution.

https://twitter.com/SarahJamieLewis/status/1374081492318515200

Once you've also spent enough time in the weeds of operating systems development you may also find that you enjoy writing fuzzers or other security-oriented tools. All of these activities are very much inter-related and build on and depend on each other.

I'm a very project-oriented person. I haven't detailed explicit skills here because if you are like me then you will learn the skills necessary to progress as you go.

And you will learn a lot of them.

Some words of advice: Pick a learning goal and guide your effort towards it e.g. if you want to learn about device drivers don't start by building a bootloader. In time you can get a good grip on all parts of the stack but it's important to make each project achievable.

Don't be afraid to throw away work and start again.

In the beginning it can take weeks to get even the simplest things working and you may feel attached to your collection of hacks and debugging attempts. You will build it faster the second time, and even faster the 20th time.

Sometimes you build on top of something and then discover it actually doesn't support your use case.
These are hilarious learning opportunities and you should be thankful for them.

https://twitter.com/SarahJamieLewis/status/1219503269023584257

Remember that this is supposed to be fun. Find a project that sounds interesting and dive into it. Sometimes you might lack all the skills to see it all the way through - that's fine, find another fun project focused on developing those skills.

If you do end up taking on some of these projects (or any related projects), please do let me know :)