Thread by @ridiculous_fish

1/12 Let's talk about posix_spawn, and when to use it.

https://twitter.com/kolyshkin/status/1232049269219057665

2/ posix_spawn was reaction to the high cost of fork/exec. It traded off flexibility, and in return, it bought efficient kernel implementations (macOS, FreeBSD, Solaris?), but also permitted user-space implementations built atop fork/exec (Linux).

3/ High cost: fork (in principle) means copying the whole address space. Linux is on-brand with its optimistic COW and OOM killer. It's very fast (until you explode, acceptable risk).

4/ Linux does posix_spawn in user space, on top of fork - but sometimes vfork (an even faster, more dangerous fork). The vfork usage is what can make posix_spawn faster on Linux.

5/ Maybe your fork is not so fast, because your kernel cares about delivering on the memory it promises, or you sweat port rights, or don't have a MMU, etc. Here you can do a kernel-implemented posix_spawn, like xnu and BSD.

6/ Flexibility: You fork, then just write code. Set up the fds and tty etc, then exec. The same system calls you use everyday play double-duty to prepare the child proc. Very clever!

7/ spawn has no such luxury: everything must be wrangled into a declarative API . To reach feature parity, spawn "would need to provide a programming language as a parameter." (That quote is in POSIX!)

8/ What's missing? One hole is the syscall you've never heard of: tcsetpgrp(), which hands-off tty ownership.

The "correct" usage with fork is a benign race, where both the parent (tty donator) and child (tty inheritor) request that the child own the tty.

9/ There is no "correct" tcsetpgrp usage with posix_spawn: no way to coax the child into claiming the tty. This means, when job control is on, you may start a process which immediately stops (SIGTTIN) or otherwise. Here's ksh getting busted: mail-archive.com/ast-developers…

10/ Another hole: error messages. You can call posix_spawn with many file actions: chdir, open, etc. Any of these may fail, but all you get back is a single errno. You don't know *which* file failed to open.

11/ But perf! On my Mac, #fishshell runs /bin/true in 0.87ms, compared to ~1.3ms for bash and zsh. (Amortized 1k runs)

On my Linux box, fish is at 0.22ms, zsh at 0.3, and bash is at 1.4! This is vfork at work.

Run `set -g fish_use_posix_spawn 0`, and fish joins the slow crowd.

12/ So should you use posix_spawn instead of fork? Yes, if you can and want the perf: it's significantly better on both Mac and Linux. But be mindful of the holes!