,
7 tweets,
2 min read
Read on Twitter
1/ #Django will use watchman to do autoreload if it's installed since 2.2. The default reloader runs stat on all the files in your project once per second, whereas watchman will use os file system events, which is much more efficient.
2/ I never cared about this until I spent an afternoon figuring out why Docker for Mac's hyperkit process sits on 120% - 150% cpu use while running my mid sized django project in a container.
3/ i used 'docker stats' to deduce which container, then 'top' and 'ps auxf' to work out which python process was the problem. Then did a thread dump of the django server using faulthandler.dump_traceback() triggered from an API call, and then 'docker log'
4/ I was expecting some crazy background thread was doing something weird, but nope, just the autoreload loop, launched from StatReloader. Reading the source revealed the WatchmanReloader.
5/ Installing watchman into the container solved the issue. It didn't initially work because of some old code accidentally adding / to sys.path, and watchman refused to monitor /. I traced through watchman using pdb to work out why. This took a while.
6/ aside: knowing #pdb is a great skill. It's available inside containers and servers where you couldn't easily otherwise use a debugger. It'll probably be in python forever. You can exec arbitrary python inside your process.
7/ anyway, all this is a non issue unless you're using #docker for mac, where the host i/o is just so so so very slow and has been for years. I wish there was an alternative. But happy days - after I did this, hyperkit only uses 20% cpu for no reason. Yay!
