Today we're releasing Autopilot - a Python framework for performing complex behavioral experiments by distributing them over swarms of Raspberry Pis. Let me tell you how we've rewritten what you should consider possible in your experiments /1
site: auto-pi-lot.com
site: auto-pi-lot.com
We think that to make transformative progress in understanding the brain, we need to study complex, naturalistic behavior. We know we're not alone there, but the technical demands of these experiments start to sound almost comical: eg. how hard would it be for you to... /2
measure pupil dilation, respiration, and running speed; track the position of a dozen points in 3d, record hundreds of channels of ephys while an animal performs some complex task in a virtual reality space that you render in real time? /3
Experiments like these are becoming increasingly common, but currently they rely almost exclusively on custom-built, single-purpose hardware and software that, if they're released at all, are extremely difficult to recreate-- so most results in neuroscience are unreplicable. /4
The open-source tools that exist are not suited to handle large numbers of custom hardware components, and instead usually depend on limited ecosystems of expensive, custom-built components. There was effectively no good way to do complex, replicable behavioral experiments. /5
We needed a system that a) was flexible enough to accommodate any hardware and any task design, b) fast enough to record from and manipulate that hardware at the millisecond timescales of the brain, and c) prioritized doing good, reproducible science. So we made one. /6
We did this by radically embracing parallelism: Autopilot lets you combine arbitrary numbers of hardware components by allowing you to distribute them across threads, processors, and even multiple computers. /7 
We use the surprisingly capable Raspberry Pi 4 as a platform. They run a full Linux distribution, and let us marry a high-level, user-friendly language like Python with low-level, high-performance code. Fast and flexible, the best of both worlds. /8
As a result, Autopilot has an order of magnitude better performance and measurement precision compared to existing systems, and also makes use of the wide array of inexpensive components and community support available to the Raspberry Pi. /9
Because we use Python throughout the system, we don't need to rely on task definition formalisms that limit experimental flexibility like finite-state machines -- anything you write in Python can be a task. /10
As an example, if you wanted to make a maze task where an animal has to follow a specific path to reach a reward, you could sort of fake it with a finite-state machine but it would be monstrously complicated (see the state diagram). The basic logic is 22 lines in Autopilot /11
Most behavioral tasks rely on a series of shaping steps to familiarize the animal with the task, but we noticed that most behavioral systems don't allow that. Autopilot lets you chain multiple tasks together into training protocols, eliminating ad-hoc shaping schemes /12
To make experiments reproducible, we have split task logic from the weird idiosyncracies of your particular experimental setup and the custom parameters and stimuli that make your task unique. Anyone can recreate your experiment with a single file. /13
Other behavior systems produce data that requires extensive cleaning and annotation to analyze and publish it. Autopilot standardizes the data for each task with human-readable tags, so your data is publishable at the time of acquisition. /14
Doing behavioral experiments is often an iterative process -- parameters like reward volume and timing need to be refined, but most other systems don't document these changes. Experiments also often rely on 'auxiliary' logic like bias correction, but this is rarely reported. /15
These "minor details" really matter, and essential to a successful replication. Rather than leaving it to a methods section, Autopilot automatically documents all parameter changes and tasks an animal ever experiences, eliminating ambiguity in reporting. /16
Autopilot is an order of magnitude less expensive to implement. Grant funding is extremely inequitable, so given the extreme expense of hardware and development time, most labs outside of elite institutions can't afford to do cutting-edge behavioral research. /17
Expense is also a bottleneck for high-powered neuroscience. Recording high numbers of cells from a small number of animals is a nightmare for statistical rigor and will flood the literature with false positives. Ephys and 2p are expensive, but training time really keeps n low./18
For day-to-day use, behavior systems need a GUI for controlling task operation and visualizing results. We developed one that lets you manage many independent behavior boxes, here's our system training 10 mice in parallel /19
We let you customize plots for your task by mapping its data to graphical elements, mimicking the grammer of graphics and ggplot2. Here's an example of two mappings and the plots they produce /20
The real transformative element of Autopilot is the ability to control what we call "behavioral topology" - or the distribution of labor across computers. You can duplicate tasks across many pis in parallel, use many pis to perform a single task, and anything in between /21
Autopilot lets you unify really complex experiments that take place over multiple timescales and use multiple sets of equipment. Experiments that combine longterm, naturalistic observations of animal behavior with multiple tasks are the way of the future and this is the engine/22
We've tested it extensively, and demonstrate an order of magnitude lower latency (end-to-end, trigger to sound onset) than a comparable system. The networking protocol at its core is fast enough to implement realtime data processing into closed-loop experiments. /23
We implemented a split-pi task in a visual change-detection task as a proof of concept. Message processing was faster over the network than a comparable system directly connected over serial, and triggered stimuli with a 2-3 frame delay from a separate pi. /24
We've designed Autopilot in such a way that users are incentivized to write reusable tasks for a central, shared library of tasks (like psyarxiv.com/upynr/ calls for). We hope this eventually dramatically reduces the number of times the wheel is reinvented /25
We've written 3 levels of documentation: this paper is the design document, the user guide tells you how to use it, and the complete, bottom-level API documentation tells you how to hack it. We want users to know exactly how everything in the system works and why. /26
We're working on the rest of our web infrastructure now, including a hardware page that can take markdown descriptions and render them into a library of schematics and parts list so anyone can contribute their designs. The prototype is here: auto-pi-lot.com/hardware /27
We have a lot planned for the future. One example: we want to tightly integrate with tools like @DeepLabCut @OpenEphys and MoSeq to make a unified platform for naturalistic behavior. /28
@DeepLabCut @OpenEphys To develop the system, I'm looking for collaborators. I want to do a behavior tour next summer, come to your lab for a few weeks, implement your behavior, and expand the scope of what's possible for everyone /29
In the meantime I'm looking forward to the deluge of github issues. Let me know what doesn't work, what needs to be documented further -- I want to make it work for you. This has been a huge labor of love for me and I'm very excited to see how people use it <3 /end
I couldn't be at #SfN19 this year, but folks there might want to see what we cooked up.
@OpenBehavior @Labrigger @AcademicChatter @UOPsych @IONatUO
@OpenBehavior @Labrigger @AcademicChatter @UOPsych @IONatUO
@OpenBehavior @Labrigger @AcademicChatter @UOPsych @IONatUO @wehrlab @Datta_Lab here's a tool to do the types of experiments described in your recent piece in Neuron :)
this is supposed to be this image, whups
