Thread by @WiringTheBrain

𝗧𝗵𝗲 𝗚𝗲𝗻𝗼𝗺𝗶𝗰 𝗖𝗼𝗱𝗲 - 𝘁𝗵𝗲 𝗴𝗲𝗻𝗼𝗺𝗲 𝗶𝗻𝘀𝘁𝗮𝗻𝘁𝗶𝗮𝘁𝗲𝘀 𝗮 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝘃𝗲 𝗺𝗼𝗱𝗲𝗹 𝗼𝗳 𝘁𝗵𝗲 𝗼𝗿𝗴𝗮𝗻𝗶𝘀𝗺 🧬
very excited to share this new preprint from me and Nick Cheney 😀🧵 arxiv.org/abs/2407.15908-

In which we consider how best to conceptualise the role of the genome in specifying the form of the organism. In other words, how it is that cats have kittens and dogs have puppies.

Clearly, the form of the organism that emerges depends on the genetic material in the fertilised egg (see Dolly, below), but how should we think about this relationship?

There are lots of metaphors that people use for the genome: a blueprint, a program, a recipe, or just a “resource” that the cell or organism draws on. But none of these is really apt and all are rather vague and unformalisable.

The most popular ones – a blueprint or a program – give a picture of the encoding of form that is too deterministic, direct, decomposable, and isomorphic.

That is, they imply that separate bits of the genome somehow specify or directly encode separate bits of the organism or distinct processes of development. We've known for a long time that this is not the case.

In this paper, we draw inspiration from machine learning and neuroscience to propose a different idea: that the genome instantiates a generative model of the organism, encoded as a compressed representation in a space of latent variables. arxiv.org/abs/2407.15908

We propose a loose analogy with variational autoencoders, which learn from training data a compressed representation of some kinds of entities (say images of horses), and which also learn how to decode this representation to generate a novel token of this class.

That sounds a lot like the job description of the genome, where evolution acts as the encoder, and development acts as the decoder.

Importantly, the encoding of different aspects of organismal form in this latent space is highly indirect, distributed, and largely non-decomposable.

This fits with emerging (or re-emerging) work in developmental biology which models gene regulatory networks as dynamical systems, where the collective interactions constrain the possibility space and channel developmental trajectories towards various attractor states.

In particular, it draws a direct parallel between Waddington’s visual metaphor – the epigenetic landscape – and the kinds of energy landscapes generated by some machine learning systems.

A key element of this model is what the genome does NOT encode (because it doesn’t have to). The genome doesn’t have to actively direct every developmental process – it just has to *constrain* the self-organising biophysical processes of morphogenesis.

And of course the genome does nothing by itself. DNA is incredibly inert – that’s its whole shtick, really: to be a stable store of information. It’s the cells that are doing the work.

This generative model has a number of important properties:
1. Compression through a bottleneck layer.
2. Encoding in a latent variable space.
3. Abstract, indirect representations.
4. Intrinsic variability of outputs.
5. Robustness.
6. Evolvability.

The robustness and evolvability are key here and these properties derive from the compressed, distributed, collective encoding

But this leads to a quandary when thinking about effects of genetic *variation* on different organismal phenotypes...

If most traits are highly polygenic (affected by many genetic variants) and most variants are highly pleiotropic (affecting many traits) then how could traits ever change independently? Why isn't there a kind of genetic gridlock?

Here, we may get some useful lessons from machine learning and neuroscience about the emergence of disentangled representations

Machine learning models, like VAEs, can be trained so as to generate independent, disentangled representations of separate features of the training data (like facial pose, age, gender, expression, etc.).

Despite the fact that many individual elements of the model carry some kind of information about many traits, all tangled up, distinct *sets* of elements can encode information about distinct traits in orthogonal subspaces

This is similar to the idea of separate low-dimensional subspaces in neural manifold encodings:

In the genome, we think the same kind of emergent modularity can create orthogonal representations of different traits in latent variable space, explaining how they can be independently selected for

Lots more in the paper, of course! Including thoughts on how this kind of model could be formalised in indirect encodings of artificial life forms (ALife)… arxiv.org/abs/2407.15908

And, sorry, I neglected to tag Nick's twitter account: @CheneyLab above! (messing everything up in this thread... like a rookie!)