Self-supervised learning promises to leverage vast amounts of data, but existing methods remain slow and expensive. Introducing contrastive detection, a new objective that learns useful representations for many tasks, with up to 10x less computation: dpmd.ai/3jsOV9l 1/
Contrastive detection amplifies the learning signal from each image by carving it into pieces and learning from each simultaneously. This works particularly well when transferring to challenging tasks like detection, segmentation, and depth estimation. 2/
These results raise the possibility of learning algorithms that are more widely accessible, requiring neither human annotations nor vast amounts of computation.
Reinforcement learning typically trains & tests agents on the same game. New work shows how our team trains generally capable agents on huge game spaces, resulting in agents that generalise to held-out test games, & learn behaviours like experimentation dpmd.ai/open-ended-blog 1/
Rather than training on a limited number of tasks, our team defines a whole universe of tasks that can be procedurally generated, from simple object finding games to complex strategic games like Capture the Flag. 2/
By constructing a hierarchical learning process with an open-ended and iteratively refined objective, it was possible to train agents that never stop learning, and develop increasingly general behaviour across games. 3/
Mixed Integer Programming is an NP-hard optimisation problem arising in planning, logistics, resource allocation, etc.
Presenting a solver with neural heuristics that learns to adapt to the problem domain, outperforming SCIP on Google-scale MIPs: dpmd.ai/13349 (1/)
Practical applications often focus on finding good solutions fast rather than proving optimality. In follow-up work, Neural Neighborhood Selection finds better solutions even faster by learning heuristics for large neighborhood search: dpmd.ai/10201 (2/)
The neural solver learns even on single problem instances, improving the best known solutions to three open MIPLIB problems.
Yesterday we announced early collaborations using the #AlphaFold Protein Structure Database, which offers the most complete and accurate picture of the human proteome to date. So how is AlphaFold helping these organisations with their work…? 1/
The Drugs for Neglected Diseases initiative (@DNDi) has advanced their research into life-saving cures for diseases that disproportionately affect the poorer parts of the world. 2/
The @CEI_UoP is using #AlphaFold's predictions to help engineer faster enzymes for recycling some of our most polluting single-use plastics. 3/
Today with @emblebi, we're launching the #AlphaFold Protein Structure Database, which offers the most complete and accurate picture of the human proteome, doubling humanity’s accumulated knowledge of high-accuracy human protein structures - for free: dpmd.ai/alphafolddb 1/
We’re also sharing the proteomes of 20 other biologically-significant organisms, totalling over 350k structures. Soon we plan to expand to over 100 million, covering almost every sequenced protein known to science & the @uniprot reference database.
We’re excited to see how this will enable and accelerate research for scientists around the world. We've already seen promising signals from early collaborators using #AlphaFold in their own work, including @DNDi, @CEI_UoP, @UCSF & @CUBoulder: dpmd.ai/alphafold-case… 3/
Many models bake in domain knowledge to control how input data is processed. This means models must be redesigned to handle new types of data.
Introducing the Perceiver, an architecture that works on many kinds of data - in some cases all at once: dpmd.ai/perceiver (1/)
Like Transformers, Perceivers process inputs using attention. But unlike Transformers, they first map inputs to a small latent space where processing is cheap & doesn’t depend on the input size. This allows us to build deep networks even when using large inputs like images. (2/)
Perceivers can learn a different attention pattern for each type of data (shown for images and video), making it easy for them to adapt to new data and unexplored problems where researchers may not know what kinds of patterns they should be looking for. (3/)
In a new paper, our team tackles a fundamental AI problem: how can we simultaneously parse the world into objects and properties, while simultaneously inducing the rules explaining how objects change over time: dpmd.ai/3fmrxsn (1/)
Work by @LittleBimble with @pfau, @pushmeet, Matko Bosnjak, Lars Buesing, Kevin Ellis, and Marek Sergot. (2/)
This system combines the Apperception Engine with a binary neural network to learn a provably 100% accurate model of non-trivial environments (e.g. Sokoban) from noisy raw pixel data. (3/)