Yesterday at @archipelacon I presented "Grand Futures: the Interplay Between Science Fiction and Planning Really Far Ahead" - how the science and science fiction of megascale engineering and longtermism have interacted with each other. I ended up making a big diagram of who seems to have influenced who/what. Here is a current version (soon to be superseded) as PDF: tinyurl.com/megascalemap

I think exposure to generative AI makes many people aware of the Library of Babel in a way they are not ready for. It is one thing to know there is a near-infinite space of possibilities, it is another thing to play around and be led by the latent manifold into the Library. The thing that makes Borge's story so good is how it erodes meaning by infinity: everything is in there, yet nothing can be found in the noise. AI allows nearly anything to be found, but we become aware that it could have been different.

Prediction: people will say o3 scoring 25.2% on FrontierMath is nothing, "after all it is not perfect." Conveniently forgetting that these problems are ridiculously hard (did 87.7% on GPQA Diamond). And some will keep on talking about stochastic parrots... like parrots.

Some context:
x.com/tamaybes/statu…
epoch.ai/frontiermath/t…
arxiv.org/abs/2411.04872
arstechnica.com/information-te…

The recent discussion triggered by @tylercowen about what investment strategies AI doomers ought to exhibit (he thinks they should short the market, everybody else disagrees) got me thinking about my own investment approach.

https://x.com/jonatanpallesen/status/1850992956024803631

It is mostly based on uncertainty. Empirically we know people are pretty bad at long-term predictions.
I also suspect I am a bit of a naïf. But here goes:

Yesterday in our very friendly Biennale discussion professor Camil Ungureanu made a series of criticisms of how transhumanist visions could go wrong practically and morally. I despaired at responding to each of them, and then realized that there is a solution in going meta. (No, this was not a Gish gallop - all points were relevant issues worth talking about! But time was limited...)en.wikipedia.org/wiki/Gish_gall…

Here is a plot of putative "richest person in the world" wealth as a fraction of US GDP over 200 years. It is not a large fraction. Data from , GDP data from , US GDP estimated as average of decade, everything deflated.lovemoney.com/galleries/7453…
measuringworth.com/datasets/usgdp/

Two days ago I (like most people) had never heard of SVB. That we tend to discover systemically essential points of failure by them failing is deeply disturbing, especially if we want to make a more resilient world. In 1993 the world discovered the hard way that a speciality resin used to affix integrated circuits was mostly produced in a single chemical plant. That had an explosion destroying production and stores. apnews.com/article/8fe292…

Saying LLMs are just glorified autocomplete forgets that most human thought, action and speech is also glorified autocomplete.
Maybe the fraction that isn't autocomplete is the truly valuable part, but I suspect it is not a major fraction of everyday useful action. I am pretty serious about this: the basal ganglia action pattern generators look like they are selected by softmax influenced by cortical stimulus fit. They get made/updated by RL rather than backprop, but very much a stimulus-response chain. med.libretexts.org/Bookshelves/Ph…

Forwarding this on behalf of Nick Bostrom since he’s not active on Twitter. He wants to explain his views and publicly apologize for an old email that somebody might put out to damage him. nickbostrom.com/oldemail.pdf The context is a thread on a mailing list in 1995 talking about offensive communication styles where he did the classic freshman philosophy student trick of writing something deliberately offensive to make a point. It didn’t turn out well (it never does): he promptly apologized.

This is an important point. I *also* recall being super-skeptical about the JWST (I somewhat rudely asking John C. Mather if he could really get sigmas enough to handle the 344 single point failure - I owe him and the entire team an impressed apology). However...

https://twitter.com/surge_future/status/1612781521282256897

So, clearly sometimes good planning and design can make things that are exceptionally reliable. This seem to happen mostly when (1) underlying physics and technology well understood, (2) designs optimized for reliability, (3) error correction mechanisms in production and use.

#FridayPhysicsFun – “Launch him/it into the sun!” My husband was griping over breakfast about the expression: surely this is not a good way of disposing of anything?
tvtropes.org/pmwiki/pmwiki.… Is it actually possible to make something that will hit the “surface” of the sun?

There is a lot to unpack in this interview. I can also see a lot of lawyers cringing.
(A little thread on FTX, what I think EA did wrong, and systemic risk) vox.com/future-perfect… I have, as far as I know, not been directly affected by the whole FTX implosion (no direct funding). But it certainly made my world a bit more turbulent. One should never let a good crisis go to waste: this is a good time for EA to do some institutional reinvention.

This looks like the first step onto something interesting.

https://twitter.com/paperswithcode/status/1592546933679476736

Trying to get a review of lava redirection produces a paper trying to model a layer of lava. "In other cases it may be necessary to divert the flow of lava from one location to another, such as from a volcano to a settlement." Good to know it is just diverting things *away*.

#FridayPhysicsFun - @KarimJebari and me have a new paper that is putting the longterm into longtermism. It is about doing good for the environment a billion years hence. link.springer.com/article/10.100… The main point is that if we survive or leave automation in place, can extend the lifespan of the biosphere by perhaps a billion years or more. Ecocentric ethics places value on the Earth’s biosphere, so it should approve of this. But this thread is about the physics involved.

The more I learn writing prompts for AI-generated art, the more I note that it is a language based on clichés. This is not a criticism! Clichés, tropes, standard imagery are easy references to high-dimensional vectors in concept space. "evil stepmother", "greg rutkowski", "baroque" and "central perspective" link to very complex things, and we can invoke them easily and conventionally.

If this result holds up generally, merging networks trained on different parts of data looks feasible. Huge implications - privacy through federated learning, parallelization of learning, merging of em shards!

https://twitter.com/SamuelAinsworth/status/1569719494645526529

The basic idea is simple: you can permute hidden layer neurons, so there are actually far fewer internal models than it looks. Training gets to one, with linear mode connectivity. Can hence interpolate differently trained networks if one is careful.

Is there any good study of the average lifespans of villages, towns and cities? My impression is that they do not disappear very often, despite there being a fair number of examples of ghost towns (that are often lightly inhabited). en.wikipedia.org/wiki/List_of_g… This is linked to my bigger interest in what determines the lifespans of social structures and projects. Generally, constant risk over time seems to be the generic case for states, empires, species and companies; increasing risk only in software and individual organisms.

Nice thread about why democratizing AI makes more sense the weaker the AI is. But this only looks at "offensive" capabilities: clearly AI can also protect.

https://twitter.com/MariusHobbhahn/status/1559925158818750464

The real issue might not even be the offense-defense balance, but whether defense is reliable enough. A world where bad actors occasionally have great wins may be worse than one where they can often gain small. Some credit fraud ok, not everybody's accounts drained.

#FridayPhysicsFun - I wrote an answer on Physics Stack Exchange about moving the Earth outwards to compensate for solar brightening. physics.stackexchange.com/questions/7227… The first consideration of how to move the planet was made by Archimedes boast: "Give me a place to stand and with a lever I will move the whole world." physics.stackexchange.com/questions/4831…

#FridayPhysicsFun – Coolest fact I learned this week: under some conditions light-emitting diodes can be more than 100% efficient, and act as refrigerators. Normally energy conversion introduces losses: there is a production of entropy turning high-quality energy (e.g. mechanical motion, electricity) into disordered low-quality (e.g. heat), and turning low-quality into high-quality is less efficient.

There is an old Swedish rhyme about the cuckoo: "södergök är dödergök, västergök är bästergök, östergök är tröstergök och norrgök är sorggök."
Roughly: "south-cuckoo is death-cuckoo, west-cuckoo is best-cuckoo, east-cuckoo is consolation-cuckoo, north-cuckoo is grief-cuckoo". If you take the rhyme seriously each cuckoo hence radiates a kind of field in cells, like a cellphone tower for superstition. It presumably declines as 1/d^2 with the sound intensity.