before any of this gets fixed it's necessary to understand why men in tech dress like shit. hacker culture and the web allowed people to rebel against the stifling business culture of the 90's. in software, especially foss, people were judged by their work, not their appearance.
the first wave of truly successful new-breed software companies were founded and succeeded by the hacker ethos. in contrast to 90's / 2000's microsoft, nobody gave a shit what you wore to work at early google as long as you did the job. the best flocked to these co's.
but then the hackers got rich and famous. as in every society since the dawn of time, people wanted to emulate the elites, and the elites dressed like shit. new founders wanted to 'look like' founders and 'fit in', so they too, dressed like shit.
the only things in the new testament about jesus’ life after age 12, before starting his ministry at age 30, are the verses mark 6:3 and matthew 13:55
in most english translations, they say jesus was (the son of) a carpenter, but this isn’t quite right
in the original greek, the word is τέκτων, tekton, a general term for artisans, craftsmen, and builders - indeed, this word has the same root as τεχνολογία - technologia, the study and discourse of craft
jesus spent his early adult life as a builder, making useful things
because there are no stories in the new testament about jesus’s life during this time, one might think it’s unimportant. this would be missing the point.
now that the dust has settled on the first round of the openai debacle, it’s time to start asking some questions
two of the board members who voted altman out, helen toner and tasha mccauley are deeply enmeshed in ‘effective altruism’
‘effective altruism’ as a movement has a lot wrong with its ideas, some of which i’ve documented before, but this isn’t about that - it’s about basic competence
the people who tell you over and over that they’re most concerned with the “long term future of humanity” have demonstrated over and over again that they don’t understand even the short term consequences of their actions
regardless of how exactly this shakes out - and it’s looking increasingly likely that this was a poorly executed power grab by mental defectives - it’s important to remember that ‘effective altruists’ have been consistently wrong about literally everything
they completely failed to predict the dominant paradigm for ai (llms), instead building castles out of sand on the idea of recursively self-improving agents and ideas like “instrumental convergence” and “orthogonality”, none of which apply because an llm has no agency
they have predicted catastrophe with every single model release, bringing down the perfectly harmless ‘galactica’ from meta - months later, the much more capable llama2 is out, with zero harm produced
i spent the weekend working to try to get more data about the question of whether or not gpt has world models
my results seem to suggest it doesn't, but i found some interesting things along the way. writeup below.
the entire question really kicked off for me when people were using gpt's performance on chess under certain conditions as strong evidence for the idea that it has world models
at the same time, it couldn't play well from random positions
the claim that gpt played chess well because it has a world model of chess is a very very strong claim, which entails that gpt knows the rules of chess, can consistently apply them, and knows good strategy
making retrieval practical for ai applications takes a lot more than repackaging a vector database built for semantic search, or adding vector search to an existing database.
in the first place, the core vector search and storage architecture for ai is fundamentally different from that made for semantic search
in semantic search / recommender systems, all the data in a single very large index is accessible to every user
additionally, the data is updated only rarely - most mutations are additions, with relatively few and infrequent updates and deletions