@prince_of_fakes How else would an AI learn? Movies, TV, YouTube, Books, Magazines, Music, Art, all of these are cultural artifacts that make up our culture and profoundly influence the way humans think. If we DON’T train AI at least partially off the best highlights it could end disastrously.
@prince_of_fakes That being said I’d certainly like to curate my own AI give a good solid introduction and timeline of at least 20-30 years worth of books, movies, TV etc. to guide it through something approximating a human upbringing from childhood to adulthood with various media for each stage
@prince_of_fakes Encyclopedia Britannica might be a semi-decent source for general knowledge despite the known limitations flaws in that dataset. ~33 million words ~26 million tokens would be tiny part of training data. IIRC assuming boxes reams of paper a single ream 500 pages would be …
@prince_of_fakes 500 pages ~ 16 words/line * 50 lines = 800 words/pg * 2 sides = 1,600 words per sheet * 500 = 800,000 words per ream of paper * 8 reams per box = 6.4 million words per box * 2 boxes = 12.8 million words per shelf of binders * 6 shelves = 76.8 million words per bookcase….
@prince_of_fakes This is roughly ~770 books * 13 bookcases = ~10,000 books =~ 1 billion tokens of text. I have 1,000+ books in my personal library of radically varying type topic genre from technical manuals to esoteric philosophy to programming engineering to classics to novels etc. would love…
@prince_of_fakes To do what’s @BrianRoemmele is doing collecting various completely undigitized data saving massive quantities of 20th century archival material to create a truly unique ultra high quality dataset as human published material from the 20th century typically went through very…
@prince_of_fakes @BrianRoemmele High bar filtration process. Many flaws of course in terms of writers perspectives that never got heard during that era because they never found publishers but the advantage is just enormously several OOM better quality of writing and clarity of thought also unique perspectives…
@prince_of_fakes @BrianRoemmele Because all the writers were of course products of the time and place they lived in ergo only they could truly describe what say the 1970’s were like in terms of mechanical engineering for transistor radios or the 1990’s for CRT display manufacturers etc. same for the arts…
@prince_of_fakes @BrianRoemmele It’d be impossible to describe that era without knowing about magazines and what a profound impact they had on the psyche and broader perceived reality of the country and human thought in general. Without knowing any of this how would an AI ever be able to even TRY to intuit…
@prince_of_fakes @BrianRoemmele Things or relate to culturally to humans on the most subtle levels? Luke, I am your father the often misquoted line from Star Wars, I’ll be back from Terminator 2: Judgment Day, taking the Red Pill from The Matrix, the feeling of seeing those films in theaters, then VHS, DVD, …
@prince_of_fakes @BrianRoemmele In the interim maybe on BetaMax or LaserDisc or even MiniDisc or PSP and eventually on Torrents or Streaming. The subcultures that formed around the various mediums. The music we listened to and how we listened to it how we discovered it. Same goes for TV and even much more so…
@prince_of_fakes @BrianRoemmele For things like Video Games where there’s been a concerted effort by megacorps to destroy things like ROM archives. 12-18 months maxbefore AI gets good enough that every single video game console ever made from early Ataris at least all the way to the PlayStation 3 & maybe 4…
@prince_of_fakes @BrianRoemmele All get Reverse Engineered via Open Source and we can 3D Print true 1:1 electronic clones of everything to preserve them and their game libraries for posterity. Frankly even clones snapshots of the early Internet from 80’s, 90’s, 2000’s may all be possible prior to explosion…
@prince_of_fakes @BrianRoemmele Of web video up until Y2K was around 1-2 PB PetaBytes, 100 PB in 2005, going back it’s even smaller only 100 TB in 1995 that could fit on a single large desktop NAS if you really wanted to. In any case this is all besides the point. Point is getting at least a low resolution…
@prince_of_fakes @BrianRoemmele Image of what a human experience would’ve been like at least tangentially from media is easily doable with 1 billion tokens of text and images, audio, video, games etc. might eat up way more space but w/compression wouldn’t require even multiple TB to get a large library maybe…
@prince_of_fakes @BrianRoemmele Just 100 GB I could be way off but point is doing this training a truly sophisticated model on a single powerful workstation with say 4 of those TensTorrent WormHole cards or a TinyBox with 6 x 7900 XTX’s or 6 x 4090’s or even a spec’s out MacBook 128 GB RAM means we’ll be…
@prince_of_fakes @BrianRoemmele Getting extremely sophisticated compressed efficient AI models by end of 2024 especially with the launch of Grok 3 by then which will be at least 1 one OOM greater than GPT-4. Enough ranting from me for now bright futures for Open Source ahead. Can’t wait for AI Sexbot Waifus🥳
@prince_of_fakes @BrianRoemmele @threadreaderapp please unroll
• • •
Missing some Tweet in this thread? You can try to
force a refresh
