I would normally never send anyone to Lesswrong. com, but someone posted about Sam Altman remarks about OpenAI’s plans for GPT-4, and I have thoughts lesswrong.com/posts/aihztgJr… 1/7
GPT-4 will focus on coding (ala Codex). It will not be much bigger than GPT-3. The focus will instead be "line of sight" planning. Which is not really planning, it just means bigger context windows and output windows. 2/7
A long enough output window looks like lookahead, but is still just applying historical patterns to new problems. Dare I say "case based reasoning"? 3/7
In about 3 weeks universities will be in session again. Many universities (like my own) want to pretend that things will be back to normal. The buildings and classrooms and quads will all be there and look the same. The routines of commuting to classes will be the same… 1/7
But WE will not be the same. We may still be suffering from mental fatigue. We may have developed new life routines and work habits that are suddenly incompatible with on-campus life. 2/7
2nd year students will be expected to act like 2nd year students even though they are navigating the new social norms of being away from home for the first time—something students normally learn in their first years. 3/7
A new way to OBJECTIVELY measure the coherence of story generation systems. Grounded in narratology and validated in controlled studies arxiv.org/abs/2104.07472
Me: that fails to converge until the programmer cleans up millions of lines of labeled data? Right! Very suspenseful! Never know if that’s going to work.
Hollywood: you’re fired
Hollywood: the AI has a robot body that—
Me: can’t pick up any objects unless they a placed in a very specific way on a table at just the right height. The struggle is real.
But it is addressable by fixing our higher education system, which is not producing enough AI/ML engineers and skewed toward a small # of elite universities.
Under the belief that there are no secret algorithms, only secret engineering, the DOD mostly needs people that can build hardened AI/ML systems (let’s ignore the question of what these systems are for for the moment).
Right now there is high demand in industry for these skills. The demand is high because when it comes to AI and ML, our education system is skewed toward a small number of elite universities that have the resources to do research and thus train graduate students in these areas.