Profile picture
Smerity @Smerity
, 14 tweets, 4 min read Read on Twitter
The @OpenAI team have done amazing work and really opened my eyes towards many of AI's possibilities, the vast majority positive. Instead of discussing my opinion of the nitty gritty, I'll try to stay at the highest levels of abstraction as to how this has impacted my thinking.
I want to wax poetic about the models (LSTM+PPO pushed far beyond what people likely thought possible, mirroring @Smerity et al + @GaborMelis et al in LSTM language modeling), the game (DotA as a complex test-bed), or the stupendous compute (180 years of gaming per day), but ... Blurry photo of Smerity in the foreground of the OpenAI 5v5 DotA tournament
... the real focus should be on how these bots are better team players than humans.
Humans pride themselves on this. Every darn Disney film focuses on this. Society is based on this.
Yet these "dumb" bots used self play and coarse objectives to learn better team work than humans.
For me this registered as a distinctly different beast as to what ML/AI is traditionally superhuman at. I was a heavy DotA player (started WC3 era and have played 830 hours of Dota 2) and these bot games are best described "restricted gameplay but with a sophisticated team plot".
I honestly believe the "restricted gameplay" part is irrelevant as I see no major obstacles (other than immense engineering which @OpenAI are well equipped) that would prevent near bot perfection. It's the team work angle that is distinctly unique and the new addition for me.
The bots knew when to take the fall to set their team up, when to waive their local reward for the team's global reward, and had no sense of a hero complex (I'm going to attack as I have a time limited Double Damage rune even though my team isn't ready!).
Humans can't learn this properly. At the amateur level, amateurs can't rely on their team mates. At the professional level, there are only so many hours in a day to perfect co-operation and players suffer from rage quits / going full tilt / ... even at the highest tier of sports.
This is an iterated prisoner's dilemma but humans will never spend enough time in this jail to work out how to optimally play the game through team work and collaboration. Conversely, we can happily lock our friendly @OpenAI Five team into hundreds of years of gameplay per day.
Why is this exciting for me? This has implications far beyond DotA. These bots never see traditional human strategies, they only play the game as the rules and objectives dictate it should be played. If there's a positive sum way to play a "human" zero sum game, it'll find it.
I would argue humans have forced themselves into a corner in many places due to our limits - slowly evolving rules that have high overheard to set up and poor interpretability. Ex: Loop holes are ~when laws aren't updated to account for changed context or adversarial intentions.
These bots weren't perfect at most of the skills humans optimize for. In a million ways they appear to have suboptimal behaviour to humans (getting gold (last hitting, jungling, farming, ...), strategy (creep blocking, pushing towers, ...), ...) - but that really didn't matter.
The bots, working as a team, saw the objective pipeline and optimized for the parts that mattered. They don't get all the gold+items if it means their team members are underpowered. They don't get the ultra-quadruple-mega-kill for glory when it'll make your team lose the game.
We could see a future where many of society's patchwork forced complexities fall away as these automated systems reveal the premature optimizations we've tricked ourselves into and the obvious blindspots we've missed that until now are only visible with hindsight.
We as humans aren't smart enough to see through that fog of complexity and complex interaction - but the systems we write might be. They might help us achieve the objectives which we've been lossily and haphazardly walking towards for hundreds of years.
Missing some Tweet in this thread?
You can try to force a refresh.

Like this thread? Get email updates or save it to PDF!

Subscribe to Smerity
Profile picture

Get real-time email alerts when new unrolls are available from this author!

This content may be removed anytime!

Twitter may remove this content at anytime, convert it as a PDF, save and print for later use!

Try unrolling a thread yourself!

how to unroll video

1) Follow Thread Reader App on Twitter so you can easily mention us!

2) Go to a Twitter thread (series of Tweets by the same owner) and mention us with a keyword "unroll" @threadreaderapp unroll

You can practice here first or read more on our help page!

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just three indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member and get exclusive features!

Premium member ($3.00/month or $30.00/year)

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!