,
24 tweets,
9 min read
Read on Twitter
Today at 3am Tokyo time, @DeepMindAI presented one new member of the Alpha family: #AlphaStar, playing StarCraft 2.
#AlphaStar beat two SC2 pro gamers, @LiquidTLO and @Liquid_MaNa, and this, Ladies and Gentlemen, is a real achievement. So can we say that SC2 is beaten, like Go of Chess? Definitely no.
#AlphaStar has some critical limitations to surpass before being proclaimed as the AI beating the game. Here is for me a list of the main 4 barriers and things to improve.
1. The most important one IMO: Having only ONE agent playing the game, able to apply different strategies and beating a pro gamer in a 'best of 5' or 'best of 7'. Having 5 different agents playing just one game each against a pro proves nothing.
2. Having a bot playing on several maps, not just the 2-player Catalyst LE map. It clearly makes learning and decision-making easier. Also, a 4-player map or more would be great: it forces the AI to scout where is its opponent, and to adapt its strategy in function.
3. Protoss vs Protoss match-up only is quite limited. I think it is fair to specialize a bot in one race (although a random bot would be very cool!), but a bot must also be able to beat pro Zerg and pro Terran to "beat the game".
4. #AlphaStar is very good at micro for sure, but is weak at macro for me - so I disagree with @DeepMindAI claiming it has a strong macro in their blog post deepmind.com/blog/alphastar…
Each agent applies one strategy only, and seems enable to adapt when they are perturbed.
Each agent applies one strategy only, and seems enable to adapt when they are perturbed.
We can clearly see that on the exhibition match against @Liquid_MaNa. The bot let MaNa having an observer spying it the whole time (and I won't be surprised if #AlphaStar can actually 'see' there is an invisible unit between its main and its natural)
And we see #AlphaStar did know how to react with MaNa's immortal drops. Like commentators said: a single air unit and it was over. But no, #AlphaStar does not realized an air unit was necessary, or at least just letting some units in the base to "welcome" the next drop.
I also have other comments: David Silver suggested that #AlphaStar saturating bases with workers gathering mineral could be something the SC2 community would investigate to bring new strategies, a bit like AlphaGo's Move 37 during Game 2 against Lee Sedol.
I am skeptical about that. Unlike a Go mid-game move, optimizing mineral gathering is fairly easy to study and has been intensively studied indeed, see liquipedia.net/starcraft2/Min… for instance.
I read that after his games against #AlphaStar, @Liquid_MaNa saturated its main in an #AlphaStar manner during WCS Winter qualifications. He has been qualified indeed, but nothing indicates it is thanks to this.
As always, there was the usual questions and debate about the bot's APM. IMO, wondering if the bot has limited APM or not is to move away from real questions. APM is not interesting. Bot APM and Human APM are not comparable.
In SC1 for instance, a bot must handle all workers "by hand" : select one worker, click an a mineral patch, select the worker again, click on the base. Restart the same process all game long. For all workers. And these actions are counted in the bot's APM.
I don't think Bot APM in SC2 are counted this way (although I am not sure). But limiting a bot's APM to have a more "human-like" condition is nonsence to me.
Is the goal to have a bot playing like a Human being? No, not here at least. The goal is to have an AI mastering a task, and here the task is playing a game.
A bot will NEVER play in the same condition as a flesh-and-bone player, even if you limit its APM, camera view and reaction time. It has no keyboards, no mouses, no screens, no stress, no feelings, no fatigue, etc.
The goal is to have an agent exploiting at best machine capacities to perform a task. Limiting a bot's APM is like willing to remove Human's intuition to have a fair game. Human vs machine games are not fair by essence. You must deal with it.
A final word: #AlphaStar has been trained on SC2 v4.6.2. I guess @DeepMindAI must restart the whole 200-year training to have #AlphaStar playing correctly on other SC2 versions, as well as AlphaGo must be retrained if you change komi.
This is a problem with current reinforcement learning techniques: slightly change the rules and you probably have to retrain the whole thing. And it learns slowly. Yes, it was a one-week training, but representing 200 years of SC2 play.
This is I think the main challenge with reinforcement learning methods for the next few years: having models robust to change and needing drastically fewer training examples would be the true game changer.
unroll dude. @threadreaderapp
-link to rebuild my original thread-
