I'm at the @OpenAI Five Benchmark match today. I'll try to live tweet though my internet is spotty 😅 
The first match has already started and @OpenAI's bots already have a decisive advantage with first blood. This gives you extra money and (against humans) can result in tilting ;)
To give an idea of the spectacle, there are bars, custom drinks (based on @DOTA2 items), and swag to cheer your team (bot or human) onward!
The stage is also perfect ^_^
At ten minutes in, OAI5 (@OpenAI Five) are already in the enemy base. This is Not Good for the humans :) They've taken a barracks which means their creeps (dumb soldiers) are far stronger for a single lane. For the humans to push that lane back is now far more work.
What started as a 2:1 kill advantage for OAI5 has always hit a 4:1 kill advantage. They're now attack mid after destroying top (reminder: there are only three lanes). At this stage not a single building of OAI5 has been destroyed.
Humans are at least still enjoying themselves as in chat they're complaining that they think OAI5 may be bots. "REPORTED" is a fairly common cry in amateur scenes when a skilled player kills you ;)
OAI5 also had some trash talking in announcing "We believe we have a 99% win probability at this stage". If a core component of playing @DotA2 is trash talking, these bots may have it already ^_^
OAI5 are now walking on the last lane. Team kill for OAI5. Humans surrender. A game ended in 13 minutes is fast! O_o Key strats for OAI5 was perfectly using stuns and disables, especially early. Humans got only a handful of kills. I hope the pro team are ready w their A game!
More a smerity.com/articles/2018/… but tldr, any time you pick up a well-shuffled deck of cards, you are almost certainly holding an arrangement of cards that has never before existed and might not exist again.
It's even more complex in @DOTA2 given the full pool of heroes is >100!
It's even more complex in @DOTA2 given the full pool of heroes is >100!
As an example, invisibility wasn't allowed previously - not only is it now allowed but one of the 18 allowed heroes is Riki. Humans hate Riki as a good Riki player can decimate your team, appearing and disappearing out of smoke, invisible most of the game.
dota2.gamepedia.com/Riki
dota2.gamepedia.com/Riki
Think of the monster under your bed appearing and backstabbing you. Cloak and Dagger. That's the literal name of his ability. Countering him generally involves using special items to force his visibility. Adding him to any line-up will change the required strategy substantially!
Whilst waiting I will take this opportunity to mention how magical San Francisco can be, enabling and connecting the people and tech behind today, but also acknowledge the unsustainable and unjust wealth gap that exists in this city. This was the end of the line to enter today :(
The match against the professionals is about to start! This human team is truly 99.95 percentile and I've followed them for years well before AI entered the @DOTA2 scene. If this team can't form a good defense, we'll likely only ever see a good OAI5 fight when it's OAI5 v OAI5!
Hero selection now: OAI5 has Gryo (aggressive early game) and Sniper (ranged, harasser) already. Gryo destroyed the humans in the early match as it can steamroll if you don't shut it down early. Sniper is generally not used in pro play.
Humans have selected Earthshaker (great for controlling fight - can block enemies from walking with giant fissures), Crystal Maiden ("glass cannon" - can do good damage but very vulnerable - good at freezing an enemy in place), and Razor (steals damage by an electric link).
An interesting note - OAI5 didn't have to select heroes before. This is referred to as drafting. Heroes are selected one by one, your team, enemy team, etc, until a full draft. This is a make or break part of the game as you can counter the enemy team's picks decisively.
OAI5 trash talked saying they think their probability of winning is 80-90-95%! ;) Humans used smoke - temporary group invisibility - to get into the enemy's territory - but they didn't end up fighting. The bots are far more grouped than the humans so far.
OAI5 got a second kill against the humans at top and then teleported a hero to bottom. The hero at bottom then helped get a third kill against the humans. "Good rotation" would be the phrase. At top, humans show aggression, _nearly_ kill the Gryocopter, but he escapes barely.
OAI5 trash talk: 98% chance of winning. Evil bot! At least the humans aren't getting steamrolled so far! Go humans! A 3-0 lead is not impossible to recover from. Wait ... 4-0 ... and the bots are using smoke (group invisibility) the second after they kill the human.
The bots just re-evaluated their win percentage - 99%. Kills are 5-0 in favour of OAI5. Humans finally got a kill! Blitz on Shadow Fiend took first (human) kill! Unfortunately OAI5 got another kill - a Gyro and Earthshaker used two successive stuns (teamwork!) to kill on bot :(
Humans and bots had a massive engagement at bottom. OAI5 came out on top but at least blood was spilled on both sides. 9-3. The humans are impressively competent against OAI5 so far but the bots are slowly building their advantage. Game is now paused due to lag issues for humans.
Game is back! The bots are now pushing bottom to get the first tower on the path to the human base. OAI5 took down 2 humans - one before the fight even started (surprise!) - the humans got one kill back! - but their tower is dead. Another human is down. Not good for humans. 13-4.
It's ten minutes in and the pro humans have only lost one tower. This is far better than the earlier humans - but not enough. 15-4 in favour of OAI5 and they're now focusing on mid. The humans are losing towers and can't defend - and if they do they die.
OAI5 trapped 3 humans inbetween trees at mid - and killed them all. Eek. Blitz decided to stay the hell away from OAI5 and push bottom. The bots rotated down and took him out. Oh geez. 19-4 in favour of OAI5. 22-4. Humans rotated down to support Blitz and paid the price.
At this stage all the Tier 1 towers on the human side are dead. The bots are systematically working their way through the defenses. The Tier 2 tower at bottom is now also gone. Reminder: Tier 1, Tier 2, Tier 3, Tier 4, Ancient to win. So far the humans have take no towers. 24-4.
OAI5 lose a hero! Most of the human team ganked (trapped) an OAI5 bot in the river. Unfortunately the bots took the Tier 2 top tower in that time ... Humans rotate to defend top and have a decisive(ish) victory. 27-7 - humans traded 1:1 for kills. A low level win for humans!
OAI5 are pushing bottom lane, moving into the base. Human falls immediately. This is bad. The pro team at 17 minutes are where the high level amateur team were at in ~8 minutes. OAI5 pushed hard (30-8 in their advantage) and then backed away when the humans defended well.
The bots attack at bottom again and team wipe the humans. Damn. That's ... insane O_o The humans already used their buybacks (instant respawn via money) so now they need to wait. The Tier 3 and two barracks at bottom are gone. Reminder: none of the OAI5's buildings have fallen.
As the barracks at bottom are dead, OAI5's creeps in bottom lane are now stronger. Humans will need to spend time and effort pushing bottom lane out. Bots rotate to mid, take Tier 2 tower, take Tier 3, ... humans announce gg at ~20 minutes. Time for cheese strats from humans?
Humans take a second to huddle regarding strategy whilst the OAI5 computer happily sits glowing next to them.
New game starting! They're drafting heroes now. The win probability can show us which heroes OAI5 are afraid of in real time. As I guessed earlier in this tweetstorm, OAI5 are "scared" of Riki (76% win guesstimate vs higher for others). Humans chose Riki. 🥳🎈🗯️
Reminder: Riki is invisible most of the time and destroys many standard fight mechanics. His Smoke Screen slows 10-25% and causes misses 40-70%. Blink Strike teleports him behind a target enemy, causing bonus damage. Cloak and Dagger means he's invisible most of the time.
If the humans use Riki in a way that OAI5 don't expect, they may win. Even with that, OAI5 thinks their win percentage is >90% - so ... First blood goes to humans! Earthshaker blocked a bot in the river (no entrance / exit) and OAI5 bot fell. Good start - go humans!
OAI5 likely need to use wards and other items well to counter Riki - it's the only way to counter his invisibility. The ward placement by them so far doesn't seem great - two observation wards in almost the exact same spot.
OAI5 push hard on bottom, aggressively going under the enemy's tower (very destructive at early levels), and killed a human. 1-1 so far. Tiny gold advantage to OAI5. Still anyone's game. The play so far looks fairly standard @DOTA2. OAI5 catch another human at bot. 2-1.
OAI5 trash talk update: "We estimate the probability of winning at 95%"
OAI5 attack again at bottom, kill two humans. 4-1. ~1k gold advantage. ~10k gold advantage is generally considered impossible to recover from in professional play.
OAI5 attack again at bottom, kill two humans. 4-1. ~1k gold advantage. ~10k gold advantage is generally considered impossible to recover from in professional play.
Humans score a kill against OAI5 at mid but lose a hero themselves. 5-2. At top the humans push hard, go deep into enemy territory to chase a dying hero, get the kill. 5-3. Gold advantage now <1k - getting better! 7-3 after lost human fight at bot.
Humans have used dust (temporary invisibility) to track an OAI5 hero. They thought they were chasing to kill but bots rotated quickly to top, pushing behind enemy lines. 9-4. We're rapidly approaching Level 6 where the heroes get their ults. Bots push aggressively at mid, 10-4.
OAI5 took the Tier 1 tower at mid - and killed some more humans (11-4) - but were really just interested in the tower. Eek. Blitz rushed Shadow Amulet which makes you go invis. Good strat against AI as they don't know how to deal with invisibility well.
Shadow Amulet makes you invisible where you stand BUT (a) it takes a second or two to cast, during which you're vulnerable and (b) you become visible the second you move again. This item is used in many other complex item builds all of which involve invisibility.
The human strategy this round may well be trying to take advantage of OAI5's weak strategy when it comes to invisibility. I will note that I have seen a Sentry Ward (shows invisible units) at mid - so they do know how to ward against invisible units!
Humans did an amazing attack push at top - but then had to fall back. Very low health human successfully saved by his team and then killed OAI5 chaser. Joking that they "baited" OAI5 (tricked into chasing). 15-6. A small win but they'll take it. Gold advantage is 1k toward OAI5.
Humans successfully ganked (surprise attacked) an OAI5 hero. 17-8. Humans have a small (<1k) gold advantage! OAI5 confidence has fallen (!!!) - but only to 98% 😭 Large team fight at mid - 19-9 with humans seeming to come into their own. Riki invisibility strat is paying off.
Largest human win so far - Riki scouts out two OAI5 bots whilst invisible, humans jump on opportunity, kill both OAI5 bots. Unfortunately this win is short live - humans lose three and don't trade for anyone else. 22-11 favouring OAI5. So far zero buildings hurt on OAI5's side.
AI note - OAI5's Gryocopter is farming (killing neutral creeps in the forest). This is important as OAI5 didn't do much farming at all in the last public match. Sadly I am following the broadcaster's screen so can't investigate these myself.
20 minutes in and OAI5 are hitting high ground (i.e. into the human's base).
OAI5 trash talk: Above 99%.
Entire human team is wiped out, OAI5 have lost a single hero. OAI5 having taken Tier 3 tower and barracks now backing off to regroup.
OAI5 trash talk: Above 99%.
Entire human team is wiped out, OAI5 have lost a single hero. OAI5 having taken Tier 3 tower and barracks now backing off to regroup.
31-12. OAI5 are rotating to bottom. Humans already have 2 dead out of 5. 33-12. Tower is down and the barracks is being wiped out. Humans are making the last try, jumping in with all their ults. Sadly all humans die again. 9k gold advantage to OAI5. We're essentially gg.
OAI5 are systematically marching down the only remaining lane, top. 38-12. 11k gold advantage. Unlikely the humans score even one more kill against OAI5. Humans announce gg.
Main issue: humans used their secret advantage and went for the heroes that were causing the OAI5 bots the most concern, specifically those heroes and items that leveraged OAI5's likely weakness in handling invisibility, yet they were still decisively wiped 😭
General note: the OAI5 bots have a reaction time of 200ms.
Human reaction time: "The average (median) reaction time is 215 milliseconds, according to the data collected so far"
I would argue the issue is not reaction time but ability to evaluate global information in detail.
Human reaction time: "The average (median) reaction time is 215 milliseconds, according to the data collected so far"
I would argue the issue is not reaction time but ability to evaluate global information in detail.
A human reaction to a stimulus may be possible in 215ms - but that's likely an uninformed muscle memory response. OAI5 - and essentially any other automated decision maker - can perform a far richer analysis and make a far better decision in the same time.
Even if you pushed the OAI5 reaction time to 500ms you'd likely still be annoyed at how well it responded to everything. The issue for humans is less likely to be reaction time and far more likely that the only decision they can make in such a small time span is a suboptimal one.
Another issue is that the set of heroes that are best equipped to cause pain for OAI5 are also difficult for humans. Why? In a standard human vs human game Riki needs to start slow and build toward a terrifying hero. Many of these heroes peak in power and potential later game.
To get to that stage however the rest of the team needs to "carry" them to that stage, providing them safe options for experience and gold. OAI5 decimate that. I have yet to see a team get a good start against OAI5. That means none of these later stage heroes really pay off.
Whilst I tend to think OAI5 will be annoyingly brilliant later game as well, what we're mostly seeing here are humans who can't get that beachhead in the early game.
For game 3, the OAI5 team are being selected by humans from the audience and/or @Twitch! It would also be fascinating to see a purposefully badly matched game. Think Pokemon when you're Misty's gym (i.e. all water) and then need to verse Pikachu (uh oh).
Audience selected Slark as first pick for OAI5. Worrying for the humans as, just like Riki, he's a hero that sneaks about in the shadows, leaps out, kills you, and then disappears. Many professional games end up steamrolled by a Slark with a good start.
dota2.gamepedia.com/Slark
dota2.gamepedia.com/Slark
The line-up for OAI5 this round is fairly Looney-Tunes. Two big scary tanks, Sven and Axe, with two good invisibility / ganker (surprise attack) heroes, Slark and Riki, and the Queen of Pain who can blink (teleport a few metres) for escape and attack.
OAI5 don't have any support characters however - so no-one to support the main damage dealers. OAI5 thinks it has a 2.9% chance to win. Go humans! I'll admit that I'm still very worried though - that's 2.9% when it's playing against itself.
OAI5 has an odd early strategy - grouping up as four at bottom, leaving top entirely undefended. With that strategy they get first blood on bottom however. MoonMeander, the hero having to try to take on bot, complains in chat that this is desperation play :P 2-0 for OAI5.
For the humans Gyro gets an early kill. And another. 4-3 in favour of OAI5. That's desperately needed as all the heroes on OAI5's side will scale well into late game if not handled. Sadly a human offensive at top failed and it's now 7-4. There are problems already :S
The traditional issue with humans playing the OAI5 hero line-up would be that the heroes wouldn't share experience well. Humans are generally pretty greedy and all of the OAI5 line-up needs experience for the heroes to scale well.
OAI5 think it has a ~90% win probability. Game is 8-6 but the humans are actually 1k up on gold. The humans take OAI5's tower mid - potentially the first time in this series so far that that happened (!!!). Note: OAI5 didn't actually defend that tower, they just gave it away.
Riki did a trademark perfect escape - started teleporting back to base when the surrounding human heroes didn't have enough damage to take him out whilst the teleport "channeled" (~charged) and none of the human heroes had stuns. 10-7.
OAI5 are taking a different strategy to their usual - they're not grouping together and they're farming. 11-7 (OAI5) with <1k gold (humans). Heroes are reaching Level 6 which is when they can get their ultimate ability. 12-8 in a 1 for 1 trade at bot. Not terrible, not good.
QoP was stalking the humans at top. Walked in, saw it was a bad idea, jumped away, then returned a few seconds later when more friendly heroes were near. 13-10. Not the best engagement for OAI5 but it does show a back and forth strategy where it's re-evaluating potential fights.
The OAI5 Slark was perfectly harassing. Slark almost died at mid, ran away, saw a vulnerable hero at top, chased that hero, realized it was too dangerous, so ran away. This provided enough space for the rest of the OAI5 team to take out the final Tier 1 towers.
OAI5 hero at bottom sacrifices their life to kill the Tier 2 tower. Selfless act. Why is that a good idea? Gold from Tier 2 towers is quite high - but if you leave it close to destroyed the human team can "deny" it - denying you gold. Good little robot ^_^
15-15 - even game with slight gold advantage (1k) toward the humans. 16-15 after three human heroes take down OAI5's Slark - but that was a major distraction whilst the rest of the OAI5 team roamed relatively free.
The humans took down another Tier 1 tower! Second building destroyed by humans so far this game! Before celebrating too much however, remember that OAI5 already has all the human's Tier 1 towers ... Eek. Humans take another OAI5 hero - but requires 4 of them (and an ult). 17-15.
So far the OAI5 team have not grouped up at all. Very different to the strategy in past games. They've realized they won't win team fights so are avoiding them and split pushing. Humans finally get a strong decisive win - 21-15 in the favour of humans with a 4k gold lead!
5k gold lead - this is getting worrying for the poor bots. The real question is whether the ~90% win probability it still sees exists. Is it from late game? Two OAI5 heroes gank MoonMeander (human) and kill him - but both OAI5 die themselves. 24-16. 6k gold lead. 25-16.
OAI5 have re-evaluated their win percentage - it's now below 1%. Eek! 27-16. Any momentum OAI5 had is gone. Commentators joke that this could be OAI5's next level mind games / trash talk ;) Humans leading 28-16 with 9k gold lead. End of days for bots likely.
OAI5 heroes go insanely aggressive and over-extend into the enemy's base. 30-17 with an 11k gold advantage. In professional games this would generally be seen as gg territory. 31-17. 32-17. Can OAI5 recover? 33-17. 34-17. 15k gold lead.
Sven is crazy and runs into the enemy base to escape. The OAI5 heroes are acting fairly crazy. Good move by OAI5's Slark however - used a Shadow Amulet to go invisible, just as the humans tried to use against OAI5 in the last game!
To note: OAI5 didn't learn how to do that from analyzing the last game, it likely has just learned how to play like that in the many years of gameplay it has experienced. The main interesting point is that both humans and bots tried to use the same strat when behind. 39-18 + 18k.
OAI5 Slark also went to give his Shadow Amulet to a team mate - so they know how to share objects and that such an object would actually be useful. Sadly that didn't work however. Humans are taking the Tier 3 tower and barracks at OAI5's base (bottom).
42-19 with 22k gold advantage. The comeback mechanic is that any death from the humans now is heavily penalized as OAI5 will get a good deal of gold, a good deal of experience, and the time to respawn is quite long at these levels. OAI5 also pulled off a successful gank. 42-21.
OAI5's Riki investigated Roshan but didn't go for it. 1% win percentage. Gold advantage is down to 21k - nope, back up to 22k. 43-21. The small momentum that the OAI5 team had seems to be transient. 43-22 after another good gank by the OAI5 team.
Sadly the team that ganked successfully then got obliterated by the humans. Sven and Axe are a crazy duo and tried to solo attack the enemy base. Commentator speculates that they're trying to essentially rat - or pull the enemy's attention from attacking to defending.
Humans had taken bottom and middle and are now working on top. OAI5 Slark attacked top lane at human's base - good rat tactic but failed. Humans win! A bit of a Pyrrhic victory given OAI5 was given a horrifically poor team combination to go with ;)
The real challenge would be to have the humans set a horrible team for OAI5, give OAI5 a day to train (180 years worth of working out how to win with a suboptimal team line-up) and then see what happens! :) With that final human victory however, I think I'm done. gg all! ^_^
