OpenAI, the AI research organization, can claim a world first: its artificial intelligence system is trained to play the complex game of strategy. dota 2 has surpassed a world champion e-sports team. The competition was held today in San Francisco and was called the OpenAI Five final, which put an end to the public demonstrations of the organization of their Dota– bringing technology together on a high note.
The competition on the human side included five top dota 2 professionals from the OG team, which won the most coveted esports award last year when it was ranked # 1 at The International, the premier annual event dota 2 Tournament with prizes now totaling $ 25 million. OG faced off in a best-of-three contest against OpenAI Five bots, all trained using the same deep reinforcement learning techniques and independently controlled by different layers of the same system. Reinforcement learning is effectively a trial and error approach to self improvement, in which AI is introduced into the game environment without any understanding of how the game works and is extensively trained using reward systems and other incentive mechanisms.
Today’s performance is by far the highest quality demonstration of OpenAI Five’s capabilities to date, as the system lost nearly two games to less capable esports teams last August. According to OpenAI co-founder and president Greg Brockman, who is also the organization’s chief technology officer, OpenAI Five improves when playing in a fast-paced virtual environment. “OpenAI Five is based on deep reinforcement learning, which means we don’t code it to play. We code how to learn,” Brockman told the crowd before the competition. “In its 10 months of existence, it has already played 45,000 years of dota 2 gameplay That’s a lot, still not bored. “
OpenAI’s Dota 2 robots have been trained for the equivalent of 45,000 human years
dota 2 is a highly complex strategy game, featuring over 100 unique characters, deep skill trees and item lists, and an incredible variety of variables that unfold on screen at any given time in a game. As such, OpenAI imposes certain limits when its artificial intelligence system plays professional gamers, highlighting the number of heroes used by the two teams of five players.
In this case, each squad had 17 heroes to choose from. OpenAI also chose the game mode called “Captain’s Draft”, which allows each team to strategically ban heroes to prevent the other team from selecting those characters before using a different order of selection. That allows the captain to build strengths between hero combinations and take advantage of enemy hero weaknesses through strong counters once teams start filling the roster one by one. Like the previous matches, OpenAI also disabled the summoning and illusion functions, both of which involve the introduction of additional variables in the form of hero copies and unique creatures that OpenAI has not trained its system to take into account.
Beyond that, the game is played like a normal dota 2 match, with the ultimate goal of destroying the enemy team’s “old” or a large tower at the end of each team’s territory that becomes vulnerable only when the enemy team successfully destroys smaller towers during the course of the match, between heroes and -Heroes team fighting enemies.
In the opening match of the day, OpenAI Five stunned OG and claimed victory by relying on a number of aggressive tactics, including the quirky decision to spend earned in-game currency to instantly revive heroes after death, even upon death. beginning of the match. As Greg Brockman, CTO at OpenAI pointed out, OpenAI likes strategies that favor short-term gains, revealing its shortcomings in mastering the kind of long-term planning that humans are great on and generally depend on. to win such strategy contests. However, in this match, the early buybacks paid off and OpenAI Five gained an advantage that OG simply couldn’t overcome as the match dragged on in the 30 minute range.
We see this happen in test games all the time: robots buy, humans laugh, and then humans lose. It’s hard to know if it will happen here too …
– Greg Brockman (@gdb) April 13, 2019
In the second match, OpenAI performed even better, gaining an early lead against OG in the opening minutes and then ruthlessly advancing on the human players until they were victorious in just over half the time required to win the first match. Mike Cook, an avid dota 2 The gamer and viewer who specializes in mixing AI and game design, noted how unusually aggressive the OpenAI Five was that he started playing in Game 2 and how little OG was doing to combat their advances on the map. Cook specifically noted how well OpenAI Five was able to leverage their specific hero picks.
This is probably over now, unfortunately. OpenAI has four of the top five heroes ranked by net worth. At ten minutes against bots with running OpenAI, this is really bad. #openaifive
– Mike Cook (@mtrc) April 13, 2019
For OpenAI, the victory here is not just a cause for celebration in and of itself, but a testament that its approach to reinforcement learning and overall philosophy on AI are producing milestones. The research team will no longer conduct public demonstrations of its AI bot, but is now working on software that will allow humans to collaborate alongside the OpenAI Five software in real time, play as a team with the bots, and learn from their strategies. peculiar and unprecedented. and behaviors. The organization is also launching a platform for the public to play against OpenAI Five, a mode called Arena, which will be open for three days starting April 18.
Special Announcement: We are inviting the entire internet to play OpenAI Five (either as a competitor or a teammate) at once.
Sign up today! Very excited to see what we learn from observing OpenAI Five in the wild. pic.twitter.com/TaMhxdgVIt
– Greg Brockman (@gdb) April 13, 2019
OpenAI says that collaboration software may never reach the public, although I was able to test it out for myself here at the event. (Despite having world class dota 2 On my team, unfortunately, I was crushed in a much less dramatic way than OG. But Sam Altman, co-founder and CEO of OpenAI, says this kind of work is evidence that collaboration with AI agents could yield huge benefits down the road. .
OpenAI wants to take its Dota 2 learnings and apply them to new challenges, like robots
“That is an important lesson in how the world is going to work, train these things and make them work in parallel,” says Altman in an interview with The edge. “Collaboration is one of the most positive visions we have for the future of the world: artificial intelligence works together with humans to make humans better and have more fun and more impact.”
Altman says OpenAI will likely continue to foray with dota 2 and other video game environments, mainly because they are such good benchmarks for AI and good benchmarking tools for measuring progress. But it tells me that there is probably no video game now that a system like OpenAI Five cannot dominate at a level beyond human capacity. For the broader artificial intelligence industry, the domain of video games can become obsolete, simple bets are required to prove that your system can learn fast and act in the necessary way to tackle more difficult and real tasks with more significant benefits.
Ultimately, OpenAI wants to take its dota 2 learn and expand them to new domains outside of games and eventually into the real world. To that end, the organization is working on using reinforcement learning and other techniques to imbue robotic hands with more skilled, dexterous, and humane movements.
“What OpenAI is trying to do is build general artificial intelligence and share those benefits with the world and make sure it’s secure,” Altman says, referring to the quest to build a multipurpose AI system capable of performing any task that a human could do. can. “We weren’t here to beat videogames, as much fun as it is. We are here to uncover secrets along the AGI path.”
Correction: An earlier version of this article said OpenAI co-founder Sam Altman was the president of the organization. He is, in fact, the CEO, while CTO Greg Brockman is its president.