The best performing AI in blockchain games, a new DRL model published by rct AI based on training AI in Axie Infinity, AI surpasses the real player’s level
The discussion about AI production, creation, and human/AI interaction seems to never stop. When a more digital world arrives, how will AI evolve as important productivity? rct AI, as an AI technology & blockchain game company, recently published a new DRL model based on training AI in Axie Infinity to achieve the best performance of AI in blockchain games. This RL-based AI algorithm for large-scale action space, ACAR(Action Clustering using Action Representation), has achieved a double improvement in efficiency and winning rate in a large number of simulated battle data, which has surpassed the real player’s level. Some even call it Alpha Go in blockchain games.
The emergence of ACAR will upgrade or innovate the use space and development direction of AI in blockchain games. It will provide more research and application directions for the improvement of man vs machine battle computing power, the construction of an immersive virtual world, and the deep interaction of users. The paper on the ACAR algorithm — “Towards Modern Card Games with Large-Scale Action Spaces Through Action Representation” was selected for the IEEE Conference on Games, CoG 2022, and was invited to give an oral presentation.
About the new DRL model
In the last few years, the advent of AlphaGo and AlphaGoZero has made people realize that reinforcement learning is an algorithm that can perform as well as or even better than humans in games of chance. Traditional card and chess games have always been the classic direction of AI research. Popular topics include Texas Hold’em, Mahjong, and Doudizhu. These games all contain the intractable problem of “huge and complex action space”. In addition, AI also needs to find out the most beneficial strategy for itself in the current game based on the opponent’s game strategy and play style.
rct AI has always insisted on “Future web3 with AI”, and brings the AI battle research of card games to the blockchain game directly. This time, we chose the most popular card battle blockchain game Axie Infinity as the research object, because it not only contains the above-mentioned “huge and complex action space” (the action space combination of all cards is around 10²³), and the nature of the game of chance, but also contains a huge number of card groups. (There are more than ten popular card groups in the whole game, at least dozens of card groups in total, and hundreds of cards with different effects).
Since it is impossible to add prior knowledge of human strategies to all decks before training, the opponent’s card groups will not be the same in different games. This adds another challenge to the learning of the DRL (Deep Reinforcement Learning) model.
In continuous experiments, rct AI proposes a more efficient and general RL-based AI algorithm ACAR to solve the above problems. First of all, our team introduced a pre-trained embedding function to solve the huge action space in this problem. Through the battle with different opponents, the final effect of the executed action is used to learn the representation of different combined actions to achieve the effect of efficiently exploring the action space. After having the embedding function for action representation, our team can not only use the evaluation network Q to evaluate the output actions in the Policy network in the subsequent RL training, but also use the embedding function to reduce the dimension of the current network output action, and then use Q to evaluate the adjacent actions of the reduced dimensionality action to select the optimal action in the current state.
The RL training combines the Embedding function, and finally, in a large number of simulated battle data, the rct AI algorithm not only outperforms the other two Baseline algorithms in most decks (the average winning rate is 5% and 7% higher) but also the response efficiency is also completely better than the other two algorithms (only 43% and 39% of the average time consumed in the other 2 algorithms).
Application scenarios
The launch of this research result has attracted widespread attention in the industry, opening up a new blue ocean for the application of AI in the field of web3 games. According to statistics from SupraOracles, the overall market value of web3 games has exceeded 40 billion US dollars.
This research allows us to predict that, at least in blockchain games, AI’s participation in the economic cycle will bring true sustainability and stability, and completely liberate the liquidity of digital assets in the game.
First of all, for game projects, the number, type, and combat power of game players will be expanded. In web3 itself, virtual identities and digital citizens cannot be avoided. We have given it more capabilities that are comparable to or even surpass those of humans, allowing these AI-powered players to enrich the game ecosystem with high quality and efficiency, to help game projects achieve their goals at different stages in terms of user volume and types, project revenue, data retention, and product co-creation.
Secondly, for players, under the current mainstream game financial system of “P2E” “P&E”, a double upgrade of game experience and revenue will be achieved. On the one hand, users will have extensive and ever-changing gameplay, and will no longer face the trouble that other users’ levels are too different from their own. AI virtual humans can equally participate in various gameplays with human players, bringing players a more personalized intelligent interactive experience. On the other hand, AI-powered players can collaborate and divide labor with real players. While real players can deploy strategies, AI-powered players can yield farming, which will improve the efficiency of game battles, thereby providing the benefits of more stable automation projects.
Previously, people in the industry were thinking, when a more digital world arrives, how will AI evolve as an important productive force, what role will it play, and will the interaction between humans and AI produce new variables? We have to admit that AI can help humans explore the potential of web3, and this is no longer simply stimulated by the collision of people.
Since establishment, rct AI has been committed to using artificial intelligence to provide complete solutions for the game industry and to create a true Metaverse with artificial intelligence-generated content. Our team has brought core technologies and innovative products into various industries and launched Chaos Box, Morpheus Cloud, etc. so that dynamic and personalized interactive digital content can be generated in real-time in multiple scenarios of different types. The gameplay, plot generation, NPC personalization, character dialogue, and other aspects are comprehensively intelligent, helping to create a truly dynamic and intelligent game ecology.
The papers recognized by the world’s leading academic conferences in gaming and artificial intelligence were all done by the algorithm engineers in the Deterrence team, our decentralized virtual being platform. So far, we has successfully established a mature web3 business ecosystem, including the self-reproducing virtual being NFT “Nüwa” released by rct AI and artist Song Ting; Delysium, the world’s first playable AAA MMORPG web3 game; and Soularis, jointly launched with China’s leading virtual being ecosystem NEXT GENERATION Culture to create a virtual society where users and AI avatars coexist and digital identities and assets are interoperable.
Relying on solid technology barriers and cutting-edge ecological business layouts, rct AI has accumulated a lot of experience in the fields of technology, games, entertainment, and Metaverse all around the world, reaching more than 200 million users.
About rct AI
rct AI is providing AI solutions to the game industry and building the true Metaverse with AI generated content on blockchain. By using cutting-edge technologies, especially deep learning and reinforcement learning, rct AI creates a truly dynamic and intelligent user experience both on the consumers’ side and production’s side.
rct AI believes the next generation of gaming experiences will be more open-world, more immersive, and more intelligent. The interaction between human players and the virtual world will be more dynamic and multiplex.
However, with traditional technology and workflow, it’s impossible to go even further. To realize this vision, rct AI is helping designers and game developers create a truly dynamic and intelligent gaming experience with various products and solutions to the industry.
