miHoYo develops AI model that can play complex games like Honkai: Star Rail just like a human would, with mouse and keyboard inputs 

The project is being headed by the Honkai AI R&D Team, in collaboration with researchers from the National University of Singapore and Tsinghua University.

On November 13, miHoYo’s in-house Honkai AI R&D Team, in collaboration with researchers from the National University of Singapore and Tsinghua University, published a research paper and project webpage about their latest AI game agent, named Gamer. The project introduces Gamer as an AI model capable of playing complex 3D and 2D video games from start to finish without touching a single line of backend code. 

According to the research team’s paper and project page, Gamer does not rely on in-game data or dedicated APIs. Instead, the AI makes inputs relying only on what is shown on-screen, emulating “human-like behavior” in the process. More specifically, Gamer uses screen images captured at 5 Hz per second as input and outputs keyboard and mouse movement commands. 

To demonstrate the AI’s capabilities, the paper showcased gameplay footage of Gamer successfully navigating through main quests and solving puzzles in the company’s own game, Honkai: Star Rail. The footage shows the AI moving characters toward objectives, following NPCs around to progress, and even clearing enemy encounters on its own. Gamer also demonstrated the ability to independently complete five mini-games, ranging from puzzles requiring logical thinking to challenges demanding precise timing and accuracy. 

A GitHub page for the model has also been published, noting that the inference engine (Streaming-vLLM) used to run Gamer is scheduled to be released soon. It is important to note, however, that since the paper states that results were collected using high-performance data center GPUs such as H200s, it is unlikely that the model will see any personal use. 

In response to concerns that the technology might encourage cheating and automated scripts, the research team emphasized that Gamer is part of an academic research project exploring universal agents for interactive environments. Its potential applications include accessibility assistance, game testing, and research on human-computer interaction. The team explicitly stated that it “does not endorse using Gamer for cheating, unauthorized access, exploiting security vulnerabilities, or any behavior that harms users or undermines fairness.” 

Related: AI voice service ordered to pay miHoYo $112K after reproducing the voices of 63 Genshin Impact characters without permission 

miHoYo to shut down its AI-powered “virtual girlfriend” simulator BSide: Olivia Lin less than two months after launch 

Akira Sada
Akira Sada
Articles: 56

Leave a Reply

Your email address will not be published. Required fields are marked *