A wild ambition about game AI

What exactly are we playing when we play games?

The game has always been called the ninth art, and the top eight paintings, sculptures, architecture, music, poetry (literature), dance, drama, and movies can actually be more or less reflected in the game. For a person with aesthetic expression needs, almost all his fantasy and aspirations can be realized in the game. In my opinion, this is the real charm of the game.

It is true that, driven by economic interests, most games are difficult to get involved with art. But it is undeniable that games have brought many changes to people’s lives. It has not only become the main pastime for many people, but has gradually formed its own unique cultural circle (such as forums such as Azeroth National Geographic).

So what are we playing when we are playing games? Here I want to speak from the uniqueness of the game. The more obvious characteristics of the game, and the key difference between the game and other art forms is the interactivity.

There are two sides to interaction, one is who you interact with, and the other is your level of interaction. In the process of playing the game, the roles we interact with will change with the different plots and different competitive occasions, and our own interaction level is constantly changing as we play the game.

I think the so-called pleasure of playing games is largely derived from the improvement of these two dimensions. It can be said that the change of the role mainly depends on the design of the game plan, such as playing a more powerful BOSS and playing a newer role. The acquisition of these parts depends to a large extent on the improvement of the level of interaction. In the process of the game, we constantly surpass ourselves and constantly defeat the opponent to obtain the ultimate sense of accomplishment. So when we play games, we are actually playing a big game of self-development.

Game and flow

Therefore, how to effectively guide players to improve their interaction level in the game is actually a core issue in game design. However, each person’s learning ability and learning preferences are different, which creates different experience curves for different players . A design that is friendly to some players may not be friendly to other players. So the game is too simple will make the game without challenges and lack of fun, the game is too difficult will make it difficult for players to gain a sense of accomplishment to persuade players (of course, there are some players like to feel abused, such as the Black Soul series, Sekiro and other games with high difficulty The operation is famous, and many players are happy to be repeatedly tortured in it).

[ 游戏《只狼》玩家死亡画面(恶搞) ]

From here we can see that a proper game design can really make players have fun in it. This is actually the flow that is often said in psychology. When the user needs a high level of skill to complete the interactive behavior, and through the perception of high challenges, and the two reach a certain balance, there will be a flow experience. Creating a flow experience for players is actually the key to providing an excellent development environment for players as mentioned earlier.

[ 心流理论 ]

How to create flow in the game?A machine learning perspective

But how to quantify the description of heart flow, so as to guide the development of the game, and optimize the player’s experience is a very difficult task. Why is it difficult?because

  • We don’t know how high the challenge is appropriate for users?
  • We don’t know how much effort users need to put in to achieve the required high skills?

Usually we can adapt different AIs for players (in the story game, the difficulty of the BOSS, in the competitive game, the strength of the AI ​​to accompany the player) to adjust the player’s game experience. As for how to generate these AIs, you can use the traditional way of writing rule models (behavior trees, finite state machines, etc.), or consider using machine learning (such as OpenAI Dota 5, AlphaGo, etc.) to generate them.

But how to adjust it?The most traditional and easiest way is generally left to the player to choose

[ 游戏《瘟疫工厂》难度选择界面 ]

In online games, the player’s past performance is generally used to grade the player’s abilities to match players or AIs with similar abilities.

[ 游戏《王者荣耀》玩家匹配界面 ]

But this is actually far from enough, because the rating generated by a fixed rating or player’s past performance is static and rough within a period of time.The player’s experience comes directly from a level or a game, and in this segment

  • Players of the same level have different dimensions of skills. For example, some players are good at offense, and some players do better defense
  • Players of the same level have different preferences for the game process during the game. For example, some players prefer to crush rounds, some prefer fierce battle rounds, etc.

For these reasons, it is difficult to better optimize the player’s flow experience by measuring the average of the players in a window.

If you want to create a more refined experience control during the player’s game, so that the player can win or lose the game according to some specific modes, thereby creating a flow for the player. We must use a large amount of player data to make a more detailed analysis of the player’s experience.

Let’s take the control of the AI ​​in the game as an example. We can imagine a framework as shown in the figure below

Based on the player’s social relationships, past game data, etc., we can build a model to depict the player’s portrait; based on a large number of player battle data or use reinforcement learning that does not rely on human data to estimate to get a judgment on the game situation or the current game A measure of the player’s challenge.

We use an in-game control model to comprehensively use player portraits and situation prediction information, and use the control model to generate adjustments to the AI ​​in the game.

Here comes the first question, how can the control model adjust the AI?

We can consider perturbing AI’s decision-making from the input part and output part of AI, specifically

Input part:

  • Control the accuracy of model perception
  • Disturb the current perceived state (e.g. wrong estimation of own skill time, ignoring teammate state etc.)
  • Visibility (the degree of visibility to the enemy’s state, such as cheating)
  • Control AI’s style preference (radical preference, conservative preference)

Output part:

  • Limit the output of actions (AI is not allowed to perform certain operations, etc.)
  • Perturb the output (wrong decision, etc.)

The above part needs to be combined with situation prediction to make decisions accordingly, but there are several hyperparameters (such as disturbance content, amplitude, direction, desired style preference, etc.) which is a complicated decision-making process.

Here we can consider dividing the decision of the control model into broad categories. In each type of decision, different disturbance schemes can be generated. Although the specific disturbance amplitude is more difficult to define, the direction of the disturbance is usually more intuitive, such as letting AI’s prediction becomes worse, and it only needs to give him less information. These categories and the range of specific disturbances can be summarized through expert knowledge. The next thing to do is to establish a correlation between these disturbance values ​​and the effects on the player.

The establishment of this association can be achieved by using machine learning based on player data. Specifically, the in-game control model will continuously output different control schemes. These control schemes will perturb the AI ​​model according to the preset configuration, thereby changing the AI ​​state (ability, style, etc.) in the game. There will be different feedbacks throughout the process (such as the number of swear words in the game, the behavior of likes after the game, the behavior of reporting, etc.). Based on these feedbacks, we can organize players’ experience data on the in-game control model and train and update our control model. With the accumulation of data, our control model can better grasp the preferences of different user portraits for games in different situations, so as to automatically adjust the state of AI in the game, and create a refined and high-quality game experience for players.

Having said a lot, let’s go back and take a look. We will find that the nature of the problem is actually not complicated. The process of creating a refined game experience for players is actually the same as when we are browsing shopping websites and vibrato The process is similar. Shopping websites choose different products for us to let us get a better shopping experience, and the video stream chooses different videos for us to make us more interested.

In the game, if we want to create a flow for players, we can actually use the game elements on hand to provide players with different recommendation services. In this way, the whole problem is actually very clearly decomposed into two parts:

  • How to design various game elements for recommendation?
  • How to improve the ability of the recommendation system?

From this perspective, it may not be so difficult to look at the flow design of the game. Of course, there is still a long way to go in this direction. Although the recommendation system has been researched and applied for many years, there are still many problems to be solved (for example, how to solve the cold start in the game scene?). Player feedback is not as easy to obtain as in common recommendation scenarios. The instant feedback in the game is often very sparse, and the feedback at the end of the game will become less accurate when assigning the decision of the control model.

Another important question is how to quickly design various recommended game elements? It may be one way to use machine learning to generate AI. The previous examples are using AI as an example, but there may be more angles. There is a serious question that needs to be faced by the majority of gamers. Is it possible to optimize the player’s experience only by putting AI as a deceiving player in the game? Just like wine trays and medical trays, is this “playing tray” good or bad? I can’t answer.

You may not consider thinking about game design from a recommended point of view before, but I believe this perspective can bring some changes to game design. There should be many elements that can change the player’s experience, and it will not be limited to the AI ​​part. The design of these elements to a large extent determines whether the whole framework can really play a role. Only when these elements become richer and more high-quality, can we create the ultimate player experience.

In the future, for the optimization of player experience, I think it must be based on this data-driven perspective, and new game forms should also be born under this drive. I don’t know how long this future is, but it must be coming!

Source: Tencent Game Academy
Original: https://mp.weixin.qq.com/s/g0puxOfLpJn1AsCl5WZpmg


Related Posts