Artificial Intelligence Horizon Zero Dawn
- Transfer
Horizon Zero Dawn is on the list of some of the best exclusives for the Playstation 4. In the role of hunter Eloy, players travel through the post-apocalyptic landscapes of the future to uncover the secrets of her past and learn about the causes of the destruction of the world. The decline of humanity led to the flourishing of “machines” - robots of various shapes and sizes, living freely everywhere. These robots are intelligent, coordinated and deadly: to survive, you need to think quickly, to destroy them - to prepare and plan carefully.
In this part of the article I will talk about the gameplay and decision-making systems that make these creatures alive and provide the basics of the gameplay. I will talk not only about how they make decisions about different actions (gathering resources, hunting, survival), but also how they are coordinated in the wide world, individually and in groups. In addition, I will talk about how the game creates an ecosystem of machines, which is balanced and controlled in all territories and in all areas of the new world.
Warning: this article contains Horizon Zero Dawn story spoilers.
Horizon Zero Dawn automatic animals dominate the world and retain control of the territory. Each machine has a unique aesthetics and structure, reflecting its functionality and purpose within the game narrative. During the passage of the storyline campaign, players will learn about the "Plague of Faro" - the army of rebellious military vehicles that destroyed the world, as well as about the race of machines designed by AI named GEYA, which was supposed to restore the planet to its natural state; their design repeats long extinct terrestrial animals.
Therefore, the functionality of the machine often keeps the secrets of its weaknesses. Players can use a visor to scan nearby machines to find out not only their type, but also vulnerabilities and weaknesses. In addition to weak points, the functionality of the machines within the narrative determines their in-game execution. Including:
In total, in the open spaces of the game, you can find 28 unique machines, each of which has its own passive and active behaviors. Players need to learn how to find the right approach to them. Although the greatest threat on the way to Meridian (such as "Thunderbird" and "gromozev" ) roam the world alone, players can still stumble on less lethal machines running in herds. You can often see grazing robots that are grazing alongside, such as “royals” , “buffaloes” , “scavengers” and “spearheads” : they support each other in the process of performing their basic functions. Over time, these herds increase in size and become more diverse, for example, the convoys of "scrubs", flocks of "buffalo" and "pilozubov" , protecting the "ruminants" and "spearmen" .
To cope with this task, the game requires a lot of AI systems that determine the behavior of each robot, their behavior inside the herd, the collective intelligence of the herds themselves, but at the same time balancing the strength of the herds in different parts of the open world for a more diverse gameplay. The developers of Horizon Zero Dawn , the company Guerilla Games, has begun this project, not having many of the AI tools needed to implement this gameplay.
The company previously worked on the Killzone franchise., and therefore its tulchain was mainly designed for first-person shooters and could not cope with the complexities of the architecture of a large open world. Therefore, in the process of development, the studio created new AI systems and gameplay, as well as expanded existing ones. In this and subsequent articles I will tell you about the following:
Let's start by finding out how each AI character makes decisions and how the game controls the behavior of the herds.
Each machine is represented in the game world as a separate AI of a non-player character, called an “agent”. But more interesting is that each herd is an agent. The Horizon Zero Dawn using hierarchy in which a machine may exist as a full agent individually, or as a child element of group agent which is used to store information about the herds.
Example of group hierarchy [from (Beij, 2017)]
The group agent does not exist in the game world physically, but is responsible for coordinating all the agents inside it. Each individual agent has its own set of sensors, allowing him to hear close sounds or notice the dangers next to him. But in addition, the group agent has a blackboard system, which stores in-game information that the machine may need. This allows the game to reduce performance costs by saving information that each individual agent does not have to calculate (for example, safe places or active patrol routes). In addition, more importantly, agents in a group can exchange information with each other. Therefore, when a reconnaissance vehicle finds a player, each vehicle can decide how to respond to this information. Thanks to this you can see
The AI system used in Horizon Zero Dawn is called a hierarchical task network planner (HTN). Unlike planning systems used in games like FEAR , which plan one action after another, the HTN scheduler generates plans composed of action macros. Each macro contains several actions in a given sequence. This is ideal for developing games, because designers can create macros of good behavior that they expect to see in a game. I previously explained how the HTN scheduler was used in Transformers: Fall of Cybertron, but it is worth noting that Guerilla used the HTN scheduler as Killzone 2, that is, it was not developed from scratch, but gradually improved over the course of several years in various studio projects.
HTN Scheduler Scheme [of (Beij, 2017)]
The HTN Scheduler is used by machines for two purposes: individual agents can request an action plan to solve a specific task, such as finding new areas, attacking a person, or fleeing. Group agents can execute plans that set goals that should be assigned to individual agents, which allows groups to actively share information with each other within the hierarchy, and potentially form new subgroups or destroy existing ones. In short, each individual machine still has a great responsibility for its own behavior and the execution of tasks assigned to it, and also uses the scheduler to make decisions. Groups retain coordination of individual machines, regroup them on the basis of events occurring in the world and, most importantly, assign roles to each machine,
So, all these cars, individually or in groups, are created on the map, but how does the game know how and where to create cars in the open world? On the game map there are selected areas in which you can expect the appearance of certain machines, either individually (in the case of “thunderstorms” and “petrels”), and in groups (“runners”, “gumlings”, “kites” and others). However, the rest of the game world is filled with herds of different machines. How does a game, when creating a specific machine, know where to place it in the world, and in which group to assign it, based on the needs of the structure of the world?
This is achieved by using a system called “The Collective”. The “team” is a supergroup: all groups of machines and individual machines exist within the team. It manages the addition of all machines to the world, keeps track of whether they are in a group, controls the transitions between groups and the ecosystem of machines so that it does not reduce overall performance.
Individual machines in the world can transmit requests to join existing groups of the open world, if they act alone. This is a useful function, given that many machines remain isolated after the remaining previous group either ran away or was destroyed. Therefore, the team can reuse isolated machines and attach them to other groups. For this, each car has a "passport". Their “passport” stores information about the car (for example, its level and type), and the team uses this data to determine whether it meets the requirements of another group, and only in this case moves the car to the group.
Now let's learn how herds work, and how group agents are performed when interacting with the player. Herds are usually placed in a relaxed world in which they are not anxious and are simply located in a certain area of the map. A typical group consists of a mix of scouts, gatherers, and combat vehicles. Robot-gatherers are in the center, while combat and reconnaissance patrols are around to protect them.
When a flock is placed in the world, a collection of groups in this hierarchy requests a role. This role assigns to all machines the behavior that reflects their purpose. Such behaviors can be patrols in search of threats, searching for close sources of concern, attacking enemies, or simply collecting the resources of the world. Over time, these roles may change, and events force the herd to change its behavior and structure. Usually there is a limit on the number of machines that perform a certain role inside the herd, so that their structure is not unbalanced.
The role of the machines determines the tasks that the individual agent will seek to accomplish. Since the herd initially behaves relaxedly, the collectors will use the HTN scheduler and grouped data to search for the nearest resource extraction zones. In the meantime, reconnaissance and combat vehicles generate patrol paths to walk. Patrol routes are generated automatically based on local geometry, they avoid inconvenient terrain, and, more importantly, avoid walking on tall grass and other vegetation that the player can use to sneak up on the herd. However, in order to give players a chance in battle, many patrol routes are deliberately laid out alongside vegetation so that the player can set traps or crack robots easier.
Herds adjust the balance of their structure in the presence of a threat, forming groups of “flight” (flee) and “battle” (combat) [of (Beij, 2017)]
But, of course, they do not always remain in this state; when players start attacking machines, their behavior changes accordingly. For machines that are not part of the herd, it all comes down to a specific class of machine. Only a few escape when they are provoked directly. The overwhelming majority of cars attack the player, and even collecting machines, being alone, often attack the player.
However, if a player attacks a herd, the groups within the hierarchy react in the same way, but only if the herd is already alert. As mentioned above, the group agent allows you to exchange data with each machine, but they are not updated in each game cycle. Therefore, the herds do not have a “hive mind” and the player can kill the car without disturbing the rest of the group. However, if it is noticed or the attacked machine survives, it will warn the group about the attack.
At this stage, the hierarchy within the herd is shifted, the group of machines within the herd requests a new role and priority to perform the corresponding behavior. A group made up of collectors will play the role of fleeing and start a joint escape, while the group consisting of combat and reconnaissance vehicles will rebalance themselves, and the character attacking the herd will be the priority for the roles. Although in many cases it is just a lone player, such a scheme allows the system to scale for story missions in which the player is part of a group, or other dynamic events in the world when other human NPCs attack herds.
Although after this point everything becomes much more complicated, the player still has many opportunities to turn the situation around, because both the grouping systems and the combat AI systems provide an exciting and at the same time complex gameplay. This is achieved through a combat AI system that attempts to structure the battle in such a way that it is not chaotic or broken.
Formed combat groups have different roles and balanced priorities, so large herd machines are distributed into groups where possible, and smaller ones line up to support them. In addition, groups choose machines to attack a player, not only on the basis of whether the HTN planner considers that he can perform a certain behavior, but also depending on whether this attack is interesting. If all the cars simply attacked the player at the same time, then he would rather quickly suffer a painful death. Therefore, combat systems use an approach similar to that used in Halo 3 , when the balance of enemy attacks on a player is based on whether the attack will be interesting in the context of combat.
This requires an auxiliary function of selecting an action, which calculates how interesting it will be if the machine attacks. It depends on the current state of the machines, on whether the player knows that the car is near him, on the proximity to the player and on the amount of damage received and inflicted by her. This value becomes an input to the HTN planning system; his task is not only to ensure a certain complexity, but also to add a share of diversity to the battle.
But more importantly, when the HTN scheduler is asked to generate actions for the attacking groups, while the attack is chosen, the movement of the player and waiting for their turn are also distributed to other group machines. This is done intentionally, to create the opportunity for the player to either attack the passive car, or to counterattack the one that just hit it. Gradually, the player will be able to weaken them, and this is mainly due to the fact that the combat system intentionally leaves itself open to attacks.
As is clear from this article, cars in Horizon Zero Dawn can be deadly, either alone or in a group. Thanks to planning a network of hierarchical tasks and a hierarchy of intelligent agents, machines can roam the world by themselves or, if necessary, protect and support each other. That interaction with such an exciting world does not allow the player to relax.
In the second part I will talk in detail about the tools and systems used by individual AI characters: about the sensor systems used to recognize the nearest sources of threats and attack them; how AI systems should work in tandem with animations so that the machines look dynamic but real; Pro navigation systems, allowing machines of different sizes to move through the same terrain; and also about how flying machines, for example, “Thunderbirds”, use a completely unique navigation system that allows them to explore the world and hunt the player in the best possible way.
In this part of the article I will talk about the gameplay and decision-making systems that make these creatures alive and provide the basics of the gameplay. I will talk not only about how they make decisions about different actions (gathering resources, hunting, survival), but also how they are coordinated in the wide world, individually and in groups. In addition, I will talk about how the game creates an ecosystem of machines, which is balanced and controlled in all territories and in all areas of the new world.
Warning: this article contains Horizon Zero Dawn story spoilers.
Ecology Horizon Zero Dawn
Horizon Zero Dawn automatic animals dominate the world and retain control of the territory. Each machine has a unique aesthetics and structure, reflecting its functionality and purpose within the game narrative. During the passage of the storyline campaign, players will learn about the "Plague of Faro" - the army of rebellious military vehicles that destroyed the world, as well as about the race of machines designed by AI named GEYA, which was supposed to restore the planet to its natural state; their design repeats long extinct terrestrial animals.
Therefore, the functionality of the machine often keeps the secrets of its weaknesses. Players can use a visor to scan nearby machines to find out not only their type, but also vulnerabilities and weaknesses. In addition to weak points, the functionality of the machines within the narrative determines their in-game execution. Including:
- Robots such as "collector" and "cleaner" , for example, "bison" and "gum beetle" . These working-class machines are usually found in open grassy areas and cultivate the land. They are relatively gentle and often avoid confrontation, but when provoked they become deadly.
- Scout- type vehicles , for example, “ryskari” and “long-legged”, patrol the area in search of threats and warn the nearest vehicles so that they can fight or flee.
- Machines such as "transport" , for example, "hippos" and "scarabs" take resources from the robot-gatherers, which makes them valuable prey. They are slower, slower and usually protected by more agile robots.
- And, finally, combat vehicles, for example, "pilozuby" and "trapping" usually attack the player and nearby threats.
In total, in the open spaces of the game, you can find 28 unique machines, each of which has its own passive and active behaviors. Players need to learn how to find the right approach to them. Although the greatest threat on the way to Meridian (such as "Thunderbird" and "gromozev" ) roam the world alone, players can still stumble on less lethal machines running in herds. You can often see grazing robots that are grazing alongside, such as “royals” , “buffaloes” , “scavengers” and “spearheads” : they support each other in the process of performing their basic functions. Over time, these herds increase in size and become more diverse, for example, the convoys of "scrubs", flocks of "buffalo" and "pilozubov" , protecting the "ruminants" and "spearmen" .
To cope with this task, the game requires a lot of AI systems that determine the behavior of each robot, their behavior inside the herd, the collective intelligence of the herds themselves, but at the same time balancing the strength of the herds in different parts of the open world for a more diverse gameplay. The developers of Horizon Zero Dawn , the company Guerilla Games, has begun this project, not having many of the AI tools needed to implement this gameplay.
The company previously worked on the Killzone franchise., and therefore its tulchain was mainly designed for first-person shooters and could not cope with the complexities of the architecture of a large open world. Therefore, in the process of development, the studio created new AI systems and gameplay, as well as expanded existing ones. In this and subsequent articles I will tell you about the following:
- On the system of the hierarchy of agents that controls the behavior of each machine and individually, and as part of a flock or herd.
- On the expansion of the planning and execution systems originally created for Killzone, allowing each machine to make decisions.
- A set of sensor systems that allow machines to quickly and efficiently recognize threats and, if necessary, investigate them.
- On the dynamic and intelligent navigation system that provides interesting patterns of motion for ground robots that correspond to the geometry of the world and different size machines.
- On a completely separate navigation system designed for flying machines such as kites and petrels.
- Pro animation system that takes into account the structure of each machine and provides an intelligent approach to their movement and attacks.
- The system called The Collective, which controls each individual machine, active in the game, and how they are distributed among the herds.
Let's start by finding out how each AI character makes decisions and how the game controls the behavior of the herds.
Agent hierarchy
Each machine is represented in the game world as a separate AI of a non-player character, called an “agent”. But more interesting is that each herd is an agent. The Horizon Zero Dawn using hierarchy in which a machine may exist as a full agent individually, or as a child element of group agent which is used to store information about the herds.
Example of group hierarchy [from (Beij, 2017)]
The group agent does not exist in the game world physically, but is responsible for coordinating all the agents inside it. Each individual agent has its own set of sensors, allowing him to hear close sounds or notice the dangers next to him. But in addition, the group agent has a blackboard system, which stores in-game information that the machine may need. This allows the game to reduce performance costs by saving information that each individual agent does not have to calculate (for example, safe places or active patrol routes). In addition, more importantly, agents in a group can exchange information with each other. Therefore, when a reconnaissance vehicle finds a player, each vehicle can decide how to respond to this information. Thanks to this you can see
The AI system used in Horizon Zero Dawn is called a hierarchical task network planner (HTN). Unlike planning systems used in games like FEAR , which plan one action after another, the HTN scheduler generates plans composed of action macros. Each macro contains several actions in a given sequence. This is ideal for developing games, because designers can create macros of good behavior that they expect to see in a game. I previously explained how the HTN scheduler was used in Transformers: Fall of Cybertron, but it is worth noting that Guerilla used the HTN scheduler as Killzone 2, that is, it was not developed from scratch, but gradually improved over the course of several years in various studio projects.
HTN Scheduler Scheme [of (Beij, 2017)]
The HTN Scheduler is used by machines for two purposes: individual agents can request an action plan to solve a specific task, such as finding new areas, attacking a person, or fleeing. Group agents can execute plans that set goals that should be assigned to individual agents, which allows groups to actively share information with each other within the hierarchy, and potentially form new subgroups or destroy existing ones. In short, each individual machine still has a great responsibility for its own behavior and the execution of tasks assigned to it, and also uses the scheduler to make decisions. Groups retain coordination of individual machines, regroup them on the basis of events occurring in the world and, most importantly, assign roles to each machine,
The collective
So, all these cars, individually or in groups, are created on the map, but how does the game know how and where to create cars in the open world? On the game map there are selected areas in which you can expect the appearance of certain machines, either individually (in the case of “thunderstorms” and “petrels”), and in groups (“runners”, “gumlings”, “kites” and others). However, the rest of the game world is filled with herds of different machines. How does a game, when creating a specific machine, know where to place it in the world, and in which group to assign it, based on the needs of the structure of the world?
This is achieved by using a system called “The Collective”. The “team” is a supergroup: all groups of machines and individual machines exist within the team. It manages the addition of all machines to the world, keeps track of whether they are in a group, controls the transitions between groups and the ecosystem of machines so that it does not reduce overall performance.
Individual machines in the world can transmit requests to join existing groups of the open world, if they act alone. This is a useful function, given that many machines remain isolated after the remaining previous group either ran away or was destroyed. Therefore, the team can reuse isolated machines and attach them to other groups. For this, each car has a "passport". Their “passport” stores information about the car (for example, its level and type), and the team uses this data to determine whether it meets the requirements of another group, and only in this case moves the car to the group.
Herd example
Now let's learn how herds work, and how group agents are performed when interacting with the player. Herds are usually placed in a relaxed world in which they are not anxious and are simply located in a certain area of the map. A typical group consists of a mix of scouts, gatherers, and combat vehicles. Robot-gatherers are in the center, while combat and reconnaissance patrols are around to protect them.
When a flock is placed in the world, a collection of groups in this hierarchy requests a role. This role assigns to all machines the behavior that reflects their purpose. Such behaviors can be patrols in search of threats, searching for close sources of concern, attacking enemies, or simply collecting the resources of the world. Over time, these roles may change, and events force the herd to change its behavior and structure. Usually there is a limit on the number of machines that perform a certain role inside the herd, so that their structure is not unbalanced.
The role of the machines determines the tasks that the individual agent will seek to accomplish. Since the herd initially behaves relaxedly, the collectors will use the HTN scheduler and grouped data to search for the nearest resource extraction zones. In the meantime, reconnaissance and combat vehicles generate patrol paths to walk. Patrol routes are generated automatically based on local geometry, they avoid inconvenient terrain, and, more importantly, avoid walking on tall grass and other vegetation that the player can use to sneak up on the herd. However, in order to give players a chance in battle, many patrol routes are deliberately laid out alongside vegetation so that the player can set traps or crack robots easier.
Herds adjust the balance of their structure in the presence of a threat, forming groups of “flight” (flee) and “battle” (combat) [of (Beij, 2017)]
But, of course, they do not always remain in this state; when players start attacking machines, their behavior changes accordingly. For machines that are not part of the herd, it all comes down to a specific class of machine. Only a few escape when they are provoked directly. The overwhelming majority of cars attack the player, and even collecting machines, being alone, often attack the player.
However, if a player attacks a herd, the groups within the hierarchy react in the same way, but only if the herd is already alert. As mentioned above, the group agent allows you to exchange data with each machine, but they are not updated in each game cycle. Therefore, the herds do not have a “hive mind” and the player can kill the car without disturbing the rest of the group. However, if it is noticed or the attacked machine survives, it will warn the group about the attack.
At this stage, the hierarchy within the herd is shifted, the group of machines within the herd requests a new role and priority to perform the corresponding behavior. A group made up of collectors will play the role of fleeing and start a joint escape, while the group consisting of combat and reconnaissance vehicles will rebalance themselves, and the character attacking the herd will be the priority for the roles. Although in many cases it is just a lone player, such a scheme allows the system to scale for story missions in which the player is part of a group, or other dynamic events in the world when other human NPCs attack herds.
Although after this point everything becomes much more complicated, the player still has many opportunities to turn the situation around, because both the grouping systems and the combat AI systems provide an exciting and at the same time complex gameplay. This is achieved through a combat AI system that attempts to structure the battle in such a way that it is not chaotic or broken.
Formed combat groups have different roles and balanced priorities, so large herd machines are distributed into groups where possible, and smaller ones line up to support them. In addition, groups choose machines to attack a player, not only on the basis of whether the HTN planner considers that he can perform a certain behavior, but also depending on whether this attack is interesting. If all the cars simply attacked the player at the same time, then he would rather quickly suffer a painful death. Therefore, combat systems use an approach similar to that used in Halo 3 , when the balance of enemy attacks on a player is based on whether the attack will be interesting in the context of combat.
This requires an auxiliary function of selecting an action, which calculates how interesting it will be if the machine attacks. It depends on the current state of the machines, on whether the player knows that the car is near him, on the proximity to the player and on the amount of damage received and inflicted by her. This value becomes an input to the HTN planning system; his task is not only to ensure a certain complexity, but also to add a share of diversity to the battle.
But more importantly, when the HTN scheduler is asked to generate actions for the attacking groups, while the attack is chosen, the movement of the player and waiting for their turn are also distributed to other group machines. This is done intentionally, to create the opportunity for the player to either attack the passive car, or to counterattack the one that just hit it. Gradually, the player will be able to weaken them, and this is mainly due to the fact that the combat system intentionally leaves itself open to attacks.
Finally
As is clear from this article, cars in Horizon Zero Dawn can be deadly, either alone or in a group. Thanks to planning a network of hierarchical tasks and a hierarchy of intelligent agents, machines can roam the world by themselves or, if necessary, protect and support each other. That interaction with such an exciting world does not allow the player to relax.
In the second part I will talk in detail about the tools and systems used by individual AI characters: about the sensor systems used to recognize the nearest sources of threats and attack them; how AI systems should work in tandem with animations so that the machines look dynamic but real; Pro navigation systems, allowing machines of different sizes to move through the same terrain; and also about how flying machines, for example, “Thunderbirds”, use a completely unique navigation system that allows them to explore the world and hunt the player in the best possible way.
Reference materials
- Julian Berteling, 2018. “Beyond Killzone: Creating New Horizon Zero Dawn Systems”, GDC 2018.
- Arjen Beij, 2017. “The AI of Horizon Zero Dawn” , Game AI North 2017.
- Wouter Josemans, 2017. “Putting the AI back into Air: Navigating the Air Space of Horizon Zero Dawn” , Game AI North 2017