Artificial intelligence in flight simulators

In articles and interviews devoted to the creation of games, the term “artificial intelligence” or AI is often mentioned. Typically, such descriptions are limited to phrases such as “realistic AI” or “advanced AI”. However, for the simulator, namely about the flight simulator, we will discuss below, AI plays a role no less than the physical model of the aircraft.



First, a little history and general concepts. It should be noted that the fundamental differences in the design of the AI ​​of different flight simulators are rather difficult to find, since the best methods for solving such problems have been known for a long time and are used everywhere. AI pilot, as well as any other AI "driver of a technical tool" - this is the so-called "state machine". The entire volume of AI tasks to be solved is divided into such a number of main modes (states) in which the behavior of AI in each of the modes can be described by its rather simple algorithm with a minimum number of conventions. For a pilot of the Second World War, this is about 20-30 “states” such as “taxiing along the airfield”, “take-off”, “flight to the point”, “flight behind the leader”, “search, selection and distribution of air targets”, “approach attack air target "," attack air target "and so on.

Each “state” also has a set of conditions - when an input occurs and a set of parameters for the output.
Inside each “state” there is an algorithm for controlling the aircraft, its systems and weapons. It depends on a number of external conditions. For example, in the state of “approaching an attack of a ground target” there are five main decision-making levels: the
AI builds a flight path and a graph of the speed change along it depending on its position, as well as the position of the ground target.
AI controls the control knob, pedals and throttle stick depending on the deviation from the constructed path and the set speed, taking into account the current speed and altitude, as well as the characteristics of the aircraft.

The AI ​​continues to evaluate the situation around itself and, in the event of a threat or premature loss of goals, as well as the emergence of more priority goals, it leaves the “state of approach” and goes into the state that will best suit the current situation.

The AI ​​continues to monitor the commands coming from the mission script system, and in the event of a new command, it will cease to carry out the attack, will switch to the state corresponding to the execution of the newly received command.
AI monitors the state of the aircraft. For example, if the remainder of the fuel or weapon does not allow him to continue the attack, then he will leave this state and go into the "state of return to base."

As already noted above, such an approach is applied to one degree or another in virtually all flight sims. In this regard, the question arises - if we compare different projects, why is the difference in the general logic of what AI does in the game so noticeable? As usual, "the devil is in the details." The general approach to the architecture of AI "settled down" in the industry many years ago and is not much different in different projects. However, the internal decision-making algorithms within these "states", as well as the conditions for switching between states in each project, are made in their own way, and it is this level of AI implementation that determines the differences between the behavior of AI aircraft in different projects.

Aircraft AI in Rise of Flight (RoF).

The AI ​​for the RoF project was basically made in the image of the predecessors. However, we needed to bring some zest that would distinguish our AI from others for the better.

AI in RoF controls the same plane as the player controls, without simplifying the physical model. This is pleasant for the user in that the AI ​​planes, like the player’s plane, move “as if alive,” they also behave in a realistic manner during take-off / landing, maneuvering and when damage is received. The player does not feel that “AI twists me because it flies on rails,” and vice versa, does not feel that “AI is too easy to twist because it flies using a simplified physical model.” All honestly, and the players felt it.
Particular emphasis was placed on an integrated approach to the logic of the "state of search, selection and distribution of air targets." The selection algorithm was made very branched, with many additional conditions. They are aimed at enhancing the influence of the air situation around the AI-plane, at making decisions on the choice of targets and their distribution among the followers.



I would like to dwell in more detail on the second feature of our AI. On the algorithm for choosing air targets. First, we started by making an expert assessment of how goals were selected in life. Moreover, for both an experienced pilot and a beginner. To do this, we invited a couple of distinguished players in the IL-2: Sturmovik and a historical consultant. For several hours they were interrogated with a predilection about what factors of the air combat situation influence the choice of the target and how they do it. It is clear that the main problem in this case is the choice of a "reference point", a measure, with which all other factors of the combat situation could be compared. With the help of such a comparison it would be possible to compare them among themselves.

The approximation time was chosen as such an evaluation criterion. A series of questions followed, for example: “Imagine, expert, here are two goals for you. Before intercepting one, you need to fly a minute, and to the second - two minutes (a minute more), but the second is faster (that is, more dangerous). How much faster should it be, so that despite being further away, you (or a real pilot) would still attack a more distant, but faster target. ” To which the expert replied “well ... I would fly to attack a faster target, which is a minute further than a slower one, only if the difference in target speeds would be more than 100 km / h.” Thus, we obtained a comparison of the importance of the difference in target speeds and the difference in interception time. Further, in the same way, comparisons of priorities with respect to the interception time were compiled for all other parameters, among which:

- The difference in maximum speeds of targets at their current altitudes.
- The difference in the maximum climb rates of targets at their current altitudes.
- The difference in the heights of the goals.
- The difference in the number of other enemy aircraft in the target area.
- The difference in the class of targets (fighter, reconnaissance, bomber).

Having processed the expert assessments of several people in a standard way, having adjusted these estimates for different levels of AI complexity, we got a standard method for sorting goals according to their priority. If we take that the difference in the interception time of 60 seconds gives +1 to the “priority of the target” piggy bank, then the difference of 120 seconds will give +2. An expert comparison suggests that if 60 seconds is +1, then the difference in maximum speeds of 100 km / h is also +1, and the difference of 50 km / h is +0.5. And so on in all respects.
Next, the target is selected from the list of all targets visible by the AI ​​pilot: first, those that fall under the filtering conditions are excluded:

- Targets that take too long to intercept (because of their long range, or because of their high altitude, or because they are very fast)
- Targets around which there are so many more enemy fighters than Allied fighters around the player’s plane. Thus, the battle with them is suicidal (this condition mainly affects AI aces).
- Targets are higher than the maximum flight altitude of an AI pilot.
- Targets, significantly higher than the AI-pilot's aircraft, whose climb rate is significantly higher.

After filtering "unpromising" goals, they are sorted by priority. For this, each goal that has passed through the filter is taken and compared with all other goals for all of the above expert parameters. The sum of the parameters is a comparison of priority between a pair of goals. Then, for each goal, all paired estimates for those pairs in which it participates are summarized. Next, the goals are sorted by final amounts, from larger to smaller.

Thus, we have a list of goals, sorted by their priority. Next, you need to assign goals as followers. This logic depends on the level of AI and the number of targets. A high-level AI leader with superiority in number and in height will send only part of the followers into battle, and with the rest he will cover them at high altitude, in case additional opponents appear. A low-level AI leader will simply send one plane of his group for each target, starting with priority ones. A high-level AI leader will send at least two planes per target. Well and so on.

AI slaves also monitor the situation. Even if the attack of the target designated by the leader occurs, in the event of the appearance of another adversary that poses a serious threat to the follower himself, they will switch to the new target. The followers also monitor the situation around the leader - if he is in danger from opponents who are not assigned to attack, they will switch to attack these enemies to cover the leader. The followers stop attacking the assigned target if they are too far removed from the leader during the attack and return to it.
As a result, RoF received the logic of target selection by the AI ​​leader of the unit, which is closest to the real practice of air combat among all the analogues presented at that time. Moreover, this logic turned out to be transparent and easily corrected, simply by changing the values ​​of the above “paired parameters”. In addition, RoF implements many interesting features of AI behavior: anti-aircraft maneuvers, a target visibility accounting system that takes into account illumination, cloud cover, and floodlight illumination. At the same time, the illumination of the target is also affected by light sources on the target itself, such as air navigation lights, machine gun firing flashes, and fires.
It should be noted that some users took some of these "features" for bugs. For example, when the AI ​​leader of the player’s link did not want to attack significantly superior enemy forces. In this case, the AI ​​leader simply did not want to get involved in a battle in which he was unlikely to survive, as well as his link. Let's take another situation when the AI ​​leader did not want to attack the scout going directly above the head of the player’s link at high altitude. In fact, the AI ​​leader simply estimated that the time to climb up to this scout was so long that by the time this was accomplished, he would most likely fly to his territory. In these cases, we had to explain that "this is a model of behavior that is close to reality, high-level AI pilots are by no means stupid and try not to get involved in adventures." Although, of course, with such a high level of AI complexity, as in RoF, problems could not be avoided. We systematically isolate and correct them. Special thanks I want to say to Alexander Petryaev, the AI ​​programmer who created the software architecture and code, that for 4 and a half years they have been serving in the RoF project, which will also serve as the basis for AI pilots in the new project of the IL-2: Sturmovik series for many more years ".

AI aircraft in the "Battle of Stalingrad" (BZS).

It should be noted that the pilot’s AI in RoF was originally developed with a long-range aim for WWII aircraft. With minimal changes, the AI ​​RoF could be used in IL-2: BzS. However, we decided that the next step forward in this area was necessary and chose the point where, in our opinion, such changes would be most in demand. In RoF, when designing and developing AI, we focused on the logic of searching and selecting targets, and in BoS, we decided to focus on combat maneuvering during close air combat. In RoF, in the “state of attack of an air target”, AI fighters always always perform one universal maneuver - they turn on the target in the shortest 3-dimensional arc, wherever the enemy is (behind, on the right, from above, from below). For World War I, this approach was justified for the reason that air combat on planes of that period is very compact, fleeting. Turning time is very short, maneuvering occurs at speeds sometimes less than 100 km / h and it is very difficult to decompose it into “maneuvers”.
However, during the Second World War, the picture of the air confrontation is radically different: the battle became more widespread, the distance between the planes during maneuvering is much greater, much higher than speed, much longer is the turnaround time. In this situation, the structuring of maneuverable combat becomes more obvious, it becomes possible to formalize this structuring. Our new AI programmer, Ilya aka "naryv" Steshov, took up this task. I must say that, first of all, he was engaged in the development and debugging of AI in the RoF project. All the changes that took place in AI this spring are his merit and the result of his activities. In addition, to draw up the terms of reference, experts were again invited to give advice on combat maneuvering - what maneuvers were used, in which cases they were used, how the level of the pilot influenced the use of certain maneuvers; what were the conditions for the beginning and termination of maneuvers. Having received an expert assessment, we have chosen the following way to solve the problem: “the state of attack of an air target” has become a “macro level”, revealing inside itself into a “state machine” of the lower level. Each maneuver of close air combat was designed as a separate “state” of this new low-level machine. At the same time, the logic of the “macro state of attack of an air target” continued to function with all the conditions of entry and exit, which allowed us to make this innovation within the framework of the RoF AI architecture without serious labor costs for architectural changes in the upper level of the “state machine”, as well as the rest of the AI ​​logic. What maneuvers and when does the AI ​​fighter use in close combat in the BZS: what were the conditions for the beginning and termination of maneuvers. Having received an expert assessment, we have chosen the following way to solve the problem: “the state of attack of an air target” has become a “macro level”, revealing inside itself into a “state machine” of the lower level. Each maneuver of close air combat was designed as a separate “state” of this new low-level machine. At the same time, the logic of the “macro state of attack of an air target” continued to function with all the conditions of entry and exit, which allowed us to make this innovation within the framework of the RoF AI architecture without serious labor costs for architectural changes in the upper level of the “state machine”, as well as the rest of the AI ​​logic. What maneuvers and when does the AI ​​fighter use in close combat in the BZS: what were the conditions for the beginning and termination of maneuvers. Having received an expert assessment, we have chosen the following way to solve the problem: “the state of attack of an air target” has become a “macro level”, revealing inside itself into a “state machine” of the lower level. Each maneuver of close air combat was designed as a separate “state” of this new low-level machine. At the same time, the logic of the “macro state of attack of an air target” continued to function with all the conditions of entry and exit, which allowed us to make this innovation within the framework of the RoF AI architecture without serious labor costs for architectural changes in the upper level of the “state machine”, as well as the rest of the AI ​​logic. What maneuvers and when does the AI ​​fighter use in close combat in the BZS: The “state of attack of an air target" has become a "macro level", revealing within itself into a "state machine" of the lower level. Each maneuver of close air combat was designed as a separate “state” of this new low-level machine. At the same time, the logic of the “macro state of attack of an air target” continued to function with all the conditions of entry and exit, which allowed us to make this innovation within the framework of the RoF AI architecture without serious labor costs for architectural changes in the upper level of the “state machine”, as well as the rest of the AI ​​logic. What maneuvers and when does the AI ​​fighter use in close combat in the BZS: The “state of attack of an air target" has become a "macro level", revealing within itself into a "state machine" of the lower level. Each maneuver of close air combat was designed as a separate “state” of this new low-level machine. At the same time, the logic of the “macro state of attack of an air target” continued to function with all the conditions of entry and exit, which allowed us to make this innovation within the framework of the RoF AI architecture without serious labor costs for architectural changes in the upper level of the “state machine”, as well as the rest of the AI ​​logic. What maneuvers and when does the AI ​​fighter use in close combat in the BZS: At the same time, the logic of the “macro state of attack of an air target” continued to function with all the conditions of entry and exit, which allowed us to make this innovation within the framework of the RoF AI architecture without serious labor costs for architectural changes in the upper level of the “state machine”, as well as the rest of the AI ​​logic. What maneuvers and when does the AI ​​fighter use in close combat in the BZS: At the same time, the logic of the “macro state of attack of an air target” continued to function with all the conditions of entry and exit, which allowed us to make this innovation within the framework of the RoF AI architecture without serious labor costs for architectural changes in the upper level of the “state machine”, as well as the rest of the AI ​​logic. What maneuvers and when does the AI ​​fighter use in close combat in the BZS:

- "Turnaround and shooting." This is the main mode, generally similar to the guidance algorithm used in RoF. It is used when the target is close to the aiming line and you need to turn it on and shoot.
“The Candle.” The plane abruptly goes up at maximum engine operation. It is used to save energy in case of large speeding when the enemy “skips”. Converts aircraft speed to altitude.
- "Half loop". The plane performs half of the "Nesterov Loop". The purpose of the application is the same as that of the “Candle”.
- "Hill". The plane gains altitude in a straight, rather steep path (but not vertical). It is used when the target “skips” with moderate speeding. In the case when the attack was made from the side or “on the forehead”. The purpose of the application is again the same - to save energy by translating speed into height.
- “U-Turn”. The plane climbs in a steep spiral with a loss of speed. The purpose of the application is the same as that of the Gorka, but can also be used after attacking the target from behind.
“The Ascending Spiral.” The aircraft performs a turn at maximum gas toward the target, at a constant speed and with climb. It is used in cases when in the process of attack the target is higher than more than 100-200 meters and to the side of us (off course).
“The downward spiral.” The aircraft performs a turn at maximum gas toward the target with a decrease and a set of speed. It is used when the target is lower than more than 100-200 meters and to the side of us (off course). It can also be used to try to throw the enemy off the tail with a sufficient margin, because very high and continuous overloads develop in downward spirals.
- "Forced U-turn." The aircraft performs a horizontal turn at maximum gas and overload. It is used when the target is at approximately the same height as our plane, on our side, but in the front hemisphere.
- "Turn". The aircraft performs a horizontal turn at maximum gas, overloading maintaining a constant optimal speed of the turn. It is used in all other cases.
- "High Yo-Yo." When performing a “turn”, the aircraft slightly reduces the roll for a short time, slightly gaining altitude due to some loss of speed. This is a submode of the bend maneuver. It is used when, at the beginning of a turn, the speed turned out to be much higher than optimal, and in order not to waste energy, this speed is translated into an advantage in height.
- "Low Yo-Yo." During a turn, the aircraft slightly increases the roll for a short time, gaining a little speed due to loss of altitude. This, in turn, allows you to quickly accelerate the turn on the target by increasing the overload. This sub-turn maneuver is used when the angle to the target does not change for a long time (when the planes cannot go around each other in the turn).
“The coup.” The aircraft "rolls over on its back" and performs the second half of the "Nesterov Loop" with maximum overload on maximum gas. It is used when it is necessary to throw the enemy off the tail and there is a sufficient reserve of height.
- "Slipping." The plane deflects the rudder, continuing the flight in a straight line. It is used to create a false impression of the direction of flight for an enemy performing aiming and shooting from a long distance.
“The Scissors.” The aircraft slows down and performs successive turns to the right and left. It is used to throw the enemy off the tail when the headroom is insufficient and when the AI ​​pilot’s plane is more stable at low speeds than the enemy’s plane.
- “Corkscrew barrel.” The AI ​​pilot “undermines” the plane up and simultaneously rolls it along the roll. It is also used to drop the enemy from the tail. The maneuver is very complex and is available only to AI-asam.
- "Defensive dovor." AI pilot keeps the enemy on a perpendicular course, constantly gaining height and falling. It is used in cases when the enemy carries out an attack from a long distance in order to make it difficult to aim.



In addition to creating a new, branched system for choosing further maneuvers in close air combat, depending on the level of AI, we carried out a number of works aimed at increasing the realism of the behavior of an AI fighter in an air battle:

- A system was introduced to limit the visibility of targets for AI with a cockpit design. It is tuned uniquely for each aircraft and takes into account not only the possible turns of the pilot’s head in the cockpit, but also possible movements (harassment, squats, peeping).
- In the mode of conducting air combat, a submode of searching for a target that was lost for any reason was introduced: the AI ​​pilot puts the plane into a turn and tries to see the target again; if it didn’t work right away, then he shifts the turn in the other direction. So until he decides that he has finally lost the target.
- Also in the air combat mode, when the target turned out to be too high, the climb mode along the optimal path was added so that the AI ​​pilot would try to catch the target in height in the most optimal way.
- When evading enemy fighters, attack aircraft and bombers now perform not only a “snake”, but also dive when the height and distance to the enemy allow it.
“We also created the functionality of an advanced aircraft pilot who can assess the situation in the area of ​​the battlefield entrusted to him and direct fighters and attack aircraft at targets using radio communications. At the same time, AI planes themselves have all the skills for interaction and radio communication with the gunner.
- In addition to the existing logic of attacking ground targets, AI planes will be trained to use unguided missiles from a dive, and Ju-87 and Pe-2 will perform bombing from a vertical dive.
- In view of the increased speeds and variety of small arms, the system of counting and maintaining the lead in firing at air targets was improved. AI pilots now take into account not only the parameters of the target’s movement, but also the differences in ballistics of various cannon and machine gun weapons.

We also managed to achieve the following:

- AI pilots were trained to conduct radio communications with the airfield and with their followers / leaders.
- AI planes were also trained to taxi from the parking lot to the runway for takeoff and from the strip to the parking lot after landing.
- AI planes now know and are able to use aircraft systems that were absent during World War I but appeared in World War II: retractable landing gears, flaps and landing flaps, tail wheel lock, aeronautical lights, landing lights, cabin lighting, an openable cabin light, the possibility of leaving the aircraft with a parachute, air brakes, wheel brakes, a controlled supercharger and a speed regulator.

It is important to note that almost all the new functionality is ready. You can personally verify this by participating in the recently launched early access. The AI ​​system already in RoF was quite advanced. In BzS, it has become even more complicated, which will inevitably lead to some bugs, but there is enough time for debugging. By the release, we expect to bring the performance of AI pilots to a very high level of quality.
Someone may ask, “Why do you need all this?” The answer to this question is quite simple - we strive to make the picture of air combat as plausible as possible, not only due to a detailed and realistic flight model, but also due to the ramified, complex and realistic logic of behavior of AI allies and AI opponents. When creating AI in a flight simulator, it is extremely important to try to find a fragile balance between excessive "robotization" and excessive "fooling" of the enemy AI. It is unlikely that there will be many who want to compete with the ideal “killing machine” that does not make mistakes, as well as shoot dumb-headed boobies who themselves climb under bullets. The task before us is complex and extremely interesting - to make the AI ​​pilot as humanized as possible and close to reality.
Looking back, one can say without false modesty that we have already succeeded in something. And it worked out well. However, we do not stand still - we are constantly developing, setting new tasks and finding ways to solve them. We are looking for like-minded people and associates. Join now!

Daniil “Han” Tuseyev, Project Manager, IL-2 Sturmovik: Battle for Stalingrad.