Underside client-side physics prediction

Original author: Joe Best-Rotheray
  • Transfer
image

TL; DR


I created a demo showing how to implement prediction on the client side of a player's physical movement in Unity - GitHub .

Introduction


In early 2012, I wrote a post about how-to-implement forecasting on the client side of a player's physical movement in Unity. Thanks to Physics.Simulate (), the awkward workaround I described is no longer needed. The old post is still one of the most popular on my blog, but for modern Unity, this information is already incorrect. Therefore, I am releasing the 2018 version.

What is that on the client side?


In competitive multiplayer games, you should avoid cheating as much as possible. Usually this means that a network model with an authoritarian server is used: clients send the information entered to the server, and the server turns this information into a player's movement, and then sends a snapshot of the resulting player state back to the client. This causes a delay between pressing the key and displaying the result, which is unacceptable for any active games. Client-side prediction is a very popular technique that hides a delay, predicting what the resulting movement will be and immediately showing it to the player. When the client receives the results from the server, he compares them with what the client predicted, and if they are different, the forecast was wrong and needs to be corrected.

Snapshots received from the server always come from the past regarding the predicted state of the client (for example, if data transfer from the client to the server and back takes 150 ms, then each snapshot will be delayed by at least 150 ms). As a result of this, when a customer needs to correct an incorrect prediction, he must roll back to this point in the past, and then reproduce all the information entered in between to return to where he is. If the player’s movement in the game is based on physics, then Physics.Simulate () is needed to simulate several cycles in one frame. If the player’s movement uses only Character Controllers (or capsule cast, etc.), then you can do without Physics.Simulate () - and I assume that the performance will be better.

I will use Unity to recreate the Glenn Fiedler Zen of Networked Physics network demo , which I have long liked. The player has a physical cube to which he can apply forces, pushing him into the scene. The demo simulates various network conditions, including latency and packet loss.

Getting Started


The first thing to do is turn off automatic physics simulation. Although Physics.Simulate () allows us to tell the physical system when to start the simulation, by default it performs the simulation automatically based on the fixed time delta of the project. Therefore, we will disable it in Edit-> Project Settings-> Physics , by clearing the " Auto Simulation " checkbox .

To begin, we will create a simple single-user implementation. Input is sampled (w, a, s, d to move and space to jump), and it all comes down to simple forces applied to the Rigidbody using AddForce ().

public class Logic : MonoBehaviour
{
   public GameObject player;
   private float timer;
   private void Start()
   {
      this.timer = 0.0f;
   }
   private void Update()
   {
      this.timer += Time.deltaTime;
      while (this.timer >= Time.fixedDeltaTime)
      {
         this.timer -= Time.fixedDeltaTime;
         Inputs inputs;
         inputs.up = Input.GetKey(KeyCode.W);
         inputs.down = Input.GetKey(KeyCode.S);
         inputs.left = Input.GetKey(KeyCode.A);
         inputs.right = Input.GetKey(KeyCode.D);
         inputs.jump = Input.GetKey(KeyCode.Space);
         this.AddForcesToPlayer(player.GetComponent<Rigidbody>(), inputs);
         Physics.Simulate(Time.fixedDeltaTime);
      }
   }
}


Moving player while network is not in use.

Sending input to the server


Now we need to send input to the server, which also executes this motion code, snapshot the cube state and sends it back to the client.

// client
private void Update()
{
   this.timer += Time.deltaTime;
   while (this.timer >= Time.fixedDeltaTime)
   {
      this.timer -= Time.fixedDeltaTime;
      Inputs inputs = this.SampleInputs();
      InputMessage input_msg;
      input_msg.inputs = inputs;
      input_msg.tick_number = this.tick_number;
      this.SendToServer(input_msg);
      this.AddForcesToPlayer(player.GetComponent<Rigidbody>(), inputs);
      Physics.Simulate(Time.fixedDeltaTime);
      ++this.tick_number;
   }
}

There is nothing special here, the only thing I want to draw attention to is adding the variable tick_number. It is needed so that when the server sends snapshots of the cube's state back to the client, we can find out what client tact this state corresponds to so that we can compare this state with the predicted client (which we will add a little later).

// server
private void Update()
{
   while (this.HasAvailableInputMessages())
   {
      InputMessage input_msg = this.GetInputMessage();
      Rigidbody rigidbody = player.GetComponent<Rigidbody>();
      this.AddForcesToPlayer(rigidbody, input_msg.inputs);
      Physics.Simulate(Time.fixedDeltaTime);
      StateMessage state_msg;
      state_msg.position = rigidbody.position;
      state_msg.rotation = rigidbody.rotation;
      state_msg.velocity = rigidbody.velocity;
      state_msg.angular_velocity = rigidbody.angularVelocity;
      state_msg.tick_number = input_msg.tick_number + 1;
      this.SendToClient(state_msg);
   }
}

It's simple - the server is waiting for input messages, when it receives it, it simulates a beat. Then it takes a snapshot of the resulting cube state and sends it back to the client. You can see that the tick_number in the status message is one more than the tick_number in the input message. This is done because it is intuitively more convenient for me personally to think of the “state of the player to the beat of 100” as “the state of the player at the beginning of the beat of 100” Therefore, the state of the player in tact 100, combined with the input of the player in tact 100, create a new state for the player in tact 101.

State n + Input n = State n + 1


I do not say that you should take it the same way, the main thing is the constancy of the approach.

You also need to say that I do not send these messages through a real socket, but imitate it by writing them in a queue, simulating delay and packet loss. The scene contains two physical cubes - one for the client, the other for the server. When updating the client cube, I disable the GameObject server cube, and vice versa.

However, I do not simulate network bounce and packet delivery in the wrong order, and that is why I make the assumption that each input message received is newer than the previous one. This simulation is needed in order to very easily execute the “client” and “server” in the same Unity instance so that we can combine the server and client cubes in the same scene.

You can also notice that if the input message is reset and does not reach the server, the server simulates fewer ticks than the client, and therefore will create a different state. This is true, but even if we simulated these omissions, the input could still be wrong, which would also lead to a different state. We will deal with this problem later.

It should also be added that in this example there is only one client, which simplifies the work. If we had several clients, we would need a) when calling Physics.Simulate () to check that only one player’s cube is enabled on the server or b) if the server received input from several cubes, simulate them all together.


Delay 75 ms (150 ms in both directions)
0% of lost packets
Yellow cube - server player
Blue cube - last snapshot received by the client


Everything looks good so far, but I was a bit selective with what I recorded on the video to hide the rather serious the problem.

Failure of determinism


Look now at this:


Oh, her ...

This video was recorded without wiping the packets, however, the simulations still differ with absolutely identical input. I don’t quite understand why this is happening - PhysX should be quite deterministic, so I find it striking that the simulations diverge so often. This may be due to the fact that I constantly turn on and off the GameObject cubes, that is, it is possible that the problem will decrease when using two different Unity instances. This may be a bug, if you see it in the code on GitHub, then let me know.

Anyway, incorrect forecasts are an essential fact in client-side forecasting, so let's cope with them.

Can I rewind?


The process is quite simple - when a client predicts movement, it saves a state buffer (position and rotation) and input. After receiving a status message from the server, it compares the received state with the predicted state from the buffer. If they differ by too much, we redefine the state of the client cube in the past, and then simulate again all intermediate measures.

// client
private ClientState[] client_state_buffer = new ClientState[1024];
private Inputs[] client_input_buffer = new Inputs[1024];
private void Update()
{
   this.timer += Time.deltaTime;
   while (this.timer >= Time.fixedDeltaTime)
   {
      this.timer -= Time.fixedDeltaTime;
      Inputs inputs = this.SampleInputs();
      InputMessage input_msg;
      input_msg.inputs = inputs;
      input_msg.tick_number = this.tick_number;
      this.SendToServer(input_msg);
      uint buffer_slot = this.tick_number % 1024;
      this.client_input_buffer[buffer_slot] = inputs;
      this.client_state_buffer[buffer_slot].position = rigidbody.position;
      this.client_state_buffer[buffer_slot].rotation = rigidbody.rotation;
      this.AddForcesToPlayer(player.GetComponent<Rigidbody>(), inputs);
      Physics.Simulate(Time.fixedDeltaTime);
      ++this.tick_number;
   }
   while (this.HasAvailableStateMessage())
   {
      StateMessage state_msg = this.GetStateMessage();
      uint buffer_slot = state_msg.tick_number % c_client_buffer_size;
      Vector3 position_error = state_msg.position - this.client_state_buffer[buffer_slot].position;
      if (position_error.sqrMagnitude > 0.0000001f)
      {
         // rewind & replay
         Rigidbody player_rigidbody = player.GetComponent<Rigidbody>();
         player_rigidbody.position = state_msg.position;
         player_rigidbody.rotation = state_msg.rotation;
         player_rigidbody.velocity = state_msg.velocity;
         player_rigidbody.angularVelocity = state_msg.angular_velocity;
         uint rewind_tick_number = state_msg.tick_number;
         while (rewind_tick_number < this.tick_number)
         {
            buffer_slot = rewind_tick_number % c_client_buffer_size;
            this.client_input_buffer[buffer_slot] = inputs;
            this.client_state_buffer[buffer_slot].position = player_rigidbody.position;
            this.client_state_buffer[buffer_slot].rotation = player_rigidbody.rotation;
            this.AddForcesToPlayer(player_rigidbody, inputs);
            Physics.Simulate(Time.fixedDeltaTime);
            ++rewind_tick_number;
         }
      }
   }
}

Buffered input data and status are stored in a very simple circular buffer, where the tick identifier is used as the index. And I chose the value of 64 Hz for the clock frequency of physics, that is, the buffer for 1024 elements gives us space for 16 seconds, and this is much more than what we might need.


Correction included!

Transfer excess input


Input messages are usually very small - the pressed buttons can be combined into a bit field that takes only a few bytes. In our message, there is still a clock number that takes 4 bytes, but we can easily compress them using the 8-bit value with the carry (perhaps, the 0-255 interval will be too small, we can insure and increase it to 9 or 10 bits). Anyway, these messages are quite small in size, which means that we can send a lot of input data in each message (in case previous input data was lost). How far back should we go? Well, the client knows the clock number of the last status message he received from the server, so there’s no point going back further than that clock. We also need to impose a limit on the amount of redundant input data, which are sent by the client. In my demo, I did not do it, but it should be implemented in the finished code.

while (this.HasAvailableStateMessage())
{
   StateMessage state_msg = this.GetStateMessage();
   this.client_last_received_state_tick = state_msg.tick_number;

This is a simple change; the client simply records the tick number of the last status message received.

Inputs inputs = this.SampleInputs();
InputMessage input_msg;
input_msg.start_tick_number = this.client_last_received_state_tick;
input_msg.inputs = new List<Inputs>();
for (uint tick = this.client_last_received_state_tick; tick <= this.tick_number; ++tick)
{
   input_msg.inputs.Add(this.client_input_buffer[tick % 1024]);
}
this.SendToServer(input_msg);

The input message sent by the client now contains a list of input data, not just one item. The part with the beat number receives a new value - now it is the measure number of the first entry in this list.

while (this.HasAvailableInputMessages())
{
   InputMessage input_msg = this.GetInputMessage();
   // message contains an array of inputs, calculate what tick the final one is
   uint max_tick = input_msg.start_tick_number + (uint)input_msg.inputs.Count - 1;
   // if that tick is greater than or equal to the current tick we're on, then it
   // has inputs which are new
   if (max_tick >= server_tick_number)
   {
      // there may be some inputs in the array that we've already had,
      // so figure out where to start
      uint start_i = server_tick_number > input_msg.start_tick_number ? (server_tick_number - input_msg.start_tick_number) : 0;
      // run through all relevant inputs, and step player forward
      Rigidbody rigidbody = player.GetComponent<Rigidbody>();
      for (int i = (int)start_i; i < input_msg.inputs.Count; ++i)
      {
         this.AddForcesToPlayer(rigidbody, input_msg.inputs[i]);
         Physics.Simulate(Time.fixedDeltaTime);
      }
      server_tick_number = max_tick + 1;
   }
}

When the server receives an input message, it knows the number of the first input cycle and the amount of input data in the message. Therefore, it can calculate the time period of the last input in the message. If this last clock is greater than or equal to the server clock number, then it knows that the message contains at least one input that the server has not yet seen. If so, it simulates all new input data.

You may have noticed that ifwe limit the amount of redundant input data in the input message, then with a sufficiently large number of lost input messages, we will have a simulation gap between the server and the client. That is, the server can simulate a cycle of 100, send a status message to start cycle 101, and then receive an input message starting at cycle 105. In the above code, the server will go to 105, it will not try to simulate intermediate cycles based on the latest known input data. Whether you need this depends on your decision and on what the game should be. Personally, I would not force the server to speculate and move a player around the map due to the poor network condition. I think it is better to leave the player in place until the connection is restored.

In the “Zen of Networked Physics” demo there is a function for the client to send “important moves”, that is, it sends redundant input data only when it differs from the input previously transmitted. This can be called delta-compression input, and with it, further reduce the size of input messages. But so far I have not done this, because there is no network load optimization in this demo.


Before sending redundant input data: if 25% of packets are lost, the cube movement is slow and jerky, it continues to be thrown back.


After sending redundant input data: with a loss of 25% of the packets, there is still a jerky correction, but the cubes move at an acceptable speed.

Variable snapshot frequency


In this demo, the frequency with which the server sends snapshots to the client varies. With a reduced frequency, the client will need more time to get a correction from the server. Therefore, when the client makes a mistake in the forecast, then before receiving the status message, he may deviate even more, which will lead to a more noticeable correction. With a high frequency of snapshots, the loss of packets is much less important, so the client does not have to wait long to receive the next snapshot.


Snapshot frequency 64 Hz


Snapshot frequency 16 Hz


Snapshot frequency of 2 Hz

Obviously, the higher the snapshot frequency, the better, so you should send them as often as possible. But it still depends on the amount of additional traffic, its cost, the availability of dedicated servers, the computational costs of servers, and so on.

Smoothing correction


We create incorrect predictions and get a bit of a correction more often than we would like. Without proper access to the Unity / PhysX integration, I’m almost unable to debug these erroneous predictions. I said this before, but I will repeat it again - if you find something related to physics, in which I am mistaken, then let me know about it.

I bypassed the solution to this problem, smearing the cracks with the good old smoothing! When a correction occurs, the client simply smoothes the position and turn of the player towards the correct state for several frames. The physical cube itself is corrected instantly (it is invisible), but we have a second display-only cube that allows smoothing.

Vector3 position_error = state_msg.position - predicted_state.position;
float rotation_error = 1.f - Quaternion.Dot(state_msg.rotation, predicted_state.rotation);
if (position_error.sqrMagnitude > 0.0000001f ||
   rotation_error > 0.00001f)
{
   Rigidbody player_rigidbody = player.GetComponent<Rigidbody>();
   // capture the current predicted pos for smoothing
   Vector3 prev_pos = player_rigidbody.position + this.client_pos_error;
   Quaternion prev_rot = player_rigidbody.rotation * this.client_rot_error;
   // rewind & replay
   player_rigidbody.position = state_msg.position;
   player_rigidbody.rotation = state_msg.rotation;
   player_rigidbody.velocity = state_msg.velocity;
   player_rigidbody.angularVelocity = state_msg.angular_velocity;
   uint rewind_tick_number = state_msg.tick_number;
   while (rewind_tick_number < this.tick_number)
   {
      buffer_slot = rewind_tick_number % c_client_buffer_size;
      this.client_input_buffer[buffer_slot] = inputs;
      this.client_state_buffer[buffer_slot].position = player_rigidbody.position;
      this.client_state_buffer[buffer_slot].rotation = player_rigidbody.rotation;
      this.AddForcesToPlayer(player_rigidbody, inputs);
      Physics.Simulate(Time.fixedDeltaTime);
      ++rewind_tick_number;
   }
   // if more than 2ms apart, just snap
   if ((prev_pos - player_rigidbody.position).sqrMagnitude >= 4.0f)
   {
      this.client_pos_error = Vector3.zero;
      this.client_rot_error = Quaternion.identity;
   }
   else
   {
      this.client_pos_error = prev_pos - player_rigidbody.position;
      this.client_rot_error = Quaternion.Inverse(player_rigidbody.rotation) * prev_rot;
   }
}

When an erroneous forecast occurs, the client tracks the difference in position / rotation after correction. If the total distance of the position correction is more than 2 meters, then the cube just moves with a jerk - the anti-aliasing would still look bad, so let it at least return to the correct state as soon as possible.

this.client_pos_error *= 0.9f;
this.client_rot_error = Quaternion.Slerp(this.client_rot_error, Quaternion.identity, 0.1f);
this.smoothed_client_player.transform.position = player_rigidbody.position + this.client_pos_error;
this.smoothed_client_player.transform.rotation = player_rigidbody.rotation * this.client_rot_error;

In each frame, the client performs lerp / slerp in the direction of the correct position / rotation of 10%, this is the standard power-law approach to motion averaging. It depends on the frame rate, but for the purposes of our demo this is quite enough.


Delay 250 ms.
Loss of 10% of packets.
Without smoothing, correction is very noticeable.



Delay 250 ms.
Loss of 10% of packets.
With smoothing, the correction is much more difficult to notice.


The end result works quite well, I want to create a version of it that will actually send packets rather than simulate them. But at the very least, this is a proof of the concept of a client-side forecasting system with real physical objects in Unity without the need to use physical plug-ins and the like.

Also popular now: