April 5, 2018 at 15:05Taming floating point numbers. Is it possible to debug shaders for mobile devices on a PC?
In the gaming industry, work on the visual quality of effects takes place at the intersection of art and technology. This is especially true for mobile applications.
In this article, we will talk about using floating point numbers when debugging shaders for mobile devices on a PC. We are sure that the experience of the Krasnodar Plarium studio will be useful for you, regardless of which engine you use.
How we identified the problem
In the process of creating effects, we constantly faced with the fact that some of them eventually crumble into pixels.
We investigated this problem and determined the causes of its occurrence: it turned out that not all gadgets are susceptible to artifacts. This mainly applies to older mobile devices or models with poor performance. Moreover, some seemingly the same effects either worked as they should, then lost the smoothness of movements and acquired pixelation.
During the analysis of technical documentation of video processors of devices at risk, we found that not all gadgets have full support for 32-bit floating-point numbers. This is what causes the artifact and image problems.
And what about the binary?
Recall how floating-point numbers look in a binary system.
You can imagine such a number as a container for storing a fixed number of significant digits and a comma anywhere in this number. Obviously, larger numbers have less accuracy.
For example, we have a significant number - 123456789. If we put a comma after the first character (1.23456789), then we can change this number to the nearest 0.00000001. But if you put a comma before the last character (12345678.9), then the accuracy will be 0.1.
If we use 32-bit numbers, then any such number accurately conveys 7 characters. This allows the application to run for about 70 hours without visible problems. It is about the time when the user sees the running game, and not on a pause or in a minimized state. Big enough stock, isn't it?
But the fact is that many mobile devices only support 16-bit floating-point numbers in a pixel (fragment) shader. In such numbers, there are only 4 exact signs, which leads to a noticeable loss of accuracy when animating textures in 10-15 minutes.
Looking for a solution:
It is unrealistic to solve the problem on each specific device, so we focused on visualizing the error on the PC when creating the effect and taking measures to eliminate it in advance.
On a PC, there is no way to directly emulate the 16-bit mode. The video card driver automatically converts all 16-bit instructions to 32 bits, and you can visually track the loss of accuracy only after a long time. Therefore, to visualize artifacts from loss of accuracy, we wrote our own time and extension manager for shaders.
Time manager
Our “debugging” time starts a few hours ahead of schedule. Thus, the artist can immediately see problems with the effect. Also, the manager automatically resets the time each time the application is paused or minimized. It is at this moment that the jump in effects is the least noticeable to the player.
With the advent of the Scriptable Rendering Pipeline process control system in the new version of Unity, we were able to abandon a separate time management script and do everything directly when creating the frame.
Shader Extensions
The idea was to overload a 32-bit floating-point number in the same way that 16-bit was overloaded. It was necessary to shift a significant part by a certain number of bits. But bit operations are not available for Shader Model 2.0, and in addition, the use of a higher shader model leads to the engine ceasing to warn effects artists about too complex calculations for mobile devices and the effect budget is exceeded.
We decided to make the most accurate imitating algorithm, which will allow us to obtain the desired result.
Thus, changing the shader a bit, we get a visually identical image for weak devices. This allows our artists to control the process of developing effects and reduces the number of errors in the work.
Hint: frac to help.
When working on effects, strive to ensure that all numbers responsible for the smoothness of the effect and requiring accuracy are in the range from 0 to 1. Use for this the operation of cutting off the integer part - frac (x).
In essence, frac is the casting of a floating-point number to an integer, followed by subtracting this integer from the original. That is, frac = x - floor (x).
Since this operation uses the original number in its original form, loss of accuracy can occur at the stage of transferring this number inside the shader. Therefore, we recommend using a time that does not increase to large values. It is good practice to force time to zero at boot times or at times when the game interface overlaps the game level.
The work done has significantly reduced the number of visual errors. Our artists got the opportunity to control the process of creating effects without a long and complicated test on mobile devices.
In this article, we will talk about using floating point numbers when debugging shaders for mobile devices on a PC. We are sure that the experience of the Krasnodar Plarium studio will be useful for you, regardless of which engine you use.
How we identified the problem
In the process of creating effects, we constantly faced with the fact that some of them eventually crumble into pixels.
We investigated this problem and determined the causes of its occurrence: it turned out that not all gadgets are susceptible to artifacts. This mainly applies to older mobile devices or models with poor performance. Moreover, some seemingly the same effects either worked as they should, then lost the smoothness of movements and acquired pixelation.
During the analysis of technical documentation of video processors of devices at risk, we found that not all gadgets have full support for 32-bit floating-point numbers. This is what causes the artifact and image problems.
And what about the binary?
Recall how floating-point numbers look in a binary system.
You can imagine such a number as a container for storing a fixed number of significant digits and a comma anywhere in this number. Obviously, larger numbers have less accuracy.
For example, we have a significant number - 123456789. If we put a comma after the first character (1.23456789), then we can change this number to the nearest 0.00000001. But if you put a comma before the last character (12345678.9), then the accuracy will be 0.1.
If we use 32-bit numbers, then any such number accurately conveys 7 characters. This allows the application to run for about 70 hours without visible problems. It is about the time when the user sees the running game, and not on a pause or in a minimized state. Big enough stock, isn't it?
But the fact is that many mobile devices only support 16-bit floating-point numbers in a pixel (fragment) shader. In such numbers, there are only 4 exact signs, which leads to a noticeable loss of accuracy when animating textures in 10-15 minutes.
Looking for a solution:
- We divided the reasons for the loss of accuracy into three groups:
- When transmitting - the value is converted from 32 bits to 16 bits.
- In the calculation, an attempt is made to operations with numbers of different orders.
- At overflow - any floating-point number has its maximum and minimum value.
It is unrealistic to solve the problem on each specific device, so we focused on visualizing the error on the PC when creating the effect and taking measures to eliminate it in advance.
On a PC, there is no way to directly emulate the 16-bit mode. The video card driver automatically converts all 16-bit instructions to 32 bits, and you can visually track the loss of accuracy only after a long time. Therefore, to visualize artifacts from loss of accuracy, we wrote our own time and extension manager for shaders.
Time manager
Our “debugging” time starts a few hours ahead of schedule. Thus, the artist can immediately see problems with the effect. Also, the manager automatically resets the time each time the application is paused or minimized. It is at this moment that the jump in effects is the least noticeable to the player.
With the advent of the Scriptable Rendering Pipeline process control system in the new version of Unity, we were able to abandon a separate time management script and do everything directly when creating the frame.
Shader Extensions
The idea was to overload a 32-bit floating-point number in the same way that 16-bit was overloaded. It was necessary to shift a significant part by a certain number of bits. But bit operations are not available for Shader Model 2.0, and in addition, the use of a higher shader model leads to the engine ceasing to warn effects artists about too complex calculations for mobile devices and the effect budget is exceeded.
We decided to make the most accurate imitating algorithm, which will allow us to obtain the desired result.
- Add some large number to the original. This large number itself is determined by the number of characters of the main number.
- Subtract it. You need to trick the compiler, which will automatically reduce the operation. To do this, we introduce a slight error in the number after the increase.
Thus, changing the shader a bit, we get a visually identical image for weak devices. This allows our artists to control the process of developing effects and reduces the number of errors in the work.
Hint: frac to help.
When working on effects, strive to ensure that all numbers responsible for the smoothness of the effect and requiring accuracy are in the range from 0 to 1. Use for this the operation of cutting off the integer part - frac (x).
In essence, frac is the casting of a floating-point number to an integer, followed by subtracting this integer from the original. That is, frac = x - floor (x).
Since this operation uses the original number in its original form, loss of accuracy can occur at the stage of transferring this number inside the shader. Therefore, we recommend using a time that does not increase to large values. It is good practice to force time to zero at boot times or at times when the game interface overlaps the game level.
The work done has significantly reduced the number of visual errors. Our artists got the opportunity to control the process of creating effects without a long and complicated test on mobile devices.