Is Unity really slow? Caution LINQ
It is often said that Unity is slow. But how much? I am developing a Pixel Studio application, this is a pixel art editor. For him, I wrote my own implementation of the GIF format. The most time consuming operation is GIF encoding, namely the LZW compression algorithm. Let's see how Unity can handle it.
First of all, I will immediately clarify why I had to write my own library. Yes, everything is simple, the System.Windows.Media.Imaging library, in which the wonderful GifBitmapEncoder is located, cannot be connected to Unity. Therefore, I read articles on HIF GIF, took the specification and made my GIF with blackjack.
I started writing the library and debugging it, of course, in Visual Studio, in the console application. I'll omit this moment, debugging was boring and long, took 3 days. I didn’t have experience in such algorithms, I usually do games. Okay, the library is ready. For example, a test “heavy” gif in 200 frames and a resolution of 256x256 is encoded in 15 seconds (on my hardware, of course, Ryzen 7). Many, I thought, and with a masterly hand made the compression process parallel (it seems, Unity should support threads). The test gif began to be encoded in 5 seconds. Fine!
Just copying all the source codes to Unity to test the functionality. Coding test gifki takes 120 seconds. In multithreaded mode (yes, the threads in Unity work, the main thing is not to touch the UI) coding of the gifs takes 180 seconds. Facepalm
Google - the problem turns out to be common. According to similar reports, the algorithmic code in Unity runs 10-20 times slower. This is supposedly due to another implementation of garbage collection and the hazy Editor Overhead. In builds (Windows, Android), the situation is similar. In an .exe build, for example, it runs slightly faster, by 20%.
Question to readers - am I doing something wrong, or is the problem taking place?
Link to the implementation of GIF, who are interested: GitHub . The library was written under C # version 6 and .NET 3.5 to be compatible with older versions of Unity. The project can be switched to .NET 4.0, ThreadPool will then work much faster.
UPD: Thanks for the comments! Everything turned out a little confusing: and the threads with locks, and the implementation of the curve. So I prepared a simple case - the operation of running through an array in LINQ. Namely, such an operation is performed when compressing LZW (only there checking the keys in the dictionary).
Perform in the console application - 5 ms.
Perform in Unity - 2100 ms.
UPD: Partially problem found in LINQ. For example, calling Contains is several times slower than Array.IndexOf. LINQ is great, it allows you to save time and make the code more beautiful. But not in such applied tasks as working with large arrays of objects. This concerns the specific implementation of LINQ in Unity.
UPD: Optimized LZW compression algorithm, removed string keys in the dictionary. Removed LINQ for large collections where it was possible. The implementation of multithreading did not touch. And everything flew, even on Android. Of course, the difference between the speed of the console application remained, but it is not so significant anymore.
Special thanks to WNeZRoS for help in optimizing the code and to all the discussion participants! Although the cause of the brakes in LINQ remains undisclosed.
First of all, I will immediately clarify why I had to write my own library. Yes, everything is simple, the System.Windows.Media.Imaging library, in which the wonderful GifBitmapEncoder is located, cannot be connected to Unity. Therefore, I read articles on HIF GIF, took the specification and made my GIF with blackjack.
I started writing the library and debugging it, of course, in Visual Studio, in the console application. I'll omit this moment, debugging was boring and long, took 3 days. I didn’t have experience in such algorithms, I usually do games. Okay, the library is ready. For example, a test “heavy” gif in 200 frames and a resolution of 256x256 is encoded in 15 seconds (on my hardware, of course, Ryzen 7). Many, I thought, and with a masterly hand made the compression process parallel (it seems, Unity should support threads). The test gif began to be encoded in 5 seconds. Fine!
Just copying all the source codes to Unity to test the functionality. Coding test gifki takes 120 seconds. In multithreaded mode (yes, the threads in Unity work, the main thing is not to touch the UI) coding of the gifs takes 180 seconds. Facepalm
Google - the problem turns out to be common. According to similar reports, the algorithmic code in Unity runs 10-20 times slower. This is supposedly due to another implementation of garbage collection and the hazy Editor Overhead. In builds (Windows, Android), the situation is similar. In an .exe build, for example, it runs slightly faster, by 20%.
Question to readers - am I doing something wrong, or is the problem taking place?
Link to the implementation of GIF, who are interested: GitHub . The library was written under C # version 6 and .NET 3.5 to be compatible with older versions of Unity. The project can be switched to .NET 4.0, ThreadPool will then work much faster.
UPD: Thanks for the comments! Everything turned out a little confusing: and the threads with locks, and the implementation of the curve. So I prepared a simple case - the operation of running through an array in LINQ. Namely, such an operation is performed when compressing LZW (only there checking the keys in the dictionary).
Perform in the console application - 5 ms.
Perform in Unity - 2100 ms.
UPD: Partially problem found in LINQ. For example, calling Contains is several times slower than Array.IndexOf. LINQ is great, it allows you to save time and make the code more beautiful. But not in such applied tasks as working with large arrays of objects. This concerns the specific implementation of LINQ in Unity.
UPD: Optimized LZW compression algorithm, removed string keys in the dictionary. Removed LINQ for large collections where it was possible. The implementation of multithreading did not touch. And everything flew, even on Android. Of course, the difference between the speed of the console application remained, but it is not so significant anymore.
Special thanks to WNeZRoS for help in optimizing the code and to all the discussion participants! Although the cause of the brakes in LINQ remains undisclosed.