Benchmarking .Net, Java, and Mono Platform Performance
Idea Java vs .Net vs Mono
The very idea of creating such a test came about because of the opposition of .Net and Java that constantly haunted me, and I decided to evaluate the real performance of these platforms as objectively as possible, then the interesting opensource development of Mono (free implementation of .Net) came into view, and it was decided to include it, and at the same time run tests under Linux. Accordingly, two similar testing programs in C # and Java were developed. The following will be the fragments of the source code in C #, the full source code can be obtained from the Google Code repository:
http://code.google.com/p/dotnet-java-benchmark/source/checkout
The purpose of this test is to compare the performance of various virtual machines that execute essentially the same code on the same computer. The following platforms took part in the competition:
- Microsoft .Net 4.0 (Windows 7)
- Oracle Java SE Version 6 Update 24 (Windows 7)
- Oracle Java SE Version 6 Update 24 (Linux 2.6.35.27 Ubuntu 10.10)
- Novell Mono 2.11 (Linux 2.6.35.27 Ubuntu 10.10)
No tune, overclocking or optimization of operating systems was performed, just installed, patched with the latest OS updates.
Testing was done on my battered laptop Dell Inspirion 6400 Intel Core Duo T2300 1.66 GHz 1.5 GB RAM. The processor is 32-bit.
The test consists of several groups of microtests:
- math operations
- random number generation
- work with arrays
- work with collections
- string conversions
The result of each microtest is the average number of operations performed per microsecond. The entire test was cyclically run 10 times, and all the results obtained by microtests were again averaged. The maximum amount of managed memory consumed by the program during the entire test was also estimated approximately.
Java
The test for Java platforms was developed in the Eclipse IDE for Windows, when it was launched on Linux, there were absolutely no problems, but in both cases it was necessary to explicitly specify the maximum memory size for the Java machine at 768 MB, the default value was too small. Both Java machines started with the following parameters:
-Xms32m -Xmx768mC #
The test for .Net and Mono was developed in Visual Studio and what was my disappointment when the first time I started under Mono, the runtime sadly informed me that there was no implementation of the int.TryParse () method, a trifle, but it was unpleasant. The construction with int.TryParse () was replaced by int.Parse () with a condition directly under Linux in MonoDevelop (which surprisingly did a great job with Microsoft's solution). The test worked, but much more interesting surprises began here. After each iteration, the system consumed systematic portions of RAM, then swap, and after all the RAM (including the swap) was exhausted, Linux destroyed this unfavorable process.
Mono gc
Initially, there was a feeling that the garbage collector did not work correctly, after each iteration of the test a decent portion of arrays and collections remained, but the memory was not freed. I started googling, I found the --desktop key in the documentation for Mono that should prevent the program from exclusively consuming memory (Currently this sets the GC system to avoid expanding the heap as much as possible at the expense of slowing down garbage collection a bit). Also there was described an interesting key --gcwith the help of which it was possible to choose one of two garbage collectors: Boehm or SGen, but this key did not work, which led to the idea of the obsolete version of Mono being installed. The latest stable version 2.10.1 was featured on the official website, but in the Ubuntu 10.10 repository there was only version 2.6.7. But the solution was soon found: through Git, the latest version 2.11 was received, successfully assembled and installed. By the way, TryParse () was already implemented there, and in general the exe file compiled in Windows under Mono started without recompilation, which is very nice. After trying different garbage collectors, it turned out that the test does not work with the Boehm collector, and successfully runs with SGen. As a result, Mono's launch options were as follows:
--desktop --gc=sgenNow the results. On all graphs, the Y axis represents the number of operations per microsecond.
Math functions
Hereinafter, I will not give the full source code, only small fragments who are interested in the link to the repository above.
namespace DotNetPerformance.Math
{
public class MathTestParams
{
public static readonly int iterationCount = 5000000;
}
class Div10Test : SomeTest
{
public Div10Test()
{
_name = "Деление целых чисел на 10";
_iterationCount = MathTestParams.iterationCount * 10;
}
public override void Do()
{
StartTiming();
for (int i = 0; i < _iterationCount; ++i)
{
int x = i / 10;
}
StopTiming();
}
}
class SinTest : SomeTest
{
public SinTest()
{
_name = "Синус";
_iterationCount = MathTestParams.iterationCount * 5;
}
public override void Do()
{
double val = 0;
double dt = System.Math.PI * 2;
dt /= _iterationCount;
StartTiming();
for (int i = 0; i < _iterationCount; ++i)
{
double x = System.Math.Sin(val);
val += dt;
}
StopTiming();
}
}
...
}
.Net has a significant gap (by two orders of magnitude), for the functions of sine, cosine, square root and division by 10, because of it I had to use a logarithmic scale, these are the only indicators in the whole test that have such a big difference between platforms. This increase in speed probably gives the use of tables in the implementation. Otherwise, all competitors are relatively close to each other; in most microtests, .Net and Mono have the advantage. Mono considers arcsine and arccosine much faster than other platforms. Java leads the way in calculating logarithms. Note that the results for Java under Linux and Windows are very close to each other.
Random numbers
namespace DotNetPerformance.RandomTests
{
public class RandomTestParams
{
public static readonly int count = 10000000;
}
class IntRandomTest : SomeTest
{
public IntRandomTest()
{
_name = "Генерация случайных чисел int";
_iterationCount = RandomTestParams.count;
}
private Random rnd = new Random();
public override void Do()
{
StartTiming();
for (int i = 0; i < _iterationCount; ++i)
{
int x = rnd.Next();
}
StopTiming();
}
}
...
}
Perhaps the microtests for random number generation are not completely objective, because different algorithms can be used in the implementation of random number generators, but I'm not interested in the proximity to the uniform distribution, but in the final generation speed that is obtained at the output. In this competition .Net is a significant leader, Mono is clearly in second place.
Arrays
namespace DotNetPerformance.ArraysTests
{
public class ArrayTestParams
{
public static readonly int arraySize = 50000000;
}
class ArrayIntAccessTest : SomeTest
{
public ArrayIntAccessTest()
{
_name = "int[] последовательный доступ к элементам";
_iterationCount = ArrayTestParams.arraySize;
}
public override void Do()
{
int[] array = null;
while (array == null)
{
try
{
array = new int[_iterationCount];
}
catch (OutOfMemoryException)
{
_iterationCount /= 2;
Console.WriteLine("!! Недостаточно памяти для полного теста");
}
}
for (int i = 0; i < array.Length; ++i)
{
array[i] = i;
}
StartTiming();
for (int i = 0; i 0; –i)
{
int x = array[i];
}
StopTiming();
}
}
...
}
In this group of microtests, .Net is again faster, Java is confidently in second place.
Collections
namespace DotNetPerformance.CollectionsTests
{
class ListTestParams
{
public static readonly int ListInsertRemoveSize = 500000;
public static readonly int ListAccessSize = 2000000;
}
class DynamicArrayInsertRemoveTest : SomeTest
{
public DynamicArrayInsertRemoveTest()
{
_name = "DynamicArray вставка и удаление элементов";
_iterationCount = ListTestParams.ListInsertRemoveSize / 10;
}
public override void Do()
{
List list = new List();
StartTiming();
for (int i = 0; i 0)
{
list.RemoveAt(0);
}
StopTiming();
}
}
...
}
Again, in most microtests, .Net is faster, Java in 4 microtests is an order of magnitude slower, with Mono the results fluctuate greatly. The low speed of working with parameterized collections of Java is probably due to the fact that it is not possible to specialize templates in Java with primitive types, which leads to the necessity of packing / unpacking primitive variables into objects.
String Operations
namespace DotNetPerformance.StringConversions
{
public class StringConversionsTestParams
{
public static readonly int iterationCount = 10000000;
}
class IntParseTest : SomeTest
{
public IntParseTest()
{
_name = "Парсинг int";
_iterationCount = StringConversionsTestParams.iterationCount / 10;
}
public override void Do()
{
string[] arr = new string[_iterationCount];
for (int i = 0; i < arr.Length; ++i)
{
arr[i] = i.ToString();
}
StartTiming();
for (int i = 0; i < arr.Length; ++i)
{
int x = int.Parse(arr[i]);
}
StopTiming();
}
}
...
}
Java significantly outperforms competitors in parsing and converting integers to strings, but lags far behind when parsing floating-point numbers. .Net is faster in floating point operations. Strange, but parsing in double Java produces much faster than in float. Mono is the fastest to remove characters from StringBuilder.
results
I’ll warn you that I’m not a promoted Microsoft fanatic, I did not expect such results, there is no propaganda here, just a subjective test. By the sum of the results .Net leads in all groups of microtests, as well as in memory consumption. In 4 out of 5 groups, Mono surpasses Java in speed, but at the same time it consumes almost twice (!) More memory. Interestingly, the total Java results for Linux and Windows are almost the same or very close to each other.
Mono's age is still too small compared to such giants as Java and .Net, but nevertheless they can already be compared, and in some ways Mono can even surpass its ideological progenitors, while significantly inferior in another, let's see what happens next.
Java has to pay for cross-platform speed, on the other hand, much more resources were probably used to create .Net.
References
Similar interesting works:
- Stefan Krause Java vs. C benchmark
- Java Micro Benchmark - performance comparison of different computers using microtests
updated
