Java is not only bloody enterprise, but also fast latency-sensitive applications
I do algorithmic trading at Raiffeisenbank. This is a rather specific area of the banking sector. We make a trading platform that works with low and predictable delays. The success of the application depends, inter alia, on the speed of the application, so we have to deal with the entire stack involved in trading: private network channels, special hardware, OS settings and a special JVM, and, of course, the application itself. We cannot stop optimizing exclusively the application itself - the OS or network settings are no less important. This requires technical expertise and erudition to understand how data flows through the entire stack, and where there may be a delay.
Not every organization / bank can afford the development of this class of software. But I was lucky that such a project was launched within the walls of Raiffeisenbank, and I had a suitable specialization - I specialized in code performance at Intel Moscow Compiler Laboratory. We made compilers for C, C ++ and Fortran. At Raiffeisenbank, I switched to Java. If earlier I made some kind of tool that many people used then, now I have moved to the other side of the barricades and am engaged in applied analysis of the performance of not only the code, but the entire application stack. Regularly, the way to research a performance problem lies far beyond the code, for example, in the kernel or network settings.
It is widely believed that Java is not very suitable for the development of highly loaded systems.
This can only partially be agreed. Java is beautiful in many aspects. If we compare it with a language like C ++, then its potential overhead may be higher, but sometimes functionally similar solutions in C ++ may work more slowly. There are optimizations that automatically work in Java, but do not work in C ++, and vice versa. Looking at the quality of the code that comes after the Java JIT compiler, I want to believe that performance will be inferior to what I could achieve in peak, no more than several times. But at the same time I get very fast development, excellent tools and a wide selection of off-the-shelf components.
Let's face it: in the C ++ world, development environments (IDEs) are significantly behind IntelliJ and Eclipse. If a developer uses any of these environments, then the debugging speed, finding bugs and writing complex logic is an order of magnitude higher. As a result, it turns out that it is easier to file Java in the right places so that it works fast enough than to do everything from scratch and for a very long time in C ++. The funniest thing is that when writing competitive code, the approaches to synchronization in both Java and C ++ are very similar: they are either OS-level primitives (for example, synchronized / std :: mutex ) or iron primitives ( Atomic * / std :: atomic <*> ) . And it is very important to see this similarity.
In general, we are developing a non-highload application))
What is the difference between highload and low latency application?
The term highload does not fully reflect the specifics of our work - we are engaged in latency-sentitive systems . What is the difference? For highly loaded systems, it is important to work on average fairly quickly, making full use of hardware resources. In practice, this means that every hundredth / thousandth /../th millionth request to the system can potentially work very slowly, because we focus on average values and do not always take into account that our users suffer significantly from brakes.
We are engaged in systems for which the level of delay is critical. Our task is to ensure that the system always has a predictable response. Potentially latency-sensitive systemsmay not be highly loaded if events occur rarely enough, but a guaranteed response time is required. And that does not make their development easier. Quite the contrary! Dangers lie in wait everywhere. The overwhelming majority of components of modern hardware and software are oriented towards good work "on average", i.e. for throughput .
Take at least data structures. We use hash tables, and if a re-hash of the entire data structure occurs on a critical path, this can lead to tangible brakes for a specific user on a single request. Or JIT compiler - optimizes the most frequently executed code pattern, pessimizing the rarely executed code pattern. But the speed of this particular rare case can be very important for us!
Maybe this code processes a rare kind of orders? Or some unusual market situation that needs a quick response? We try to make sure that the reaction of our system to these potentially rare events takes some predictable and, preferably, very short time.
This question cannot be answered in two phrases. In a first approximation, it is important to understand if there is any kind of synchronization - synchronized, reentrantlock or something from java.util.concurrent . Often you have to use synchronization on busy-spin'ah. Using any synchronization primitive is always a compromise. And it is important to understand how these synchronization primitives work and what trade-offs they carry. It is also important to evaluate how much garbage a particular piece of code generates. The best way to combat garbage collector is to not trigger it. The less we will generate garbage, the less often we will run a garbage collector, and the longer the system will work without its intervention.
We also use a wide range of different tools that allow us to analyze not only average indicators. We have to very carefully analyze how slowly the system works every hundredth, every thousandth time. Obviously, these indicators will be worse than the median or average. But it is very important for us to know how much. And tools such as Grafana , Prometheus , HDR histograms, and JMH help show this .
Can I remove Unsafe?
Often you have to use what apologists call an undocumented API. I'm talking about the famous Unsafe. I believe unsafe has become a de facto part of the public API of Java machines. It makes no sense to deny it. Unsafe uses a lot of projects that we all actively use. And if we refuse it, then what will happen to these projects? Either they will live on the old version of Java, or they will again have to spend a lot of energy to rewrite all this. Is the community ready for this? Is it ready to potentially lose tens of percent of productivity? And most importantly, in exchange for what?
Indirectly, my opinion confirms a very neat deletionmethods from Unsafe - in Java11, the most useless methods from Unsafe have been removed. I think that until at least half of all projects using Unsafe move to something else, Unsafe will be available in one form or another.
There is an opinion: Bank + Java = bloody ossified enterprise?
Our team has no such horrors. At Spring, we probably wrote ten lines, and by me)) We try not to use big technologies. We prefer to do small, neat and fast, so that we can realize it, control it and, if necessary, modify it. The latter is very important for systems (like ours), which have non-standard requirements, which may differ from the requirements of 90% of the users of the framework. And in the case of using a large framework, we will not be able to convey our needs to the community or to independently correct the behavior.
In my opinion, developers should always be able to use all available tools. I came to the Java world from C ++ and I am very struck by the division of the community into those who develop runtimevirtual machine / compiler or the virtual machine itself and to application developers. This is clearly seen in the standard JDK class code. Often JDK authors use a different API. Potentially, this means that we cannot achieve peak performance. In general, I believe that using the same API for writing both the standard library and application code is an excellent indicator of the maturity of the platform.
I think it’s very important for all developers to know how, if not the whole stack, then at least half of it works: Java code, byte code, internals of the virtual machine runtime and assembler, hardware, OS, network. This allows a broader look at the problems.
Performance is also worth mentioning. It is very important not to focus on the average and always look at the median and high percentiles (the worst of 10/100/1000 / ... measurements).
I’ll talk about all this at a meeting of Java User Group on May 30 in St. Petersburg. Meeting with Sergei Melnikov, that's just me)) You can register here .
What are we going to talk about?
Come to the meeting, it will be a great evening, there will be many interesting stories about our platform and about our favorite language.
See you ;)
Not every organization / bank can afford the development of this class of software. But I was lucky that such a project was launched within the walls of Raiffeisenbank, and I had a suitable specialization - I specialized in code performance at Intel Moscow Compiler Laboratory. We made compilers for C, C ++ and Fortran. At Raiffeisenbank, I switched to Java. If earlier I made some kind of tool that many people used then, now I have moved to the other side of the barricades and am engaged in applied analysis of the performance of not only the code, but the entire application stack. Regularly, the way to research a performance problem lies far beyond the code, for example, in the kernel or network settings.
Java is not for highload?
It is widely believed that Java is not very suitable for the development of highly loaded systems.
This can only partially be agreed. Java is beautiful in many aspects. If we compare it with a language like C ++, then its potential overhead may be higher, but sometimes functionally similar solutions in C ++ may work more slowly. There are optimizations that automatically work in Java, but do not work in C ++, and vice versa. Looking at the quality of the code that comes after the Java JIT compiler, I want to believe that performance will be inferior to what I could achieve in peak, no more than several times. But at the same time I get very fast development, excellent tools and a wide selection of off-the-shelf components.
Let's face it: in the C ++ world, development environments (IDEs) are significantly behind IntelliJ and Eclipse. If a developer uses any of these environments, then the debugging speed, finding bugs and writing complex logic is an order of magnitude higher. As a result, it turns out that it is easier to file Java in the right places so that it works fast enough than to do everything from scratch and for a very long time in C ++. The funniest thing is that when writing competitive code, the approaches to synchronization in both Java and C ++ are very similar: they are either OS-level primitives (for example, synchronized / std :: mutex ) or iron primitives ( Atomic * / std :: atomic <*> ) . And it is very important to see this similarity.
In general, we are developing a non-highload application))
What is the difference between highload and low latency application?
The term highload does not fully reflect the specifics of our work - we are engaged in latency-sentitive systems . What is the difference? For highly loaded systems, it is important to work on average fairly quickly, making full use of hardware resources. In practice, this means that every hundredth / thousandth /../th millionth request to the system can potentially work very slowly, because we focus on average values and do not always take into account that our users suffer significantly from brakes.
We are engaged in systems for which the level of delay is critical. Our task is to ensure that the system always has a predictable response. Potentially latency-sensitive systemsmay not be highly loaded if events occur rarely enough, but a guaranteed response time is required. And that does not make their development easier. Quite the contrary! Dangers lie in wait everywhere. The overwhelming majority of components of modern hardware and software are oriented towards good work "on average", i.e. for throughput .
Take at least data structures. We use hash tables, and if a re-hash of the entire data structure occurs on a critical path, this can lead to tangible brakes for a specific user on a single request. Or JIT compiler - optimizes the most frequently executed code pattern, pessimizing the rarely executed code pattern. But the speed of this particular rare case can be very important for us!
Maybe this code processes a rare kind of orders? Or some unusual market situation that needs a quick response? We try to make sure that the reaction of our system to these potentially rare events takes some predictable and, preferably, very short time.
How to achieve a predictable reaction time?
This question cannot be answered in two phrases. In a first approximation, it is important to understand if there is any kind of synchronization - synchronized, reentrantlock or something from java.util.concurrent . Often you have to use synchronization on busy-spin'ah. Using any synchronization primitive is always a compromise. And it is important to understand how these synchronization primitives work and what trade-offs they carry. It is also important to evaluate how much garbage a particular piece of code generates. The best way to combat garbage collector is to not trigger it. The less we will generate garbage, the less often we will run a garbage collector, and the longer the system will work without its intervention.
We also use a wide range of different tools that allow us to analyze not only average indicators. We have to very carefully analyze how slowly the system works every hundredth, every thousandth time. Obviously, these indicators will be worse than the median or average. But it is very important for us to know how much. And tools such as Grafana , Prometheus , HDR histograms, and JMH help show this .
Can I remove Unsafe?
Often you have to use what apologists call an undocumented API. I'm talking about the famous Unsafe. I believe unsafe has become a de facto part of the public API of Java machines. It makes no sense to deny it. Unsafe uses a lot of projects that we all actively use. And if we refuse it, then what will happen to these projects? Either they will live on the old version of Java, or they will again have to spend a lot of energy to rewrite all this. Is the community ready for this? Is it ready to potentially lose tens of percent of productivity? And most importantly, in exchange for what?
Indirectly, my opinion confirms a very neat deletionmethods from Unsafe - in Java11, the most useless methods from Unsafe have been removed. I think that until at least half of all projects using Unsafe move to something else, Unsafe will be available in one form or another.
There is an opinion: Bank + Java = bloody ossified enterprise?
Our team has no such horrors. At Spring, we probably wrote ten lines, and by me)) We try not to use big technologies. We prefer to do small, neat and fast, so that we can realize it, control it and, if necessary, modify it. The latter is very important for systems (like ours), which have non-standard requirements, which may differ from the requirements of 90% of the users of the framework. And in the case of using a large framework, we will not be able to convey our needs to the community or to independently correct the behavior.
In my opinion, developers should always be able to use all available tools. I came to the Java world from C ++ and I am very struck by the division of the community into those who develop runtimevirtual machine / compiler or the virtual machine itself and to application developers. This is clearly seen in the standard JDK class code. Often JDK authors use a different API. Potentially, this means that we cannot achieve peak performance. In general, I believe that using the same API for writing both the standard library and application code is an excellent indicator of the maturity of the platform.
Something else
I think it’s very important for all developers to know how, if not the whole stack, then at least half of it works: Java code, byte code, internals of the virtual machine runtime and assembler, hardware, OS, network. This allows a broader look at the problems.
Performance is also worth mentioning. It is very important not to focus on the average and always look at the median and high percentiles (the worst of 10/100/1000 / ... measurements).
I’ll talk about all this at a meeting of Java User Group on May 30 in St. Petersburg. Meeting with Sergei Melnikov, that's just me)) You can register here .
What are we going to talk about?
- About profiling and using the standard Linux and perf profiler: how to live with them, what they measure and how to interpret their results. This will be an introduction to general profiling, with tips, life hacks, how to squeeze everything possible out of profilers so that they profile with maximum accuracy and frequency.
- About equipment features for obtaining an even more detailed profile and viewing the profile of rare events. For example, when your code runs 10 times slower every hundredth time. No profiler will tell about this. We will write our small profiler using the standard Linux kernel mechanism and try to see the profile of some rare event.
Come to the meeting, it will be a great evening, there will be many interesting stories about our platform and about our favorite language.
See you ;)