May 13, 2016 at 15:14.NET development: nine questions for adults
.NET is becoming truly cross-platform: after a long wait, the release date for ASP.NET Core is finally announced, JetBrains is preparing an alternative to Visual Studio based on ReSharper and IDEA, Microsoft acquired Xamarin, made Xamarin Community free, and Mono transferred to the MIT license and finally Windows Server 2016 will receive support for Windows containers in Docker.
With new opportunities we are faced with new challenges:
In just three weeks, at the DotNext conference in St. Petersburg, 20 speakers will give presentations on the present and future of the .NET platform, on optimizing performance and multithreading, on the internal structure of the .NET platform and CLR, on profiling and debugging .NET code.
In the meantime, we asked four of them to share their experiences and opinions on impending changes in the .NET world. Our questions were answered:
Sasha Goldstein
For Microsoft, Roslyn is a project of great importance. Almost everyone in the C # team worked on it for seven years, probably, and could not release it for a very long time. And now he is on github in open-source and the whole process of changing the language before his eyes. On the one hand, there were a lot of people with strange ideas to add constructs from other languages to C #, on the other hand, the whole process is open and in two hours three people who write the compiler can answer you any question. For community - this is very cool.
Andrey Akinshin
All in all I really like Roslyn. As a compiler, it is better than the old one: it compiles many pieces of code more competently and more successfully in terms of performance. But if we are talking about compiling a C # code, then I am more interested in the next stage: JIT compilation. Not so long ago, Microsoft introduced a new 64-bit JIT compiler called RyuJIT.
One of its main tasks is to reduce JIT compilation time. We give the user a very quick start, but because of this we can’t do cool optimizations, so the code itself runs slower than it could, and sometimes even slower than the old JIT. At the same time, RyuJIT is able to make good use of SSE and AVX instructions, which allows with due diligence to implement highly effective programs using the power of one C # without accessing the native code.
In terms of performance, it seems to me that Microsoft is betting on another technology that it is actively developing now: .NET Native. So far it is still in a very raw state, but it looks promising.
Nikita Tsukanov The
main feature of Roslyn is that Microsoft can now add new features with reasonable speed. Previously, there was one compiler for assembly, the other for the studio. Now this is gone - there is only one compiler left and it is written in C #.
Dmitry Ivanov
From the point of view of the API, of course it got better, but there are problems with performance and extra memory, and they are far from a solution. Now in Visual Studio 2015 we cannot open giant projects together with ReSharper because there is not enough memory.
Sasha Goldstein
Mono passes to the MIT-license. So Microsoft will be able to copy and use Mono chunks in CoreFx. It seems to me that Mono should slowly fade away. Mono will have some niches on which .NET Core will not work yet. But Mono runtime itself is so-so: GC has just recently switched to Generational GC, JIT does not work very well. And the framework itself, which they wrote in many cases is not production-ready: there are a lot of bugs, bugs are detected and are not repaired for a long time. Almost all of my customers who used Mono in production had problems. This does not mean that in the end the system did not work at all, but this is not the level of quality that we are used to in .NET.
But with .NET Core, too, things are not going smoothly. The project was supposed to end in January, then in March, just a few days ago, they said that it would be ready at the end of June. API, design model, half of the code are constantly changing.
Last week, one of the clients with a project on ASP.NET MVC 6 asked how I think whether to fix the code every two weeks or is it better to do backport on ASP.NET MVC 5 and wait. The question is not simple.
Andrey Akinshin
.NET Core is certainly a very interesting project, a big step forward, but today it is still raw. Let's not forget that many libraries have not yet come out under it or are in RC. Plus, Microsoft constantly renames something, every day everything changes, some bugs are fixed, others are added. For two years, a new project system based on project.json files was developed and promoted, and just the other day it was thrown into the trash and decided to return to the good old csproj. Such situations discourage the desire for the foreseeable future to try to transfer production systems to .NET Core.
Mono has been developed for over 15 years. This is an adult, solid runtime, which runs a lot of production code. For example, I'm at JetBrains working on a Rider project (a cross-platform .NET IDE). The Backend Rider (this is pure R #) is launched under Mono; under .NET Core, it’s not very possible to run. It works quite well: a healthy application of hundreds of projects and very non-trivial logic, with all the features of .NET and all this under * nix starts up normally. Of course, there are problems of their own, but they are completely resolved in a working order. And about the transition to .NET Core: you need to wait until this new cross-platform runtime becomes more stable.
Nikita Tsukanov
Mono is a fairly mature environment, and .NET Core still loses in beta and the number of available libraries. For example, in .NET Core now there is no support for System.Drawing, but in Mono it appeared in the 2000 shaggy year.
Regarding the performance and quality of coverage of the .NET Framework, Mono is actively transferring to itself the pieces that Microsoft opens. Given that Microsoft acquired Xamarin, the process should go even faster. At some point, Mono will become an add-on over .NET Core, but for now it makes sense to use Mono.
Dmitry Ivanov
We, probably, almost all have the opinion that Mono will drop at some point. At one time, Oracle also said that it would combine HotSpot JVM with JRockit, but eventually dropped the last one.
So it’s probably worth the wait, and switch from Mono to .NET Core. We at Rider work a lot with Mono and see that there are a lot of errors and the code is much less high-quality than in the .NET Framework.
Andrei Akinshin
I believe that there is no silver bullet, there is no universal library that suits everyone. It is necessary to select a solution for your task. If you work with a database, there are databases that work faster than others on certain indicators, if you work with graphics, there are various good libraries for 2D / 3D rendering, but each one is good for its own range of tasks. It must be remembered that each project is unique in its own way. Most often, for some tasks, some solutions are suitable, for others - others.
A very popular mistake: a person heard that the “ABC” library is good, he takes it and uses it without thinking about what tasks it is intended for, on which scenarios it behaves quickly ...
An important point: many bother with performance and spend a lot of time optimizing non-essential sections of code. There will be some speed gain from this, but no one can notice it. When we want to improve performance, we need to conduct research, do benchmarks, and profile.
Dmitry Ivanov
Even within the framework of one .NET team, we have several solutions for interprocess communication, so yes, we often have to. Now, high-performance solutions usually mean some kind of server load.
If we talk about performance problems on one machine, it is also constantly being solved, because, let's say, we pretty quickly go beyond the idiomatic C # code. Quite quickly you have to switch from a large number of array references, so go to unsafe. It just doesn't work without it. And we do our protocol in the Rider Framework with an eye to high performance.
Sasha Goldstein
It is difficult for many developers to imagine what exactly is happening in a multithreaded program, that memory accesses can be moved, that synchronization needs to be done, on the other hand, if there is a lot of synchronization, bottlenecks appear. There are many tools for this, but you need to know and be able to use them.
Many developers do not have access to multiprocessor systems. When the same code is executed not on Core i7, but on a server with at least 64 processors, completely different problems appear. Some small lock that took 2% of the time suddenly starts to take up 50%, there are memory bandwidth problems (the memory system is not able to respond to requests quickly enough), and the cache is not efficiently used. .NET developers most often do not even face such tasks.
Andrey Akinshin
I think that the complexities of multi-threaded programming on .NET are the same as on any other platform. You need to understand how the multi-threaded world works. As for the synchronization primitives and concurrent data structures, in .NET this is all good. Plus, async / await has appeared in C # 5, which allows you to write multi-threaded code in a beautiful and understandable way.
Nikita Tsukanov
There is no way to normally prevent the ability of one thread to access data that is owned by another. In Rust, a more or less normal system was created for this (destroying assignment). In .NET, this is not already without it very difficult: it may happen that when we transfer something from one thread to another, due to inattention or other reasons, something superfluous can pass to another thread. And they can start working with “this excess”, although the owner thread does not know anything about it. In this case, competitive write requests occur where no one knows. As a project grows, tracking and dealing with it becomes more and more difficult.
Dmitry Ivanov
Good question. I have been programming multithreaded for so many years and telling people how to do it, but I haven’t figured it out myself. Probably, the main challenge arises at the top level - to come up with a model that successfully suits a large team of developers and a large product and make everyone follow it and hope that you will never run into the limitations of this model.
The Roslyn team decided to use a transactional model of multi-threaded interaction, and they ran into Memory Wall. We with pessimistic locks and read / write lock-model are faced with a long release of read lock and not smooth tipping.
There is no silver bullet, even newfangled actors do not save: Hello World works well for them, but in reality everything is more complicated.
Sasha Goldstein
Do not deliver, it is necessary. The only question is how necessary. Is it necessary to be able to read assembler, understand the memory model, how the processor works? Standard tip - Understand at least one level below the one you're working on. If you are working on .NET you must understand how the OS, Runtime, GC, Framework works. Without this, even if you find a problem, you won’t know how to fix it.
Andrey Akinshin
Depends on what tasks the programmer solves. 90% of the time, while there is no task to squeeze the maximum out of the car, the programmer should think about ensuring that the code is reliable, readable, without bugs. In the remaining 10% of cases when problems arose (or we guess that the problems will clearly arise), and we try to solve them, then yes, you need to know how everything happens inside.
Many people try to solve performance problems without understanding what is happening in this life: they make the wrong benchmarks, they crookedly profile, they write strange code that only slows things down and spends a lot of work time on it. To successfully cope with these 10% performance-critical cases, you need to know a lot about how everything is arranged inside.
Nikita Tsukanov
The problem of abstractions is that they tend to flow, so the team needs one person who understands how everything works inside. Around the same with a person who knows math well. The traditional low-level problems are memory leaks, exceptions in the native code, heap corruption.
Dmitry Ivanov
In my opinion, any .NET engineer should at least understand how the processor and processor caches work, a little bit of how branch prediction works and understand why serial access to the array and disk always works faster than parallel.
Sasha Goldstein
No need to guess. You need to run the tool and get information about what is happening in the process. Look at memory, CPU, how many threads are doing what. There are many free utilities now. After that there are a bunch of methods. My favorite is USE (utilization, saturation, errors). It was invented by Brendan Gregg, a Linux performance expert. Utilization means that we must determine for each resource (CPU, memory, disk ...) how much it is used. Saturation - means that we must determine for all these resources if over subscription, for example, whether we read 30 files from one disk at a time. Errors - the number of errors in the program. After that, you can optimize something: we will add processors or improve the code. That’s in general.
Andrey Akinshin
The first thing to do is to profile and understand what exactly slows down. A very common mistake is optimization without profiling. After we find a bottleneck, the second step is to ask ourselves the question: “why does this section of the code slow down”? Here, sometimes you need to have really a lot of knowledge: you need to guess what factors are really important and check only them, otherwise the programmer risks spending a lot of time on unnecessary work. And the third step is to fix the problem and carefully check that the program really now works faster (and also that nothing was broken off during optimizations).
Dmitry Ivanov
Classics recommend taking a profiler, finding bottlenecks and fixing them. But when you have already done this several times, there are no frankly bottlenecks left and you cannot speed up the program several times by fixing a couple of lines. Sometimes in these cases it is necessary to revise the approach, for example, insert Assert'es by the code, which make sure that the operations are completed in the allotted time. And to force each team member to correct performance assertions falling in his area of responsibility. It’s also useful to look at the memory traffic and start fighting it. It’s even possible to go to unsafe.
Sasha Goldstein
On Windows, you need to start with Performance Counters. After that, it depends on the situation. A bunch of problems can only be solved with PerfView, but it has big visualization problems. Visual Studio Profiler is pretty good, by the way, it has a standalone version. dotMemory or .NET Memory Profiler– for memory profiling.
Andrey Akinshin
I am a maintainer of the BenchmarkDotNet utility. The latest versions of this library are already behaving stably enough and provide ample opportunities for benchmarking. Alas, many people from the Internet are constantly trying to make their own home-made benchmarks based on Stopwatch and a couple of cycles. At the same time, they are surrounded at every step by problems that they are not aware of, and which can completely spoil the conclusions of the experiment. But this is only part of the trouble: after all, you can miraculously pass all the rakes and in this particular situation write the correct code that really measures something useful. But only every second novice performance engineer launches his benchmarks in DEBUG, because this configuration is selected by default in the studio.
Is it worth explaining that this can completely ruin the measurements taken. BenchmarkDotNet scolds the user for debug configuration and attached debugger in big red letters, complies with the general benchmarking methodology, does competent warming up, runs each method several times in different processes, allows you to compare different environments, considers statistics, etc. Now a community has gathered around the project, and the library is becoming a convenient and popular tool for a number of performance studies.
As a profiling, I can recommend JetBrains products: dotMemory and dotTrace. I often use them, and I really, really like them.
Dmitry Ivanov
All our tools: dotTrace, dotMemory. Enough for almost everything. Sometimes you need to look at sys internals utilities.
Sasha Goldstein
There are a lot of materials online, there are my courses on pluralsight, quite a lot of information can be found in the old blogs of Microsoft developers (Rico Mariani, Chris Bru, Vance Morrison). About books: there is my book Pro .NET Performance , there is a newer book - Ben Watson's "High Performance .NET Applications" . It is also written quite interestingly. Plus, you need to know and understand something about the .NET device. There is a classic book CLR via C # . It's a bit outdated, but some points are simply not mentioned anywhere else, such as how exceptions and delegates work.
Andrey Akinshin
I would go up and down and would not go to the CPU level (as some are trying to do), if there is no basic knowledge about algorithms, data structures and the platform with which you work. According to .NET, you can read the same Richter, understand how the base classes work and what they are good for. It is extremely important to understand the runtime device: how GC works, how classes differ from structures, what a JIT compiler is, etc.
Further, it would be nice to understand how modern hardware works, in particular Intel processors (at least in general terms). In this regard, there are many resources on the Internet, but it is often not very easy to read: you cannot read a book in the evening and figure out the details of the CPU device.
However, a person who does not specialize in performance, but works in an enterprise, does not need this. I advise such people to pump general erudition and broaden their horizons. In this regard, it’s great to attend conferences like DotNext. I myself really like to watch reports on areas that I do not professionally do. In any case, I don’t have time to thoroughly deal with all areas of programming, but at least I will have a general idea and will be able to talk on relevant topics with other programmers. If I have to solve problems related to a new area for me, then I will know which directions to start looking and what to google.
Dmitry Ivanov
Go to the report of Sasha Goldstein :) Read his book, and then be sure to do tasks requiring performance optimization, otherwise the brain will simply discard the information as unnecessary.
Sasha Goldstein
I have two reports: about a profiling tool and about memory models. In the first we will use PerfView. I have prepared code examples with performance problems that we will analyze. PerfView is the front end for ETW technology. By the way, about her there is another report on DotNext. ETW is a technology with which you can collect a log. Many components of Windows, .NET, CLR, ASP.NET write to this log. And they write these logs at a very high speed. PerfView is an effective analysis of ETW logs: CPU, runtime, database access, allocation profiling.
Second report- about memory models. Memory Model is a “scary” name for all memory operations in all high-level languages. If the program works in one thread, everything is simple, but if there are many threads, the situation changes. Writing and reading may not go in the order we specified, the compiler can generally optimize some of the memory accesses. Same thing with processors. And when the port is made from Intel to ARM, which in all iPhones, androids, or Power PC, which in all cars, a lot of everything stops working, because the processor allows itself more.
I’m going to show some theoretical and practical examples when Intel works correctly, but it breaks on an iPhone or when Intel switches over when switching from one thread to several, although almost no one looking at this code can say that there is some kind of that is the problem. After we analyze the subtleties of the Memory Model, we will move on to examples of how to build the correct multithreaded code with the correct synchronization.
Andrey Akinshin
I will talk about arithmetic. I have seen many times how programmers make annoying errors in simple arithmetic expressions, make bugs that are then searched for for weeks (why, then, does the math part work incorrectly there?) - simply because they don’t understand how numbers work with floating point.
Even fewer people think about the performance of arithmetic and what simple metamorphoses can be performed on a mathematical formula so that it starts working faster.
For example, the CPU can parallelize commands that go even to one thread at the hardware level, but for good parallelization you need good code that you need to write correctly. If we find a bottleneck and we need to optimize the calculations, then it makes sense to apply additional knowledge and conjure a bit over the code.
Nikita Tsukanov I
plan to talk about Docker as an element of infrastructure for development and deployment. How to quickly and efficiently deploy and manage microservices. I will also talk about how to run Windows Server Core in Docker.
Dmitry Ivanov
Rider is a hybrid of Idea and ReSharper. We use a common reactive model. Idea sets some parameters, R # responds to it and sets the necessary data for rendering, and such a process can be repeated several times.
The Rider Framework is a Java and .NET reactive library and model generator from DSL in target languages. Our DSL is written in Kotlin, because Kotlin allows some meta-programming in groovy-style, due to its syntactic features. The report will be live-demo.
I think that my report will be of interest to those who want to make their application out-of-process and / or are interested in reactive frameworks and using a reactive model for a desktop application.
If you have questions for speakers , you want to receive a dose of hardcore .NET and find out how and how many times the weather will change in St. Petersburg, we are waiting for you on June 3 at the DotNext 2016 Piter conference .
With new opportunities we are faced with new challenges:
- How will the same code work under .NET Core and Mono, on Windows and Linux, in the docker container?
- Is it worth it to switch to .NET Core now and how to get the most out of the new platform?
- What are the prospects for Mono and Xamarin?
- What changes have occurred “under the hood” of .NET with the transition to Roslyn and .NET Core?
In just three weeks, at the DotNext conference in St. Petersburg, 20 speakers will give presentations on the present and future of the .NET platform, on optimizing performance and multithreading, on the internal structure of the .NET platform and CLR, on profiling and debugging .NET code.
In the meantime, we asked four of them to share their experiences and opinions on impending changes in the .NET world. Our questions were answered:
- Leading global expert on productivity of the .NET platform, eight-time Microsoft MVP, author of the excellent book on .NET performance “Pro .NET Performance” Sasha Goldstein ;
- The chief developer of the reactive multiprocess communication protocol in Rider, Dmitry Ivanov from JetBrains;
- Another Rider developer from JetBrains, .NET MVP, Ph.D., silver medalist of ACM ICPC, postdoc at the Weizmann Institute of Science Andrei Akinshin ;
- CTO Promarket and Mono and Linux expert Nikita Tsukanov .
What has changed in .NET with the transition to Roslyn?
Sasha GoldsteinFor Microsoft, Roslyn is a project of great importance. Almost everyone in the C # team worked on it for seven years, probably, and could not release it for a very long time. And now he is on github in open-source and the whole process of changing the language before his eyes. On the one hand, there were a lot of people with strange ideas to add constructs from other languages to C #, on the other hand, the whole process is open and in two hours three people who write the compiler can answer you any question. For community - this is very cool.
Andrey AkinshinAll in all I really like Roslyn. As a compiler, it is better than the old one: it compiles many pieces of code more competently and more successfully in terms of performance. But if we are talking about compiling a C # code, then I am more interested in the next stage: JIT compilation. Not so long ago, Microsoft introduced a new 64-bit JIT compiler called RyuJIT.
One of its main tasks is to reduce JIT compilation time. We give the user a very quick start, but because of this we can’t do cool optimizations, so the code itself runs slower than it could, and sometimes even slower than the old JIT. At the same time, RyuJIT is able to make good use of SSE and AVX instructions, which allows with due diligence to implement highly effective programs using the power of one C # without accessing the native code.
In terms of performance, it seems to me that Microsoft is betting on another technology that it is actively developing now: .NET Native. So far it is still in a very raw state, but it looks promising.
Nikita Tsukanov Themain feature of Roslyn is that Microsoft can now add new features with reasonable speed. Previously, there was one compiler for assembly, the other for the studio. Now this is gone - there is only one compiler left and it is written in C #.
Dmitry IvanovFrom the point of view of the API, of course it got better, but there are problems with performance and extra memory, and they are far from a solution. Now in Visual Studio 2015 we cannot open giant projects together with ReSharper because there is not enough memory.
Sasha Goldstein
Mono passes to the MIT-license. So Microsoft will be able to copy and use Mono chunks in CoreFx. It seems to me that Mono should slowly fade away. Mono will have some niches on which .NET Core will not work yet. But Mono runtime itself is so-so: GC has just recently switched to Generational GC, JIT does not work very well. And the framework itself, which they wrote in many cases is not production-ready: there are a lot of bugs, bugs are detected and are not repaired for a long time. Almost all of my customers who used Mono in production had problems. This does not mean that in the end the system did not work at all, but this is not the level of quality that we are used to in .NET.
But with .NET Core, too, things are not going smoothly. The project was supposed to end in January, then in March, just a few days ago, they said that it would be ready at the end of June. API, design model, half of the code are constantly changing.
Last week, one of the clients with a project on ASP.NET MVC 6 asked how I think whether to fix the code every two weeks or is it better to do backport on ASP.NET MVC 5 and wait. The question is not simple.
Andrey Akinshin
.NET Core is certainly a very interesting project, a big step forward, but today it is still raw. Let's not forget that many libraries have not yet come out under it or are in RC. Plus, Microsoft constantly renames something, every day everything changes, some bugs are fixed, others are added. For two years, a new project system based on project.json files was developed and promoted, and just the other day it was thrown into the trash and decided to return to the good old csproj. Such situations discourage the desire for the foreseeable future to try to transfer production systems to .NET Core.
Mono has been developed for over 15 years. This is an adult, solid runtime, which runs a lot of production code. For example, I'm at JetBrains working on a Rider project (a cross-platform .NET IDE). The Backend Rider (this is pure R #) is launched under Mono; under .NET Core, it’s not very possible to run. It works quite well: a healthy application of hundreds of projects and very non-trivial logic, with all the features of .NET and all this under * nix starts up normally. Of course, there are problems of their own, but they are completely resolved in a working order. And about the transition to .NET Core: you need to wait until this new cross-platform runtime becomes more stable.
Nikita Tsukanov
Mono is a fairly mature environment, and .NET Core still loses in beta and the number of available libraries. For example, in .NET Core now there is no support for System.Drawing, but in Mono it appeared in the 2000 shaggy year.
Regarding the performance and quality of coverage of the .NET Framework, Mono is actively transferring to itself the pieces that Microsoft opens. Given that Microsoft acquired Xamarin, the process should go even faster. At some point, Mono will become an add-on over .NET Core, but for now it makes sense to use Mono.
Dmitry Ivanov
We, probably, almost all have the opinion that Mono will drop at some point. At one time, Oracle also said that it would combine HotSpot JVM with JRockit, but eventually dropped the last one.
So it’s probably worth the wait, and switch from Mono to .NET Core. We at Rider work a lot with Mono and see that there are a lot of errors and the code is much less high-quality than in the .NET Framework.
Andrei Akinshin
I believe that there is no silver bullet, there is no universal library that suits everyone. It is necessary to select a solution for your task. If you work with a database, there are databases that work faster than others on certain indicators, if you work with graphics, there are various good libraries for 2D / 3D rendering, but each one is good for its own range of tasks. It must be remembered that each project is unique in its own way. Most often, for some tasks, some solutions are suitable, for others - others.
A very popular mistake: a person heard that the “ABC” library is good, he takes it and uses it without thinking about what tasks it is intended for, on which scenarios it behaves quickly ...
An important point: many bother with performance and spend a lot of time optimizing non-essential sections of code. There will be some speed gain from this, but no one can notice it. When we want to improve performance, we need to conduct research, do benchmarks, and profile.
Dmitry Ivanov
Even within the framework of one .NET team, we have several solutions for interprocess communication, so yes, we often have to. Now, high-performance solutions usually mean some kind of server load.
If we talk about performance problems on one machine, it is also constantly being solved, because, let's say, we pretty quickly go beyond the idiomatic C # code. Quite quickly you have to switch from a large number of array references, so go to unsafe. It just doesn't work without it. And we do our protocol in the Rider Framework with an eye to high performance.
Sasha Goldstein
It is difficult for many developers to imagine what exactly is happening in a multithreaded program, that memory accesses can be moved, that synchronization needs to be done, on the other hand, if there is a lot of synchronization, bottlenecks appear. There are many tools for this, but you need to know and be able to use them.
Many developers do not have access to multiprocessor systems. When the same code is executed not on Core i7, but on a server with at least 64 processors, completely different problems appear. Some small lock that took 2% of the time suddenly starts to take up 50%, there are memory bandwidth problems (the memory system is not able to respond to requests quickly enough), and the cache is not efficiently used. .NET developers most often do not even face such tasks.
Andrey Akinshin
I think that the complexities of multi-threaded programming on .NET are the same as on any other platform. You need to understand how the multi-threaded world works. As for the synchronization primitives and concurrent data structures, in .NET this is all good. Plus, async / await has appeared in C # 5, which allows you to write multi-threaded code in a beautiful and understandable way.
Nikita Tsukanov
There is no way to normally prevent the ability of one thread to access data that is owned by another. In Rust, a more or less normal system was created for this (destroying assignment). In .NET, this is not already without it very difficult: it may happen that when we transfer something from one thread to another, due to inattention or other reasons, something superfluous can pass to another thread. And they can start working with “this excess”, although the owner thread does not know anything about it. In this case, competitive write requests occur where no one knows. As a project grows, tracking and dealing with it becomes more and more difficult.
Dmitry Ivanov
Good question. I have been programming multithreaded for so many years and telling people how to do it, but I haven’t figured it out myself. Probably, the main challenge arises at the top level - to come up with a model that successfully suits a large team of developers and a large product and make everyone follow it and hope that you will never run into the limitations of this model.
The Roslyn team decided to use a transactional model of multi-threaded interaction, and they ran into Memory Wall. We with pessimistic locks and read / write lock-model are faced with a long release of read lock and not smooth tipping.
There is no silver bullet, even newfangled actors do not save: Hello World works well for them, but in reality everything is more complicated.
Sasha Goldstein
Do not deliver, it is necessary. The only question is how necessary. Is it necessary to be able to read assembler, understand the memory model, how the processor works? Standard tip - Understand at least one level below the one you're working on. If you are working on .NET you must understand how the OS, Runtime, GC, Framework works. Without this, even if you find a problem, you won’t know how to fix it.
Andrey Akinshin
Depends on what tasks the programmer solves. 90% of the time, while there is no task to squeeze the maximum out of the car, the programmer should think about ensuring that the code is reliable, readable, without bugs. In the remaining 10% of cases when problems arose (or we guess that the problems will clearly arise), and we try to solve them, then yes, you need to know how everything happens inside.
Many people try to solve performance problems without understanding what is happening in this life: they make the wrong benchmarks, they crookedly profile, they write strange code that only slows things down and spends a lot of work time on it. To successfully cope with these 10% performance-critical cases, you need to know a lot about how everything is arranged inside.
Nikita Tsukanov
The problem of abstractions is that they tend to flow, so the team needs one person who understands how everything works inside. Around the same with a person who knows math well. The traditional low-level problems are memory leaks, exceptions in the native code, heap corruption.
Dmitry Ivanov
In my opinion, any .NET engineer should at least understand how the processor and processor caches work, a little bit of how branch prediction works and understand why serial access to the array and disk always works faster than parallel.
Sasha Goldstein
No need to guess. You need to run the tool and get information about what is happening in the process. Look at memory, CPU, how many threads are doing what. There are many free utilities now. After that there are a bunch of methods. My favorite is USE (utilization, saturation, errors). It was invented by Brendan Gregg, a Linux performance expert. Utilization means that we must determine for each resource (CPU, memory, disk ...) how much it is used. Saturation - means that we must determine for all these resources if over subscription, for example, whether we read 30 files from one disk at a time. Errors - the number of errors in the program. After that, you can optimize something: we will add processors or improve the code. That’s in general.
Andrey Akinshin
The first thing to do is to profile and understand what exactly slows down. A very common mistake is optimization without profiling. After we find a bottleneck, the second step is to ask ourselves the question: “why does this section of the code slow down”? Here, sometimes you need to have really a lot of knowledge: you need to guess what factors are really important and check only them, otherwise the programmer risks spending a lot of time on unnecessary work. And the third step is to fix the problem and carefully check that the program really now works faster (and also that nothing was broken off during optimizations).
Dmitry Ivanov
Classics recommend taking a profiler, finding bottlenecks and fixing them. But when you have already done this several times, there are no frankly bottlenecks left and you cannot speed up the program several times by fixing a couple of lines. Sometimes in these cases it is necessary to revise the approach, for example, insert Assert'es by the code, which make sure that the operations are completed in the allotted time. And to force each team member to correct performance assertions falling in his area of responsibility. It’s also useful to look at the memory traffic and start fighting it. It’s even possible to go to unsafe.
Sasha Goldstein
On Windows, you need to start with Performance Counters. After that, it depends on the situation. A bunch of problems can only be solved with PerfView, but it has big visualization problems. Visual Studio Profiler is pretty good, by the way, it has a standalone version. dotMemory or .NET Memory Profiler– for memory profiling.
Andrey Akinshin
I am a maintainer of the BenchmarkDotNet utility. The latest versions of this library are already behaving stably enough and provide ample opportunities for benchmarking. Alas, many people from the Internet are constantly trying to make their own home-made benchmarks based on Stopwatch and a couple of cycles. At the same time, they are surrounded at every step by problems that they are not aware of, and which can completely spoil the conclusions of the experiment. But this is only part of the trouble: after all, you can miraculously pass all the rakes and in this particular situation write the correct code that really measures something useful. But only every second novice performance engineer launches his benchmarks in DEBUG, because this configuration is selected by default in the studio.
Is it worth explaining that this can completely ruin the measurements taken. BenchmarkDotNet scolds the user for debug configuration and attached debugger in big red letters, complies with the general benchmarking methodology, does competent warming up, runs each method several times in different processes, allows you to compare different environments, considers statistics, etc. Now a community has gathered around the project, and the library is becoming a convenient and popular tool for a number of performance studies.
As a profiling, I can recommend JetBrains products: dotMemory and dotTrace. I often use them, and I really, really like them.
Dmitry Ivanov
All our tools: dotTrace, dotMemory. Enough for almost everything. Sometimes you need to look at sys internals utilities.
Sasha Goldstein
There are a lot of materials online, there are my courses on pluralsight, quite a lot of information can be found in the old blogs of Microsoft developers (Rico Mariani, Chris Bru, Vance Morrison). About books: there is my book Pro .NET Performance , there is a newer book - Ben Watson's "High Performance .NET Applications" . It is also written quite interestingly. Plus, you need to know and understand something about the .NET device. There is a classic book CLR via C # . It's a bit outdated, but some points are simply not mentioned anywhere else, such as how exceptions and delegates work.
Andrey Akinshin
I would go up and down and would not go to the CPU level (as some are trying to do), if there is no basic knowledge about algorithms, data structures and the platform with which you work. According to .NET, you can read the same Richter, understand how the base classes work and what they are good for. It is extremely important to understand the runtime device: how GC works, how classes differ from structures, what a JIT compiler is, etc.
Further, it would be nice to understand how modern hardware works, in particular Intel processors (at least in general terms). In this regard, there are many resources on the Internet, but it is often not very easy to read: you cannot read a book in the evening and figure out the details of the CPU device.
However, a person who does not specialize in performance, but works in an enterprise, does not need this. I advise such people to pump general erudition and broaden their horizons. In this regard, it’s great to attend conferences like DotNext. I myself really like to watch reports on areas that I do not professionally do. In any case, I don’t have time to thoroughly deal with all areas of programming, but at least I will have a general idea and will be able to talk on relevant topics with other programmers. If I have to solve problems related to a new area for me, then I will know which directions to start looking and what to google.
Dmitry Ivanov
Go to the report of Sasha Goldstein :) Read his book, and then be sure to do tasks requiring performance optimization, otherwise the brain will simply discard the information as unnecessary.
Sasha Goldstein
I have two reports: about a profiling tool and about memory models. In the first we will use PerfView. I have prepared code examples with performance problems that we will analyze. PerfView is the front end for ETW technology. By the way, about her there is another report on DotNext. ETW is a technology with which you can collect a log. Many components of Windows, .NET, CLR, ASP.NET write to this log. And they write these logs at a very high speed. PerfView is an effective analysis of ETW logs: CPU, runtime, database access, allocation profiling.
Second report- about memory models. Memory Model is a “scary” name for all memory operations in all high-level languages. If the program works in one thread, everything is simple, but if there are many threads, the situation changes. Writing and reading may not go in the order we specified, the compiler can generally optimize some of the memory accesses. Same thing with processors. And when the port is made from Intel to ARM, which in all iPhones, androids, or Power PC, which in all cars, a lot of everything stops working, because the processor allows itself more.
I’m going to show some theoretical and practical examples when Intel works correctly, but it breaks on an iPhone or when Intel switches over when switching from one thread to several, although almost no one looking at this code can say that there is some kind of that is the problem. After we analyze the subtleties of the Memory Model, we will move on to examples of how to build the correct multithreaded code with the correct synchronization.
Andrey Akinshin
I will talk about arithmetic. I have seen many times how programmers make annoying errors in simple arithmetic expressions, make bugs that are then searched for for weeks (why, then, does the math part work incorrectly there?) - simply because they don’t understand how numbers work with floating point.
Even fewer people think about the performance of arithmetic and what simple metamorphoses can be performed on a mathematical formula so that it starts working faster.
For example, the CPU can parallelize commands that go even to one thread at the hardware level, but for good parallelization you need good code that you need to write correctly. If we find a bottleneck and we need to optimize the calculations, then it makes sense to apply additional knowledge and conjure a bit over the code.
Nikita Tsukanov I
plan to talk about Docker as an element of infrastructure for development and deployment. How to quickly and efficiently deploy and manage microservices. I will also talk about how to run Windows Server Core in Docker.
Dmitry Ivanov
Rider is a hybrid of Idea and ReSharper. We use a common reactive model. Idea sets some parameters, R # responds to it and sets the necessary data for rendering, and such a process can be repeated several times.
The Rider Framework is a Java and .NET reactive library and model generator from DSL in target languages. Our DSL is written in Kotlin, because Kotlin allows some meta-programming in groovy-style, due to its syntactic features. The report will be live-demo.
I think that my report will be of interest to those who want to make their application out-of-process and / or are interested in reactive frameworks and using a reactive model for a desktop application.
If you have questions for speakers , you want to receive a dose of hardcore .NET and find out how and how many times the weather will change in St. Petersburg, we are waiting for you on June 3 at the DotNext 2016 Piter conference .
