July 31, 2015 at 12:00“Speed is a feature that no one ever refuses” - an interview with Dmitry Zhemerov from JetBrains
Today is Friday, and Friday at Habré is a great day for something unusual. Today I bring to your attention an interview with Dmitry yole Zhemerov , a man who had a hand in IntelliJ IDEA, PyCharm, Kotlin and many other JetBrains products.
What we talked about:
Under the cutscene is the transcript of this interview.
- Good afternoon! This is “No Slides.” We have a graduation today with Dmitry Zhemerov, a veteran of JetBrains. Dima, let's start by saying a little about yourself: what did you do, including in JetBrains?
I joined JetBrains in 2003. I was the 28th or 29th employee, and during this time I managed to do a lot of things to do. At first I was the Omea project development manager- there was such an information aggregator, that is, it was a program that collected information from various sources such as mail, news, RSS feeds. He allowed us to do a search on all of them, to organize, categorize and so on. Then the Omea project was closed, and I switched to the IntelliJ IDEA team. We can say that I was at the origins of such an entity as the IntelliJ platform, that is, I was engaged in the translation of the monolithic development environment IntelliJ IDEA into a platform on the basis of which you can build IDEs for different languages, and not just Java. It is clear that a lot of people participated in this, but I also did quite a lot there. Then I worked on various IDEs based on this platform: I managed to work out both RubyMine, and PyCharm, and WebStorm.
-Where do you think IDEA is now moving as a product, as a platform? There is a feeling that some stagnation has occurred: as an IntelliJ IDEA user, it seems to me that nothing is happening. New versions are coming out, they may work faster, support new frameworks. So is it, isn’t it?
Faster is the most important thing. Speed is a feature that no one ever refuses. We have been investing in the quality of user interfaces, usability for a rather long time, so that everything is neat, smooth. Big features that change your experience as a developer also appear in some quantities. For example, Inline Debugging, which appeared in the 14th version. When you step through the code, you are immediately right in the code, in the editor you see the value of the variables that are calculated on these lines. Or, for example, shortly before that, support for several carriages appeared in the editor - this is also a big feature, which for many matters. What will be next, I really can’t say, because our development process is not planned very far ahead, and it’s just that a developer might suddenly get the idea: “Let’s deal with this functionality. Let me do it. ” He will go and do, and she will appear in the release, and everything will be great.
- About 3 years ago, Oleg Stepanov and Maxim Shafirov became CEO of JetBrains. First, why are there two? I do not remember any such cases when the CEO is two at once. What has changed over the years? What, perhaps, tasks and goals were set for them, if this is open information, and how successful is it? What has changed in the company?
Two CEOs - it turned out to be quite convenient, because there are two offices. Oleg lives in Munich, Max lives in St. Petersburg, and each of them is primarily responsible for what happens in their office. There is just a lot of work, and somehow they share this work among themselves. Max is now very actively involved in our entire sales infrastructure, that is, our internal corporate information system, if you like, in the store. So that users have a good experience in this place. Oleg is engaged in quite a lot of our server tools in order to make some unified solution from our zoo represented by YouTrack, TeamCity and Upsource, which can simply be put and it will solve all the problems with the development process that are in your team.
-That is, in addition to the separation by office, is there still a separation by product?
This is not so much for products as for areas of activity. With us, like everyone else in JetBrains, this somehow organically happens. When we implemented this scheme, we didn’t have it right away that we’ll agree on who is responsible for what - it’s just along the way that this activity, it’s necessary to deal with it, and, accordingly, the person who can take it on himself, takes it upon herself.
- How do you rate these 3 years? What has changed globally?
In terms of the organization of work in principle, JetBrains remained the same. Some rather large projects happened, for example, moving to our new office on Vasilyevsky Island, in which Sergey Dmitriev, who was previously the head of the company, did not take any part at all. Or, for example, in the organization of sales. A lot of things have changed during this time inside our company, some things could be done great.
- Two CEOs - is this an experiment that is ongoing? Are its results already visible, or is it still in the process? Or, in principle, any global changes were not expected?
No, in fact, in the first place, the motivation for this decision was simply very simple. At one point Sergey Dmitriev said: “I'm not young anymore, roughly speaking. "I can either continue to do further JetBrains, or I can leave JetBrains, which was made, to some other people from the team, and go go do some other activities.” Judging by the fact that for these 3 years he has been successfully engaged in other activities, without being distracted by JetBrains or very little by being distracted by JetBrains, then, in general, everything turned out. JetBrains grows during this time, successfully breaks and so on. This task has been fully completed.
- On your old sitethere is a record dated 2003 that you got a job at IntelliJ Labs. Now we know the company as JetBrains. Can you talk a little about the relationship between JetBrains and IntelliJ Labs?
It is very simple. IntelliJ Labs is the name by which the company appeared. In 2000-2001, the first versions of IntelliJ IDEA were released under the IntelliJ Software or IntelliJ Labs brand. Then, when it became clear that we want to make products not only for Java, respectively, the name with the letter “J” is not suitable for us. We put the letter “J” at the beginning and came up with the name JetBrains. IntelliJ Labs is still preserved in the name of the Russian legal entity of our company - INTELLIGEY LABS LLC. JetBrains is, accordingly, the legal entity of the company's headquarters.
-JetBrains, which was in the mid-2000s, and JetBrains of the 2015 model - do you feel any difference between them, and if so, what do you like, what do you dislike?
It is clear that a company of this size simply has a different level of personal connections. When I come to St. Petersburg from Munich, I walk around the office and see a lot of people about whom I often don’t even know: they work for us in the company, they went to visit someone or they just brought pizza. These are people whom I do not know; I don’t know what they are doing. The positive thing is that JetBrains has been chronically understaffed for a very long time, that is, all the time there was a state that there was more work than the people who were available to do it. It seems to me that recently in very many projects we have come out of this state - not in all, but in many. Roughly speaking, we have as many people as needed to really move products forward.
- Now you have 300 people, 400?
400 with something.
-This is mainly Peter and Munich, right?
Yes, we now probably have 80 people in Munich and, accordingly, 400 with something in St. Petersburg so far.
- Hundreds of people. Distribution creates any difficulties for you?
It does not seem to me that it creates. As a rule, people who have worked in the St. Petersburg office for some time and who are already familiar with the team with whom they work together leave for Munich. Accordingly, there is no such thing that you have some kind of sudden person in another office with whom you need to somehow build relationships. Plus, we often go to each other.
“ Do you often come here yourself?”
Every two or three months I’m definitely in St. Petersburg. Plus, every day we have video conferencing stand-ups, that is, every day there is some amount of personal communication through video communication. It seems to me that we have no problems due to the fact that we are distributed.
Moreover, I would like us to be more distributed. I would like us to be able to hire people, for example, in Silicon Valley. Those who are satisfied with everything there, and they do not want to move.
- Do you have a working language of English, Russian?
Russian. The comments in the code are always in English, all internal communication is always in English, and verbal communication is always in Russian, except in those rare cases. We still have a certain number of non-Russian-speaking employees, and of course we communicate with them in English.
- I see. Now another interesting story. Just a couple of weeks ago, Andrei Breslav released a post that many people paid attention to, in which, in general, he was actively looking for actually product managers. This question comes from here: are you unhappy with your product managers or just looking?
We are looking for people for a position called Product Marketing Manager (PMM). In fact, we have a very good team. As far as I know, we are satisfied with everyone who works for us in this position. Andrei was looking for a person specifically in Kotlin, where now PMM is simply not there. Or, for example, in IntelliJ IDEA there is only one person now, Andrei Cheptsov, engaged in product marketing. This is a great product, complex, with a lot of different features, and it is clear that there is work for the second person, and maybe the third one too.
- About a year ago Ilya Ryzhenkov came to Kotlin. Didn't he come to the PMM position?
Now, in fact, it turned out that Ilya is primarily concerned with libraries and infrastructure around Kotlin. He leads the team that makes the framework for Kotlin, and he writes the code himself. We are constantly experimenting with new frameworks for Kotlin. We already have one, but we are trying other options, looking for how it would be possible to do better.
- And what about Andrei Breslav?
Andrei, firstly, the chief designer of the language, and secondly, until recently, he was the project manager of the entire team. I recently took away the management of the IDE team from him - we already have 5 people who are involved in plugins for IntelliJ IDEA, and now I am their leader. Andrey just decided to stop programming recently, because he does not have enough time for this. We strive to make the release as soon as possible, and at the moment there are simply a number of open questions on the language design. We want to close them as soon as possible, and Andrei is primarily focused on this now.
“ Five people — in the IDE alone?” The last time I spoke closely with the Kotlin team, it was a couple of years ago, there were only, in my opinion, 9 people. Looks like the Kotlin team has grown a lot since then?
There are more than 20 people now.
- That is, does the company really invest in Kotlin?
Yes.
- Two or three years ago, there was a feeling that Java 8 would be released, and that's it, Kotlin would not be needed further. And now you have only increased your investment. Are you making any serious bet on him? Do you write on it yourself?
Yes, we now have a number of projects that are being developed at Kotlin'e, but not as many as I personally would like. For example, the next version of YouTrack is written on Kotlin. We have one unannounced product, it is also written on Kotlin. The new CRM system that I mentioned is all written on Kotlin from start to finish. This is a business critical decision, through it all our sales go through. We trust the Kotlin code for our sales, and this, it seems to me, says something.
The guys constantly take Kotlin from the master, that is, they do not sit on the release versions, but every few days they upgrade to the latest version of Kotlin from the freshly collected. At the same time, we are satisfied, everything works.
About Kotlin vs. Java 8 - in fact, this is some common misconception. First of all, Java is pretty much tied to its legacy. In the previous issue of No Slides, this was also discussed. Sergey Kuksenko said that it would be great to throw everything out and write from scratch. He probably had in mind the library and the virtual machine, but from the point of view of the design of the Java language, there are also decisions that were once adopted and which will be with us forever. For example, Raw Types, Wildcards, separation into primitive and non-primitive types and so on - this will not go anywhere, and no matter how much Java develops, this legacy will remain forever. By itself, the fact that we are not tied to this backward compatibility, we are already untied, is already giving us the opportunity to do some interesting things.
Now, when I talk to someone about Kotlin, and from time to time I give reports, I don’t mention any lambdas for the first 20 minutes or so - I just show different features of Kotlin. There is a lot of everything that does not affect the changes that occurred between the seventh and eighth Javs at all - just things in which we improve the part of Java that is not affected by all of this.
Then, of course, lambdas. We have lambdas, and they are better than in Java 8, because we have language-level inlining support. The point is that when you use a lambda just to cycle through some very simple function - for example, find all elements that are greater than 8. The naive way to translate this is to generate an inner class with one method that checks that number is greater than 8. Then you have the filter function, which, in fact, receives an instance of this class and applies it to each element of the collection, and there is code from which it is all called.
In Java, something like this works. It uses invokedynamic and method handle. Therefore, there part of this is generated at runtime. As we learnedat the Geekout conference, Java at the moment does not know how to inline it all. And we have language level support. We just say that the Filter function is placed inline. This means that this code, which is inside the lambda itself, and the function body itself are written directly to the place where we call. Accordingly, it turns out that we do not have any performance overhead. We do not have the creation of separate objects. We do not have any virtual challenges, God forbid, still megamorphic. Our code does exactly the same thing that would happen if you wrote everything with your hands without using a lambda.
- I understand that it is invokedynamic that is some kind of hindrance for inline?
No, I would not say that it is in the invokedynamic case, but inlining works so that he does not know how to lambda through the lambda and the function through which it is transmitted. Perhaps someday this flaw in the JVM will be fixed. As far as I understand, work is underway on this, but in Kotlin at the language level this is decided.
- I see. And when did Kotlin appear? How did such an idea come to mind?
As I recall, we started around the end of 2010. I will not say that it’s directly that I came up with everything. At some point, I came to Sergey Dmitriev (one of the founders of JetBrains - author's note) and said that it would be cool if JetBrains had its own language. For me it looked like a natural development. Our slogan is “Develop with pleasure”, in Russian - “Develop with pleasure”. We do quite a lot in the field of tools for people to really develop with pleasure: refactoring, compliance, highlighting errors - all that we can do very well for 25 different languages. Anyway, we see that the IDE has to make gags for the flaws of the language.
For example, IDEA can show an anonymous class in the form of lambdas. If you use Java 6 or Java 7, you write anonymous classes: new Instance, method declaration, body. In IDEA, it all collapses into a lambda. This is a pure plug for lack of language. We decided that we could just try to make a language in which gags would not be needed, which would be just nice to write.
IDEA, of course, is still useful and helps, but it is not some kind of crutch. There was a feeling that we could do this, because we have accumulated relevant experience with the development of IDE plug-ins and language support. There was a feeling that we had enough weight in the market for people to be interested. There was a feeling that we needed it ourselves, because we wrote everything there in support of this, the fifth, the tenth, and we ourselves program in Java.
All IDEA is still written in Java 6. We started using Kotlin, but everything else is written in Java 6. There was a feeling that this is something that can fire. It was clear that this was a long story, but I probably underestimated how long it was.
Then it sank some kind of grain, there it was also discussed with different other people in different other contexts and so on. Then we were very lucky to meet with Andrei Breslav, and we, in fact, began to make the language itself. At the initial stage, I also had a hand in it - I wrote the first version of the bytecode generator for the JVM. Then other people did this, and I switched to other projects. I haven’t been doing Kotlin for quite some time, and now I’ve returned to Kotlin.
And I also came up with a name. At some point, Dmitriev came and said that you need a name right now. Like, come up with a name - and stands above the soul!
- Do you like the name?
At first, we were a little embarrassed, because we have a German office, and in German the word Kot means something not very harmonious. But in fact, neither the Germans nor anyone else - did not hear this. So at first it was like a temporary name, code name, "Project Kotlin". We only later realized that we do not need any other name, we are not bad at all.
- Just recently it was announced that you and Sveta Isakova will write a book about Kotlin. Tell me, please, what kind of book is it, who was the initiator of this project, and how is the whole process of writing a technical book about the language arranged?
We want to make a Kotlin book specifically for Java developers. Kotlin, of course, is not limited to Java alone: it has support for compilation in JavaScript, but at the moment we are simply not ready to talk about it, because we have a lot of things there that will not be completed and will most likely be completed after the release of 1.0. Now we are focused on supporting compilation under the JVM: server development, web development, compilation for Android. Android is actually a hot area for us right now.
Accordingly, the book will be for people who already know how to program in Java, who know about Generics and about JVM. The presence of such a book will allow them to easily switch to Kotlin. The book will tell you exactly how to use Kotlin to solve their problems, and in particular, how to integrate Kotlin into an existing Java project.
An important property of Kotlin is that you do not need to throw or change anything - you just take one class and rewrite it on Kotlin. It becomes one and a half times smaller, two times more beautiful. Then you repeat the procedure, in general, with any code that you need to change for some reason. The code that you wrote in Java and don’t touch can remain in Java: it doesn’t bother anyone, it doesn’t harm anyone. Why make extra noise? And the code that you are actively modifying, you simply rewrite to Kotlin as it modifies. Actually, in IDEA we plan to go exactly this way.
Now about how the book appeared. It is well known that a book is a very useful thing for the popularity of a language. There is a book about Scala, which too many have read and praised, and the popularity of Ruby on Rails began with the book. So at some point, I realized that writing a book would be the most significant thing I could do. I talked with colleagues and they approved the idea. Sveta Isakova, an engineer at Kotlin, said she would also be interested in doing this.
We read how the process of writing a book about a programming language works. You need to have a Table of Contents (table of contents) and have the first chapter. With this you can go to the publisher. We put together a table of contents and began to prepare to write the first chapter. At this moment, a letter came in parallel from Manning, a rather abstract one, saying, let's talk about Kotlin. Not that they directly called us to write a book. We told them that we were going to write a book, and that if they were interested, then they say, let's proceed. They replied that they were interested. So we started work.
- What will the book be called?
"Kotlin in Action." In Action is the name of a series of their books.
- And how did Manning notice you in Kotlin’s plan? Do they somehow monitor the information space?
Yes, there are such people, called acquisition editors, whose duty is to look for topics about which it would be interesting to publish a book, and look for people who can write such a book. They came to us, and we agreed. There is mutual interest.
“ Then a little strange question.” Usually, when an author writes a book, he is paid a fee for this. Here you write a book about a product that is being developed not only by you, but by your company. How is this issue generally resolved? What are the approaches to this?
We just write a book, we get paid a fee. We will have a fee, we have spelled out in the contract - the usual absolutely royalty conditions.
- And does JetBrains just give you a time budget for this with Sveta?
Yes. The fact is that we do not have any strict regulations, they say, you can do this X hours a week. A book is not the only thing I am doing now. Perhaps I will regret it yet, but now I am combining the book with some other work.
- There are people like Venkat Subramaniam for whom writing books and training is generally the main activity.
I'm still a programmer, I can not help but program.
- I see. At one point, it was part of the culture in general.
So far, everyone is programming, both CEOs are programming.
-Interestingly, the Kotlin language in the form in which it will be released is not all that people who develop it think about. There is a feeling that you are still thinking for several years to come. Why did this sensation arise? I heard in one of Ilya Ryzhenkov's speeches about the Language Design Review procedure. He then explained that this is done so that you roughly understand what problems you may have in the future. Could you tell us more about this moment?
We have in our head a certain number of directions where language can develop. It is clear that over 5 years we have accumulated a certain amount of thoughts. We understand that this and this would be good to do. We roughly know, if we do this, in what place we will need to change the language. Accordingly, we now want to fix the syntax in such a state that those directions of changes that we plan do not require us to break backward compatibility with the code.
This is a completely new topic for JetBrains. Those involved in the development of IntelliJ IDEA plug-ins have been faced with the problem of breaking APIs all the time. At Kotlin, we want to avoid these problems. Moreover, we know that nothing will work if we have Kotlin’s compatibility at the same level as IDEA’s plugin compatibility. We commit ourselves that if you write code for Kotlin’s version 0.1, then the next version of Kotlin’s 1.X, and maybe 2.X, if something like that ever happens, they will continue this compile the code.
How much we can do, how much we are now laying the straw in the right places, is unknown. Life will show. Maybe we will have to make some clumsy compromises to do some things that we don’t have in our heads at all. At least, we try to fix those things that we want to change in the future in such a state that we have an opportunity for development.
- Can you give some example? Or is it hard?
Well, for example, we are thinking of making literals for collections. And we want to be able to use the colon character in collection literals.
- And what are literals for collections?
So that as in Python they could write something in square brackets, simply. Not
- That is, a free symbol, is this an example of such a gap for the future?
Yes, we just leave the opportunity for syntax development. And in fact, even more urgently - we just understand that we are now leaving some elements in release 1.0 that are not in perfect condition. Because we can’t do the release forever, we need to release it this year. Accordingly, we find such a variant of functionality, a state that can be expanded without breaking compatibility.
- Did I hear that the release will be this year?
I really want to. We now have a list of tasks that need to be done before the release. And we are trying to plan how much it all takes exactly time to understand exactly whether we will manage to release by the end of the year.
- A very interesting story, you mentioned - this is the history of the API for plugins. At one time, the NetBeans team struggled with this problem for a long time, and they solved it as follows: they took a special framework, it seems SigTest, and simply fixed the signatures of their methods, APIs, and introduced some rules by which the API can change. That is, relatively speaking, you can add a method, but you can’t delete it. And some other simple things. How did you do it?
For a very long time we did not solve it in any way, and could not solve it. Today in IDEA there is no Plugin API as a separate entity, there is no facade that represents certain functions that you use. In so many places, people have access directly to the IDEA internals. That is, the plugin descriptor can work directly with those classes that implement some kind of functionality. And therefore, it is very difficult for us to have compatibility, because it will greatly limit the development of the product.
The development of code over 15 years has been quite phenomenal. At first it was a monolithic Java IDE, which had no API at all. That is, when release 1.0 was made, we realized that making an API for plugins was two extra months of work, and then there was a very limited amount of money. It was necessary to release a product, and receive money from sales. Then in version 3.0 some kind of Plugin API appeared, then it was constantly expanding. Then came support not only for Java, but also for other languages. Version 5.0 introduced JavaScript in particular. As a result, it turned out to make an IDE, in which there is nothing about Java.
PyCharm has nothing about Java inside. It was worth a considerable amount of effort. Then we got the Kotlin compiler and Upsource. Code that was sawn along one axis to separate Java from non-Java, now had to be sawn along another axis to separate the UI from non-UI. There is no editor in the Kotlin compiler. There is a compiler. There is a file system, there is a PSI, that is, a model for working with code. There are several more such abstractions. But there is nothing visual about it.
Accordingly, it was a separate big job - to make it so that classes and platforms could be grouped in such a way that they satisfied these properties. There was no chance of maintaining API compatibility during all these transformations.
-To do this, everyone uses the versioning mechanism. You can say that inside the Major version you have a fixed API. From Major to Major, change the API and accompany the changes with guidelines or tutorials.
This we tried to do. All major perturbations usually occurred between major releases. We tried to write migration guides when we had big perturbations, however, we did not always succeed. Specifically, it has now become noticeably better, because now it seems like all the big perturbations are completed. We just have a build configuration on TeamCity, which for each build'a IDEA takes all the plugins from the plugin repository, and checks that none of them fell off. - Static signature checks?
Yes. Just checking that there won't be a NoSuchMethodError or anything else like that.
-And the ideas of putting all this into some kind of public API?
There is some kind of API.
- There is a division about the fact that this can be used, but this can not be better?
No, and I don't really believe that it will ever appear.
- About two years ago you left JetBrains for a while. Can you talk a little about this?
I left JetBrains on Google. At that time, I spent 10 years at JetBrains. I wanted to learn something new.
- What did you do when you left?
At that time, I was formally CTO, but to fulfill this role was not very successful. I was more likely a team lead of PyCharm and WebStorm. These were the two main tasks that I then dealt with. And, it seems, it was a pretty good moment to leave, because there was no big next task, in which I would definitely have to participate. It was to whom to transfer my projects.
I wanted to try something new. And I was ready for the option that I would return to JetBrains, and for the option that I would stay at Google. I discussed in advance the opportunity to return, and they told me: Yes, we will be glad to see you. In the end, it happened.
- If it's not a secret, what did you do at Google? What area did you work in?
I made development tools for the Google Cloud Platform. For example, there is such a thing - Сloud Debugger: you can put some dots on your production application, on the App engine, on the compute engine. But these are not breakpoints, these are points of information withdrawal. Right in the source code in this place you are going to call stack, the values of all local variables are collected. And all this is sent to you, and the process does not stop. That is, it really happens in production. You can set conditional breakpoints there. Finally, it is quite efficient in performance, that is, it does not lead to any noticeable slowdown. We were dealing with the front-end: the ability to look at the call stack in the browser, variable values, source code
-In the end, you didn’t like it there? Or were there any circumstances that forced you?
No, nothing forced me. At Google, it's pretty important to find your right place. Google is a big company, the company is very different. They have the slogan "do cool things that matter", that is, "do cool things that matter." But this is not true at all for all Google employees. That is, there are employees who do very cool things that are very important, but there are employees who do just the different work that needs to be done.
- Modern Google is more than 50 thousand people. And how many developers are there?
Thousands of 25-30.
-There are a lot of myths around Google, up to such that some employees complain that they can’t work, because there are two pools at the same distance to the right and left of their workplace, and they are tormented by the fact that they don’t know which one to choose . How true is this?
In fact, not consistent. That is, how to say? You can search for yourself in Google for quite some time. And you can do this with a very different degree of productivity.
- You decided to go the other way, you decided not to look. You decided to come back.
If I was ready to spend another 3-4 years on such searches, go to Mountain View for a year, try different teams, get to know a lot of different people closely, then I would be fine in Google after some time. But I just realized that now, today, there is Kotlin, from whom I am very rushing, and in which I can come and start doing, I really want JetBrains to work out with him. And do not look for anything.
There, again, these are some of the strongest people in JetBrains that I worked with - Andrei Breslav, Ilya Ryzhenkov and many other guys on the team. I'd rather be at JetBrains than at Google going somewhere. In fact, I am very glad that I went to Google, it was a valuable experience, I got the desired change of scenery. And I’m very glad that I’ve returned.
-You touched on the Kotlin team theme. It was formed somewhere in the 2011th year. The guys who were hired at that time (I just know quite a lot of people from this team) were actually yesterday’s graduates of universities. And then it turns out some incident. On the one hand, you then trumpeted at all angles that JetBrains supposedly gained a lot of experience in writing compilers and language support, and so now you will create your own language. And on the other hand, you hired yesterday’s students in large numbers, who obviously have no such experience. How so? How did this fact affect the development?
I think that influenced. For example, the team had to spend some time before the performance of the plug-in became such that it was possible to work normally with it. On the other hand, that some of the old-timers have always participated in the development. For example, Maxim Shafirov participated at a very early stage. And now he uses Kotlin, comes to design meetings, gives feedback on the development of the language. But in general, the team is now quite young. That is, there are several old-timers, for example, Valya Kipyatkov, I, Ilya Ryzhenkov. And in principle, this is enough, it seems to me, to convey experience.
- Were there any big mistakes in the stage in life in the development of Kotlin? That is, was it something that took you somewhere in the wrong direction for a long time? Or, in principle, this was not?
It was. I will not say that this is a mistake, it is rather a quest for a path that took time. One of the key points in the design of the language was that we want to solve the nullability problem so that we always know where our values can be null and where they cannot.
- The billion dollar mistake.
Yes, yes, she is. We live on the Java platform, and in Java these tools are not. Accordingly, we need to be able to work with Java code that does not have this information. Accordingly, in order to get this information for the Java class, we need to invent something.
Initially, we had a model of fixed annotation: there is a class, and somewhere next to it lies a file that came from somewhere, in which it is written in xml about each method that it is a method that has two parameters: the first is not nullable, the second is nullable, and return nullable. We wrote such a file for the JDK and decided that some other people would write such files for well-known libraries. And if you have your own project, then you can write annotations, just plain Java, directly in your code for your code. It didn’t take off.
Then we came up with KAnnotator, a solution that scans bytecode and does analysis. For example, if a parameter is passed somewhere that immediately references, then it is not nullable. And the returned value is also tracked. Accordingly, KAnnotator is an external tool that generates these same files with fixed annotations. In fact, this is a global data flow analysis in a program. It turned out to be a cool thing, but also not very practical. Because the source code breaks up into two independent parts - the code itself and these annotations, which may be lost or may be versioned. And we eventually admitted that this also does not work.
And now we have a simpler solution - platform types. These are separate entities, types for which we do not have nullability of information. They came from Java. When a person implements a Java interface method, he can write any nullability on parameter types. That is, he himself knows which nullability is correct, and either writes a question or does not write a question. Both compiler permits. And behaves in accordance with what the user wrote. But there is no separate file, no complication of the workflow. If you want, you can write annotations Nullable and NotNullable in the Java code - we still understand them.
The whole story with KAnnotator took quite a lot of our time and energy. But on the other hand, we did not know that this is a non-working option when we started.
-Not every day you write a new programming language. I'm not sure that if the team was significantly more experienced, it would save you from all the mistakes.
May be.
- If I understand correctly, you are now letting users decide for themselves where to build the perimeter of this null protection?
Perimeter is the boundary between Java and Kotlin code. That is, we still generate assertion if there are no questions there. Just the configuration of this perimeter is set by the user independently.
- That is, he can choose where on the call stack he will check this?
This is always verified at the transition level between Java and Kotlin.
-Okay Then maybe a little back to Google. When you went there, you probably expected something. And how much did this coincide with what you really saw?
Yes, I had all sorts of expectations, and not all of them came true. For example, the naive expectation that I will have a manager who will “manage” me will take care of my development and improvement. Not.
- But after all, in JetBrains, as I understand it, this was not like that either?
This was not the case at JetBrains, and I just wanted to see how it was.
- If not a secret, at what age did you get into JetBrains approximately?
I was 22, it turns out. I worked before various companies, I started working very early. In fact, the first work for which I was paid some money - I was about 17 years old.
- Was it programming?
Yes, programming. And the first work for which I was generally paid money was transfers. I was 13 then. Therefore, I already had some experience at that time. I had a successful OpenSource project, for example.
Returning to Google. Google does awesome fantastic things. You go to their site, print, it shows you in real time the search results for what you typed 100 milliseconds ago. You pressed a key, and after 100 milliseconds you see the search results for a phrase with this key, with this letter all over the world Internet. This is really very cool.
And looking at Google, you think that I’ll come there, and there’s just something completely magical, thanks to which it all works this way, and what it all holds on to. It turns out, no. It turns out there is no secret ingredient. The recipe, as I formulated for myself (probably this is the main thing that I took out from Google), is the recipe for how to make this work.
Firstly, a good storage system is very important. With replication, with scaling. That is, a storage system is really what everything rests on. It is not in vain that dozens of startups are appearing who write their NoSQL repositories.
- Do I understand correctly that this is the know-how of Google? And what is it, due to which the company is generally a technological leader?
Hard to say. In fact, Google talks a lot. And about BigTable, on which dofiga is still written. There is an article that says how it works. Then came the new Spanner system, to which quite a few services are now migrating inside Google, which does magical things in general - transparent replication between the continents. There is an article about her about how it works.
That is, there is no big secret, as such, and the word "know-how" is not very applicable. What is written in the article below is necessary correctly, well, correctly implemented. But this is not given to everyone. Accordingly, if you have good data storage, then you can and need to run many copies of your application. If you run anything on Google, then by default you run three instances. Accordingly, in comparison with this, I am very sad to look at TeamCity, Youtrack, which are all so monolithic, everything, Youtrack has a built-in database in general.
- Xodus .
Xodus, yes. There is no replication, no scaling ... That is, it is clear that such a solution at some stage simplifies life. But I say that I am sad to look at it. Because it is clear that Youtrack will not work as well as Google search.
- Vadim Gurov (former manager of Youtrack - author's note) said that once at one of the conferences they asked the Atlassian guys how many people they support SaaS version of JIRA. It turned out that 30 people. And in Youtrack, only one developer is involved in supporting the SaaS version.
This is the understaffed story I mentioned. Perhaps, from some point of view, it was not so resolved, as it seems to me.
The next ingredient to Google is feel free to use the resources. RAM is generally not very expensive. You can load all the data into memory, that is, have so much RAM to, for example, load the entire database of combat Youtrack into it, without attachments. You can put a server, or several servers in which there will be as much RAM as it should. Then do sharding, replication, science. The next ingredient, mandatory, is performance monitoring. That is, you must always monitor how this all works. 95 percentile lanency, median latency - you need to follow all this. And you just need to constantly work to improve everything that you have. And in the end, if all this is done, then it turns out Google Search.
-In other words, this is not some kind of magic and magic, but simply a streamlined process. Does this process only work in some top-notch search products? Or is this culture spreading throughout the company, across all products?
Differently. Everyone is told how to. They don’t even talk so much as examples show. But, for example, setting up monitoring, which I spoke about, is a rather large, difficult task. Which, moreover, does not end at any moment. That is, monitoring, alerting, something else like that - is continuously twisted. This alert is too noisy, he woke the programmer at night to run to repair, but nothing needs to be fixed. It was there, in fact, everything was fine, just the alert worked not on that condition. And there are projects that are assisted by SRE (Site Reliability Engineers), this is a role on Google, many people know about it, and talked a lot about it. This is one of the main things they do. All monitoring, alerting, resource allocation, proper cluster allocation.
And if you have SRE, then you are fine. But there are many small services that simply do not have SRE. That is, programmers themselves are engaged in support. And in fact, it takes a terrible amount of time and effort - maintaining the services in working condition. That the service has enough, that it does not crash, that it does not run out of storage quota, that it has a normal response time for all requests that are sent to it, that it is not DoS-it, and so on.
“ The last thing that hurt was the closure of Google Code.” Do you know something about this?
I actually even tried to save Google Code. The story is simple: Google Code was simply made on a rather old, unsupported set of frameworks, as is very often the case with Google. This is a problem that automatically happens when you have 20,000 programmers in your company. Programmers - they program. They like to do all sorts of frameworks, technologies, some middleware. And then products are built on this middleware. And then with this middleware it becomes uninteresting, or the people who did it go away and other people come and so on. And there is some new middleware, on which everything now relies on writing.
Google Code is a product written on a rather old set of technologies, which can be put into a state for a rather long and expensive time, so that it can be properly maintained and properly maintained. And plus, it’s like a service that allows you to host all kinds of things on the Internet, it has always been a big target for abuse of all kinds. Once you can upload files there, let's upload a porn, or, for example, some kind of grunted android application, or something else. And the people spent a lot of effort to fight this. There, at some point, the download functionality simply disappeared. But there were other ways of abuse ...
And now it just all dragged on, and there was a feeling that this, in fact, was not necessary. Because GitHub won, and all the same, all the same, life is all there, and Google at that time had a bunch of its projects on GitHub. And so they decided to close Google Code.
“ You say you tried to reanimate him somehow?”
I decided it wasn’t so, it’s bad and sad. And I decided to talk with relevant people. It seems I came up with what can be done. Then he came to people, but it turned out that they had already thought about all this, came up with, they have a 15-page document that says why this will not work, and ... "thanks for your feedback, but we ourselves know everything." If you think that the smartest, then Google is guaranteed this is not so. If you want to offer something - with a probability of 75%, other people have already thought about it. And they either put it in a plan and are going to do it, or they know why it will not work, or they know why it does not correspond to some goals and so on.
- Is this the effect of a large company or something else?
Google still has good people, strong people. This is the effect of Google recruiting.
- A couple of questions that interest me personally. Learn community ru_java, you participated in this community. Now the patient is rather dead. How do you think it happened? That is, the evolution of that, that is, that discus, it has shifted somewhere, has there been no need for such discussions at all? I’m not that there is something actively doing, as far as I remember. But there I answered some questions about IDEA. LJ itself is a rather strange platform. And now everything has switched to Habr, as it seems to me.
“ I expected such an answer from you.” On the habr, your last post is dated 2011, and your last call, attention, logged in, on the habr, is the end of 2013.
This is strange because I get a weekly newsletter with interesting materials.
-But, nevertheless, apparently, you have not logged in and answered comments for 2 years, and so on.
Yes, I apparently do not read logged in.
- Are you going to write in the near future? Are other people already doing this?
About Habr is worth mentioning separately. For PhpStorm, in its early stages, this channel was crucial as an opportunity to communicate with users. We constantly told what appeared there, what had changed. There was a discussion, there was always a lot of discussion, always the developers answered. PyCharm and I have always been actively involved in the discussions.
Probably should post something about Kotlin there. It’s just that no one is specifically engaged in social media promotion now, let’s say so. We write something on English-language resources, on Reddit we have discussions. There is a subreddit about Kotlin , something else. And for the Russian-speaking community, we probably just do not have enough strength. And again, I'm writing a book now. Therefore, I do not really want to fit into the fact that now I am writing something else big, different.
- And such, still personal. We have a new site JUG.ru , there is a cover photograph, I do not know if you saw or not. You are on it. What did you say on the old JUG? At JUG, I definitely talked about TeamCity, once upon a time, back in 2006, in my opinion, the year. And I was still saying something, but now I don’t remember. I am familiar with Yasha Sirotkin pretty well. - Now I want to officially invite you to your, probably, the next one already, or through your next visit to St. Petersburg. Yes, be sure to tell us something on the JUG. And maybe speak at the next conference. This is an official invitation to all viewers and readers. Yes, I’m happy to tell.
PS: And really tell:
What we talked about:
- how does IDEA develop, where does it go
- what is the difference between IntelliJ and JetBrains
- why the company has two CEOs
- what happens in Kotlin'e
- What difficulties did the Kotlin team face in developing the language?
- What is Language Design Review
- what is modern google
- why did google code
- Why Habr is important for IDE developers
Under the cutscene is the transcript of this interview.
- Good afternoon! This is “No Slides.” We have a graduation today with Dmitry Zhemerov, a veteran of JetBrains. Dima, let's start by saying a little about yourself: what did you do, including in JetBrains?
I joined JetBrains in 2003. I was the 28th or 29th employee, and during this time I managed to do a lot of things to do. At first I was the Omea project development manager- there was such an information aggregator, that is, it was a program that collected information from various sources such as mail, news, RSS feeds. He allowed us to do a search on all of them, to organize, categorize and so on. Then the Omea project was closed, and I switched to the IntelliJ IDEA team. We can say that I was at the origins of such an entity as the IntelliJ platform, that is, I was engaged in the translation of the monolithic development environment IntelliJ IDEA into a platform on the basis of which you can build IDEs for different languages, and not just Java. It is clear that a lot of people participated in this, but I also did quite a lot there. Then I worked on various IDEs based on this platform: I managed to work out both RubyMine, and PyCharm, and WebStorm.
-Where do you think IDEA is now moving as a product, as a platform? There is a feeling that some stagnation has occurred: as an IntelliJ IDEA user, it seems to me that nothing is happening. New versions are coming out, they may work faster, support new frameworks. So is it, isn’t it?
Faster is the most important thing. Speed is a feature that no one ever refuses. We have been investing in the quality of user interfaces, usability for a rather long time, so that everything is neat, smooth. Big features that change your experience as a developer also appear in some quantities. For example, Inline Debugging, which appeared in the 14th version. When you step through the code, you are immediately right in the code, in the editor you see the value of the variables that are calculated on these lines. Or, for example, shortly before that, support for several carriages appeared in the editor - this is also a big feature, which for many matters. What will be next, I really can’t say, because our development process is not planned very far ahead, and it’s just that a developer might suddenly get the idea: “Let’s deal with this functionality. Let me do it. ” He will go and do, and she will appear in the release, and everything will be great.
- About 3 years ago, Oleg Stepanov and Maxim Shafirov became CEO of JetBrains. First, why are there two? I do not remember any such cases when the CEO is two at once. What has changed over the years? What, perhaps, tasks and goals were set for them, if this is open information, and how successful is it? What has changed in the company?
Two CEOs - it turned out to be quite convenient, because there are two offices. Oleg lives in Munich, Max lives in St. Petersburg, and each of them is primarily responsible for what happens in their office. There is just a lot of work, and somehow they share this work among themselves. Max is now very actively involved in our entire sales infrastructure, that is, our internal corporate information system, if you like, in the store. So that users have a good experience in this place. Oleg is engaged in quite a lot of our server tools in order to make some unified solution from our zoo represented by YouTrack, TeamCity and Upsource, which can simply be put and it will solve all the problems with the development process that are in your team.
-That is, in addition to the separation by office, is there still a separation by product?
This is not so much for products as for areas of activity. With us, like everyone else in JetBrains, this somehow organically happens. When we implemented this scheme, we didn’t have it right away that we’ll agree on who is responsible for what - it’s just along the way that this activity, it’s necessary to deal with it, and, accordingly, the person who can take it on himself, takes it upon herself.
- How do you rate these 3 years? What has changed globally?
In terms of the organization of work in principle, JetBrains remained the same. Some rather large projects happened, for example, moving to our new office on Vasilyevsky Island, in which Sergey Dmitriev, who was previously the head of the company, did not take any part at all. Or, for example, in the organization of sales. A lot of things have changed during this time inside our company, some things could be done great.
- Two CEOs - is this an experiment that is ongoing? Are its results already visible, or is it still in the process? Or, in principle, any global changes were not expected?
No, in fact, in the first place, the motivation for this decision was simply very simple. At one point Sergey Dmitriev said: “I'm not young anymore, roughly speaking. "I can either continue to do further JetBrains, or I can leave JetBrains, which was made, to some other people from the team, and go go do some other activities.” Judging by the fact that for these 3 years he has been successfully engaged in other activities, without being distracted by JetBrains or very little by being distracted by JetBrains, then, in general, everything turned out. JetBrains grows during this time, successfully breaks and so on. This task has been fully completed.
- On your old sitethere is a record dated 2003 that you got a job at IntelliJ Labs. Now we know the company as JetBrains. Can you talk a little about the relationship between JetBrains and IntelliJ Labs?
It is very simple. IntelliJ Labs is the name by which the company appeared. In 2000-2001, the first versions of IntelliJ IDEA were released under the IntelliJ Software or IntelliJ Labs brand. Then, when it became clear that we want to make products not only for Java, respectively, the name with the letter “J” is not suitable for us. We put the letter “J” at the beginning and came up with the name JetBrains. IntelliJ Labs is still preserved in the name of the Russian legal entity of our company - INTELLIGEY LABS LLC. JetBrains is, accordingly, the legal entity of the company's headquarters.
-JetBrains, which was in the mid-2000s, and JetBrains of the 2015 model - do you feel any difference between them, and if so, what do you like, what do you dislike?
It is clear that a company of this size simply has a different level of personal connections. When I come to St. Petersburg from Munich, I walk around the office and see a lot of people about whom I often don’t even know: they work for us in the company, they went to visit someone or they just brought pizza. These are people whom I do not know; I don’t know what they are doing. The positive thing is that JetBrains has been chronically understaffed for a very long time, that is, all the time there was a state that there was more work than the people who were available to do it. It seems to me that recently in very many projects we have come out of this state - not in all, but in many. Roughly speaking, we have as many people as needed to really move products forward.
- Now you have 300 people, 400?
400 with something.
-This is mainly Peter and Munich, right?
Yes, we now probably have 80 people in Munich and, accordingly, 400 with something in St. Petersburg so far.
- Hundreds of people. Distribution creates any difficulties for you?
It does not seem to me that it creates. As a rule, people who have worked in the St. Petersburg office for some time and who are already familiar with the team with whom they work together leave for Munich. Accordingly, there is no such thing that you have some kind of sudden person in another office with whom you need to somehow build relationships. Plus, we often go to each other.
“ Do you often come here yourself?”
Every two or three months I’m definitely in St. Petersburg. Plus, every day we have video conferencing stand-ups, that is, every day there is some amount of personal communication through video communication. It seems to me that we have no problems due to the fact that we are distributed.
Moreover, I would like us to be more distributed. I would like us to be able to hire people, for example, in Silicon Valley. Those who are satisfied with everything there, and they do not want to move.
- Do you have a working language of English, Russian?
Russian. The comments in the code are always in English, all internal communication is always in English, and verbal communication is always in Russian, except in those rare cases. We still have a certain number of non-Russian-speaking employees, and of course we communicate with them in English.
- I see. Now another interesting story. Just a couple of weeks ago, Andrei Breslav released a post that many people paid attention to, in which, in general, he was actively looking for actually product managers. This question comes from here: are you unhappy with your product managers or just looking?
We are looking for people for a position called Product Marketing Manager (PMM). In fact, we have a very good team. As far as I know, we are satisfied with everyone who works for us in this position. Andrei was looking for a person specifically in Kotlin, where now PMM is simply not there. Or, for example, in IntelliJ IDEA there is only one person now, Andrei Cheptsov, engaged in product marketing. This is a great product, complex, with a lot of different features, and it is clear that there is work for the second person, and maybe the third one too.
- About a year ago Ilya Ryzhenkov came to Kotlin. Didn't he come to the PMM position?
Now, in fact, it turned out that Ilya is primarily concerned with libraries and infrastructure around Kotlin. He leads the team that makes the framework for Kotlin, and he writes the code himself. We are constantly experimenting with new frameworks for Kotlin. We already have one, but we are trying other options, looking for how it would be possible to do better.
- And what about Andrei Breslav?
Andrei, firstly, the chief designer of the language, and secondly, until recently, he was the project manager of the entire team. I recently took away the management of the IDE team from him - we already have 5 people who are involved in plugins for IntelliJ IDEA, and now I am their leader. Andrey just decided to stop programming recently, because he does not have enough time for this. We strive to make the release as soon as possible, and at the moment there are simply a number of open questions on the language design. We want to close them as soon as possible, and Andrei is primarily focused on this now.
“ Five people — in the IDE alone?” The last time I spoke closely with the Kotlin team, it was a couple of years ago, there were only, in my opinion, 9 people. Looks like the Kotlin team has grown a lot since then?
There are more than 20 people now.
- That is, does the company really invest in Kotlin?
Yes.
- Two or three years ago, there was a feeling that Java 8 would be released, and that's it, Kotlin would not be needed further. And now you have only increased your investment. Are you making any serious bet on him? Do you write on it yourself?
Yes, we now have a number of projects that are being developed at Kotlin'e, but not as many as I personally would like. For example, the next version of YouTrack is written on Kotlin. We have one unannounced product, it is also written on Kotlin. The new CRM system that I mentioned is all written on Kotlin from start to finish. This is a business critical decision, through it all our sales go through. We trust the Kotlin code for our sales, and this, it seems to me, says something.
The guys constantly take Kotlin from the master, that is, they do not sit on the release versions, but every few days they upgrade to the latest version of Kotlin from the freshly collected. At the same time, we are satisfied, everything works.
About Kotlin vs. Java 8 - in fact, this is some common misconception. First of all, Java is pretty much tied to its legacy. In the previous issue of No Slides, this was also discussed. Sergey Kuksenko said that it would be great to throw everything out and write from scratch. He probably had in mind the library and the virtual machine, but from the point of view of the design of the Java language, there are also decisions that were once adopted and which will be with us forever. For example, Raw Types, Wildcards, separation into primitive and non-primitive types and so on - this will not go anywhere, and no matter how much Java develops, this legacy will remain forever. By itself, the fact that we are not tied to this backward compatibility, we are already untied, is already giving us the opportunity to do some interesting things.
Now, when I talk to someone about Kotlin, and from time to time I give reports, I don’t mention any lambdas for the first 20 minutes or so - I just show different features of Kotlin. There is a lot of everything that does not affect the changes that occurred between the seventh and eighth Javs at all - just things in which we improve the part of Java that is not affected by all of this.
Then, of course, lambdas. We have lambdas, and they are better than in Java 8, because we have language-level inlining support. The point is that when you use a lambda just to cycle through some very simple function - for example, find all elements that are greater than 8. The naive way to translate this is to generate an inner class with one method that checks that number is greater than 8. Then you have the filter function, which, in fact, receives an instance of this class and applies it to each element of the collection, and there is code from which it is all called.
In Java, something like this works. It uses invokedynamic and method handle. Therefore, there part of this is generated at runtime. As we learnedat the Geekout conference, Java at the moment does not know how to inline it all. And we have language level support. We just say that the Filter function is placed inline. This means that this code, which is inside the lambda itself, and the function body itself are written directly to the place where we call. Accordingly, it turns out that we do not have any performance overhead. We do not have the creation of separate objects. We do not have any virtual challenges, God forbid, still megamorphic. Our code does exactly the same thing that would happen if you wrote everything with your hands without using a lambda.
- I understand that it is invokedynamic that is some kind of hindrance for inline?
No, I would not say that it is in the invokedynamic case, but inlining works so that he does not know how to lambda through the lambda and the function through which it is transmitted. Perhaps someday this flaw in the JVM will be fixed. As far as I understand, work is underway on this, but in Kotlin at the language level this is decided.
- I see. And when did Kotlin appear? How did such an idea come to mind?
As I recall, we started around the end of 2010. I will not say that it’s directly that I came up with everything. At some point, I came to Sergey Dmitriev (one of the founders of JetBrains - author's note) and said that it would be cool if JetBrains had its own language. For me it looked like a natural development. Our slogan is “Develop with pleasure”, in Russian - “Develop with pleasure”. We do quite a lot in the field of tools for people to really develop with pleasure: refactoring, compliance, highlighting errors - all that we can do very well for 25 different languages. Anyway, we see that the IDE has to make gags for the flaws of the language.
For example, IDEA can show an anonymous class in the form of lambdas. If you use Java 6 or Java 7, you write anonymous classes: new Instance, method declaration, body. In IDEA, it all collapses into a lambda. This is a pure plug for lack of language. We decided that we could just try to make a language in which gags would not be needed, which would be just nice to write.
IDEA, of course, is still useful and helps, but it is not some kind of crutch. There was a feeling that we could do this, because we have accumulated relevant experience with the development of IDE plug-ins and language support. There was a feeling that we had enough weight in the market for people to be interested. There was a feeling that we needed it ourselves, because we wrote everything there in support of this, the fifth, the tenth, and we ourselves program in Java.
All IDEA is still written in Java 6. We started using Kotlin, but everything else is written in Java 6. There was a feeling that this is something that can fire. It was clear that this was a long story, but I probably underestimated how long it was.
Then it sank some kind of grain, there it was also discussed with different other people in different other contexts and so on. Then we were very lucky to meet with Andrei Breslav, and we, in fact, began to make the language itself. At the initial stage, I also had a hand in it - I wrote the first version of the bytecode generator for the JVM. Then other people did this, and I switched to other projects. I haven’t been doing Kotlin for quite some time, and now I’ve returned to Kotlin.
And I also came up with a name. At some point, Dmitriev came and said that you need a name right now. Like, come up with a name - and stands above the soul!
- Do you like the name?
At first, we were a little embarrassed, because we have a German office, and in German the word Kot means something not very harmonious. But in fact, neither the Germans nor anyone else - did not hear this. So at first it was like a temporary name, code name, "Project Kotlin". We only later realized that we do not need any other name, we are not bad at all.
- Just recently it was announced that you and Sveta Isakova will write a book about Kotlin. Tell me, please, what kind of book is it, who was the initiator of this project, and how is the whole process of writing a technical book about the language arranged?
We want to make a Kotlin book specifically for Java developers. Kotlin, of course, is not limited to Java alone: it has support for compilation in JavaScript, but at the moment we are simply not ready to talk about it, because we have a lot of things there that will not be completed and will most likely be completed after the release of 1.0. Now we are focused on supporting compilation under the JVM: server development, web development, compilation for Android. Android is actually a hot area for us right now.
Accordingly, the book will be for people who already know how to program in Java, who know about Generics and about JVM. The presence of such a book will allow them to easily switch to Kotlin. The book will tell you exactly how to use Kotlin to solve their problems, and in particular, how to integrate Kotlin into an existing Java project.
An important property of Kotlin is that you do not need to throw or change anything - you just take one class and rewrite it on Kotlin. It becomes one and a half times smaller, two times more beautiful. Then you repeat the procedure, in general, with any code that you need to change for some reason. The code that you wrote in Java and don’t touch can remain in Java: it doesn’t bother anyone, it doesn’t harm anyone. Why make extra noise? And the code that you are actively modifying, you simply rewrite to Kotlin as it modifies. Actually, in IDEA we plan to go exactly this way.
Now about how the book appeared. It is well known that a book is a very useful thing for the popularity of a language. There is a book about Scala, which too many have read and praised, and the popularity of Ruby on Rails began with the book. So at some point, I realized that writing a book would be the most significant thing I could do. I talked with colleagues and they approved the idea. Sveta Isakova, an engineer at Kotlin, said she would also be interested in doing this.
We read how the process of writing a book about a programming language works. You need to have a Table of Contents (table of contents) and have the first chapter. With this you can go to the publisher. We put together a table of contents and began to prepare to write the first chapter. At this moment, a letter came in parallel from Manning, a rather abstract one, saying, let's talk about Kotlin. Not that they directly called us to write a book. We told them that we were going to write a book, and that if they were interested, then they say, let's proceed. They replied that they were interested. So we started work.
- What will the book be called?
"Kotlin in Action." In Action is the name of a series of their books.
- And how did Manning notice you in Kotlin’s plan? Do they somehow monitor the information space?
Yes, there are such people, called acquisition editors, whose duty is to look for topics about which it would be interesting to publish a book, and look for people who can write such a book. They came to us, and we agreed. There is mutual interest.
“ Then a little strange question.” Usually, when an author writes a book, he is paid a fee for this. Here you write a book about a product that is being developed not only by you, but by your company. How is this issue generally resolved? What are the approaches to this?
We just write a book, we get paid a fee. We will have a fee, we have spelled out in the contract - the usual absolutely royalty conditions.
- And does JetBrains just give you a time budget for this with Sveta?
Yes. The fact is that we do not have any strict regulations, they say, you can do this X hours a week. A book is not the only thing I am doing now. Perhaps I will regret it yet, but now I am combining the book with some other work.
- There are people like Venkat Subramaniam for whom writing books and training is generally the main activity.
I'm still a programmer, I can not help but program.
- I see. At one point, it was part of the culture in general.
So far, everyone is programming, both CEOs are programming.
-Interestingly, the Kotlin language in the form in which it will be released is not all that people who develop it think about. There is a feeling that you are still thinking for several years to come. Why did this sensation arise? I heard in one of Ilya Ryzhenkov's speeches about the Language Design Review procedure. He then explained that this is done so that you roughly understand what problems you may have in the future. Could you tell us more about this moment?
We have in our head a certain number of directions where language can develop. It is clear that over 5 years we have accumulated a certain amount of thoughts. We understand that this and this would be good to do. We roughly know, if we do this, in what place we will need to change the language. Accordingly, we now want to fix the syntax in such a state that those directions of changes that we plan do not require us to break backward compatibility with the code.
This is a completely new topic for JetBrains. Those involved in the development of IntelliJ IDEA plug-ins have been faced with the problem of breaking APIs all the time. At Kotlin, we want to avoid these problems. Moreover, we know that nothing will work if we have Kotlin’s compatibility at the same level as IDEA’s plugin compatibility. We commit ourselves that if you write code for Kotlin’s version 0.1, then the next version of Kotlin’s 1.X, and maybe 2.X, if something like that ever happens, they will continue this compile the code.
How much we can do, how much we are now laying the straw in the right places, is unknown. Life will show. Maybe we will have to make some clumsy compromises to do some things that we don’t have in our heads at all. At least, we try to fix those things that we want to change in the future in such a state that we have an opportunity for development.
- Can you give some example? Or is it hard?
Well, for example, we are thinking of making literals for collections. And we want to be able to use the colon character in collection literals.
- And what are literals for collections?
So that as in Python they could write something in square brackets, simply. Not
new ArrayListfrom the parameters, but just write something in square brackets, and that would be there new ArrayList. And, accordingly, we reserve a colon symbol, for example, so that it can be used in some role. - That is, a free symbol, is this an example of such a gap for the future?
Yes, we just leave the opportunity for syntax development. And in fact, even more urgently - we just understand that we are now leaving some elements in release 1.0 that are not in perfect condition. Because we can’t do the release forever, we need to release it this year. Accordingly, we find such a variant of functionality, a state that can be expanded without breaking compatibility.
- Did I hear that the release will be this year?
I really want to. We now have a list of tasks that need to be done before the release. And we are trying to plan how much it all takes exactly time to understand exactly whether we will manage to release by the end of the year.
- A very interesting story, you mentioned - this is the history of the API for plugins. At one time, the NetBeans team struggled with this problem for a long time, and they solved it as follows: they took a special framework, it seems SigTest, and simply fixed the signatures of their methods, APIs, and introduced some rules by which the API can change. That is, relatively speaking, you can add a method, but you can’t delete it. And some other simple things. How did you do it?
For a very long time we did not solve it in any way, and could not solve it. Today in IDEA there is no Plugin API as a separate entity, there is no facade that represents certain functions that you use. In so many places, people have access directly to the IDEA internals. That is, the plugin descriptor can work directly with those classes that implement some kind of functionality. And therefore, it is very difficult for us to have compatibility, because it will greatly limit the development of the product.
The development of code over 15 years has been quite phenomenal. At first it was a monolithic Java IDE, which had no API at all. That is, when release 1.0 was made, we realized that making an API for plugins was two extra months of work, and then there was a very limited amount of money. It was necessary to release a product, and receive money from sales. Then in version 3.0 some kind of Plugin API appeared, then it was constantly expanding. Then came support not only for Java, but also for other languages. Version 5.0 introduced JavaScript in particular. As a result, it turned out to make an IDE, in which there is nothing about Java.
PyCharm has nothing about Java inside. It was worth a considerable amount of effort. Then we got the Kotlin compiler and Upsource. Code that was sawn along one axis to separate Java from non-Java, now had to be sawn along another axis to separate the UI from non-UI. There is no editor in the Kotlin compiler. There is a compiler. There is a file system, there is a PSI, that is, a model for working with code. There are several more such abstractions. But there is nothing visual about it.
Accordingly, it was a separate big job - to make it so that classes and platforms could be grouped in such a way that they satisfied these properties. There was no chance of maintaining API compatibility during all these transformations.
-To do this, everyone uses the versioning mechanism. You can say that inside the Major version you have a fixed API. From Major to Major, change the API and accompany the changes with guidelines or tutorials.
This we tried to do. All major perturbations usually occurred between major releases. We tried to write migration guides when we had big perturbations, however, we did not always succeed. Specifically, it has now become noticeably better, because now it seems like all the big perturbations are completed. We just have a build configuration on TeamCity, which for each build'a IDEA takes all the plugins from the plugin repository, and checks that none of them fell off. - Static signature checks?
Yes. Just checking that there won't be a NoSuchMethodError or anything else like that.
-And the ideas of putting all this into some kind of public API?
There is some kind of API.
- There is a division about the fact that this can be used, but this can not be better?
No, and I don't really believe that it will ever appear.
A year without JetBrains
- About two years ago you left JetBrains for a while. Can you talk a little about this?
I left JetBrains on Google. At that time, I spent 10 years at JetBrains. I wanted to learn something new.
- What did you do when you left?
At that time, I was formally CTO, but to fulfill this role was not very successful. I was more likely a team lead of PyCharm and WebStorm. These were the two main tasks that I then dealt with. And, it seems, it was a pretty good moment to leave, because there was no big next task, in which I would definitely have to participate. It was to whom to transfer my projects.
I wanted to try something new. And I was ready for the option that I would return to JetBrains, and for the option that I would stay at Google. I discussed in advance the opportunity to return, and they told me: Yes, we will be glad to see you. In the end, it happened.
- If it's not a secret, what did you do at Google? What area did you work in?
I made development tools for the Google Cloud Platform. For example, there is such a thing - Сloud Debugger: you can put some dots on your production application, on the App engine, on the compute engine. But these are not breakpoints, these are points of information withdrawal. Right in the source code in this place you are going to call stack, the values of all local variables are collected. And all this is sent to you, and the process does not stop. That is, it really happens in production. You can set conditional breakpoints there. Finally, it is quite efficient in performance, that is, it does not lead to any noticeable slowdown. We were dealing with the front-end: the ability to look at the call stack in the browser, variable values, source code
-In the end, you didn’t like it there? Or were there any circumstances that forced you?
No, nothing forced me. At Google, it's pretty important to find your right place. Google is a big company, the company is very different. They have the slogan "do cool things that matter", that is, "do cool things that matter." But this is not true at all for all Google employees. That is, there are employees who do very cool things that are very important, but there are employees who do just the different work that needs to be done.
- Modern Google is more than 50 thousand people. And how many developers are there?
Thousands of 25-30.
-There are a lot of myths around Google, up to such that some employees complain that they can’t work, because there are two pools at the same distance to the right and left of their workplace, and they are tormented by the fact that they don’t know which one to choose . How true is this?
In fact, not consistent. That is, how to say? You can search for yourself in Google for quite some time. And you can do this with a very different degree of productivity.
- You decided to go the other way, you decided not to look. You decided to come back.
If I was ready to spend another 3-4 years on such searches, go to Mountain View for a year, try different teams, get to know a lot of different people closely, then I would be fine in Google after some time. But I just realized that now, today, there is Kotlin, from whom I am very rushing, and in which I can come and start doing, I really want JetBrains to work out with him. And do not look for anything.
There, again, these are some of the strongest people in JetBrains that I worked with - Andrei Breslav, Ilya Ryzhenkov and many other guys on the team. I'd rather be at JetBrains than at Google going somewhere. In fact, I am very glad that I went to Google, it was a valuable experience, I got the desired change of scenery. And I’m very glad that I’ve returned.
-You touched on the Kotlin team theme. It was formed somewhere in the 2011th year. The guys who were hired at that time (I just know quite a lot of people from this team) were actually yesterday’s graduates of universities. And then it turns out some incident. On the one hand, you then trumpeted at all angles that JetBrains supposedly gained a lot of experience in writing compilers and language support, and so now you will create your own language. And on the other hand, you hired yesterday’s students in large numbers, who obviously have no such experience. How so? How did this fact affect the development?
I think that influenced. For example, the team had to spend some time before the performance of the plug-in became such that it was possible to work normally with it. On the other hand, that some of the old-timers have always participated in the development. For example, Maxim Shafirov participated at a very early stage. And now he uses Kotlin, comes to design meetings, gives feedback on the development of the language. But in general, the team is now quite young. That is, there are several old-timers, for example, Valya Kipyatkov, I, Ilya Ryzhenkov. And in principle, this is enough, it seems to me, to convey experience.
- Were there any big mistakes in the stage in life in the development of Kotlin? That is, was it something that took you somewhere in the wrong direction for a long time? Or, in principle, this was not?
It was. I will not say that this is a mistake, it is rather a quest for a path that took time. One of the key points in the design of the language was that we want to solve the nullability problem so that we always know where our values can be null and where they cannot.
- The billion dollar mistake.
Yes, yes, she is. We live on the Java platform, and in Java these tools are not. Accordingly, we need to be able to work with Java code that does not have this information. Accordingly, in order to get this information for the Java class, we need to invent something.
Initially, we had a model of fixed annotation: there is a class, and somewhere next to it lies a file that came from somewhere, in which it is written in xml about each method that it is a method that has two parameters: the first is not nullable, the second is nullable, and return nullable. We wrote such a file for the JDK and decided that some other people would write such files for well-known libraries. And if you have your own project, then you can write annotations, just plain Java, directly in your code for your code. It didn’t take off.
Then we came up with KAnnotator, a solution that scans bytecode and does analysis. For example, if a parameter is passed somewhere that immediately references, then it is not nullable. And the returned value is also tracked. Accordingly, KAnnotator is an external tool that generates these same files with fixed annotations. In fact, this is a global data flow analysis in a program. It turned out to be a cool thing, but also not very practical. Because the source code breaks up into two independent parts - the code itself and these annotations, which may be lost or may be versioned. And we eventually admitted that this also does not work.
And now we have a simpler solution - platform types. These are separate entities, types for which we do not have nullability of information. They came from Java. When a person implements a Java interface method, he can write any nullability on parameter types. That is, he himself knows which nullability is correct, and either writes a question or does not write a question. Both compiler permits. And behaves in accordance with what the user wrote. But there is no separate file, no complication of the workflow. If you want, you can write annotations Nullable and NotNullable in the Java code - we still understand them.
The whole story with KAnnotator took quite a lot of our time and energy. But on the other hand, we did not know that this is a non-working option when we started.
-Not every day you write a new programming language. I'm not sure that if the team was significantly more experienced, it would save you from all the mistakes.
May be.
- If I understand correctly, you are now letting users decide for themselves where to build the perimeter of this null protection?
Perimeter is the boundary between Java and Kotlin code. That is, we still generate assertion if there are no questions there. Just the configuration of this perimeter is set by the user independently.
- That is, he can choose where on the call stack he will check this?
This is always verified at the transition level between Java and Kotlin.
-Okay Then maybe a little back to Google. When you went there, you probably expected something. And how much did this coincide with what you really saw?
Yes, I had all sorts of expectations, and not all of them came true. For example, the naive expectation that I will have a manager who will “manage” me will take care of my development and improvement. Not.
- But after all, in JetBrains, as I understand it, this was not like that either?
This was not the case at JetBrains, and I just wanted to see how it was.
- If not a secret, at what age did you get into JetBrains approximately?
I was 22, it turns out. I worked before various companies, I started working very early. In fact, the first work for which I was paid some money - I was about 17 years old.
- Was it programming?
Yes, programming. And the first work for which I was generally paid money was transfers. I was 13 then. Therefore, I already had some experience at that time. I had a successful OpenSource project, for example.
Returning to Google. Google does awesome fantastic things. You go to their site, print, it shows you in real time the search results for what you typed 100 milliseconds ago. You pressed a key, and after 100 milliseconds you see the search results for a phrase with this key, with this letter all over the world Internet. This is really very cool.
And looking at Google, you think that I’ll come there, and there’s just something completely magical, thanks to which it all works this way, and what it all holds on to. It turns out, no. It turns out there is no secret ingredient. The recipe, as I formulated for myself (probably this is the main thing that I took out from Google), is the recipe for how to make this work.
Firstly, a good storage system is very important. With replication, with scaling. That is, a storage system is really what everything rests on. It is not in vain that dozens of startups are appearing who write their NoSQL repositories.
- Do I understand correctly that this is the know-how of Google? And what is it, due to which the company is generally a technological leader?
Hard to say. In fact, Google talks a lot. And about BigTable, on which dofiga is still written. There is an article that says how it works. Then came the new Spanner system, to which quite a few services are now migrating inside Google, which does magical things in general - transparent replication between the continents. There is an article about her about how it works.
That is, there is no big secret, as such, and the word "know-how" is not very applicable. What is written in the article below is necessary correctly, well, correctly implemented. But this is not given to everyone. Accordingly, if you have good data storage, then you can and need to run many copies of your application. If you run anything on Google, then by default you run three instances. Accordingly, in comparison with this, I am very sad to look at TeamCity, Youtrack, which are all so monolithic, everything, Youtrack has a built-in database in general.
- Xodus .
Xodus, yes. There is no replication, no scaling ... That is, it is clear that such a solution at some stage simplifies life. But I say that I am sad to look at it. Because it is clear that Youtrack will not work as well as Google search.
- Vadim Gurov (former manager of Youtrack - author's note) said that once at one of the conferences they asked the Atlassian guys how many people they support SaaS version of JIRA. It turned out that 30 people. And in Youtrack, only one developer is involved in supporting the SaaS version.
This is the understaffed story I mentioned. Perhaps, from some point of view, it was not so resolved, as it seems to me.
The next ingredient to Google is feel free to use the resources. RAM is generally not very expensive. You can load all the data into memory, that is, have so much RAM to, for example, load the entire database of combat Youtrack into it, without attachments. You can put a server, or several servers in which there will be as much RAM as it should. Then do sharding, replication, science. The next ingredient, mandatory, is performance monitoring. That is, you must always monitor how this all works. 95 percentile lanency, median latency - you need to follow all this. And you just need to constantly work to improve everything that you have. And in the end, if all this is done, then it turns out Google Search.
-In other words, this is not some kind of magic and magic, but simply a streamlined process. Does this process only work in some top-notch search products? Or is this culture spreading throughout the company, across all products?
Differently. Everyone is told how to. They don’t even talk so much as examples show. But, for example, setting up monitoring, which I spoke about, is a rather large, difficult task. Which, moreover, does not end at any moment. That is, monitoring, alerting, something else like that - is continuously twisted. This alert is too noisy, he woke the programmer at night to run to repair, but nothing needs to be fixed. It was there, in fact, everything was fine, just the alert worked not on that condition. And there are projects that are assisted by SRE (Site Reliability Engineers), this is a role on Google, many people know about it, and talked a lot about it. This is one of the main things they do. All monitoring, alerting, resource allocation, proper cluster allocation.
And if you have SRE, then you are fine. But there are many small services that simply do not have SRE. That is, programmers themselves are engaged in support. And in fact, it takes a terrible amount of time and effort - maintaining the services in working condition. That the service has enough, that it does not crash, that it does not run out of storage quota, that it has a normal response time for all requests that are sent to it, that it is not DoS-it, and so on.
“ The last thing that hurt was the closure of Google Code.” Do you know something about this?
I actually even tried to save Google Code. The story is simple: Google Code was simply made on a rather old, unsupported set of frameworks, as is very often the case with Google. This is a problem that automatically happens when you have 20,000 programmers in your company. Programmers - they program. They like to do all sorts of frameworks, technologies, some middleware. And then products are built on this middleware. And then with this middleware it becomes uninteresting, or the people who did it go away and other people come and so on. And there is some new middleware, on which everything now relies on writing.
Google Code is a product written on a rather old set of technologies, which can be put into a state for a rather long and expensive time, so that it can be properly maintained and properly maintained. And plus, it’s like a service that allows you to host all kinds of things on the Internet, it has always been a big target for abuse of all kinds. Once you can upload files there, let's upload a porn, or, for example, some kind of grunted android application, or something else. And the people spent a lot of effort to fight this. There, at some point, the download functionality simply disappeared. But there were other ways of abuse ...
And now it just all dragged on, and there was a feeling that this, in fact, was not necessary. Because GitHub won, and all the same, all the same, life is all there, and Google at that time had a bunch of its projects on GitHub. And so they decided to close Google Code.
“ You say you tried to reanimate him somehow?”
I decided it wasn’t so, it’s bad and sad. And I decided to talk with relevant people. It seems I came up with what can be done. Then he came to people, but it turned out that they had already thought about all this, came up with, they have a 15-page document that says why this will not work, and ... "thanks for your feedback, but we ourselves know everything." If you think that the smartest, then Google is guaranteed this is not so. If you want to offer something - with a probability of 75%, other people have already thought about it. And they either put it in a plan and are going to do it, or they know why it will not work, or they know why it does not correspond to some goals and so on.
- Is this the effect of a large company or something else?
Google still has good people, strong people. This is the effect of Google recruiting.
Learn and Habr
- A couple of questions that interest me personally. Learn community ru_java, you participated in this community. Now the patient is rather dead. How do you think it happened? That is, the evolution of that, that is, that discus, it has shifted somewhere, has there been no need for such discussions at all? I’m not that there is something actively doing, as far as I remember. But there I answered some questions about IDEA. LJ itself is a rather strange platform. And now everything has switched to Habr, as it seems to me.
“ I expected such an answer from you.” On the habr, your last post is dated 2011, and your last call, attention, logged in, on the habr, is the end of 2013.
This is strange because I get a weekly newsletter with interesting materials.
-But, nevertheless, apparently, you have not logged in and answered comments for 2 years, and so on.
Yes, I apparently do not read logged in.
- Are you going to write in the near future? Are other people already doing this?
About Habr is worth mentioning separately. For PhpStorm, in its early stages, this channel was crucial as an opportunity to communicate with users. We constantly told what appeared there, what had changed. There was a discussion, there was always a lot of discussion, always the developers answered. PyCharm and I have always been actively involved in the discussions.
Probably should post something about Kotlin there. It’s just that no one is specifically engaged in social media promotion now, let’s say so. We write something on English-language resources, on Reddit we have discussions. There is a subreddit about Kotlin , something else. And for the Russian-speaking community, we probably just do not have enough strength. And again, I'm writing a book now. Therefore, I do not really want to fit into the fact that now I am writing something else big, different.
JUG and conferences
- And such, still personal. We have a new site JUG.ru , there is a cover photograph, I do not know if you saw or not. You are on it. What did you say on the old JUG? At JUG, I definitely talked about TeamCity, once upon a time, back in 2006, in my opinion, the year. And I was still saying something, but now I don’t remember. I am familiar with Yasha Sirotkin pretty well. - Now I want to officially invite you to your, probably, the next one already, or through your next visit to St. Petersburg. Yes, be sure to tell us something on the JUG. And maybe speak at the next conference. This is an official invitation to all viewers and readers. Yes, I’m happy to tell.
PS: And really tell:
Dmitry Zhemerov at Joker 2015
Experience using Kotlin with JetBrains
Kotlin is a new programming language for JVM, Android, and JavaScript that has been developed by JetBrains since 2010. The language is focused on writing elegant, compact and secure code, integrates freely with existing Java code and has powerful support in the IDE. Recently, the language has been increasingly used both internally by JetBrains and by external developers.
In the report, Dmitry will talk about using Kotlin in two JetBrains projects - in the sales support and license management system (this is a Web application made entirely on Kotlin) and IntelliJ IDEA (here Kotlin integrates into a large existing project). Various frameworks that appeared during the development process and which may be useful to you will be shown. Also, Dmitry will talk about what advantages they received from the implementation of Kotlin and what difficulties they encountered.
This and other Joker 2015 reports are on the conference website .
Kotlin is a new programming language for JVM, Android, and JavaScript that has been developed by JetBrains since 2010. The language is focused on writing elegant, compact and secure code, integrates freely with existing Java code and has powerful support in the IDE. Recently, the language has been increasingly used both internally by JetBrains and by external developers.
In the report, Dmitry will talk about using Kotlin in two JetBrains projects - in the sales support and license management system (this is a Web application made entirely on Kotlin) and IntelliJ IDEA (here Kotlin integrates into a large existing project). Various frameworks that appeared during the development process and which may be useful to you will be shown. Also, Dmitry will talk about what advantages they received from the implementation of Kotlin and what difficulties they encountered.
This and other Joker 2015 reports are on the conference website .