“So I realized that now I am a date engineer, and in a different way you can position yourself on the market”

Habr, hello! Another interview with our graduate is Nikolai Rekubratsky, team leader at XING. Nikolay spoke about life in Germany, about the resource on which he found work, how European specialists differ from Russian in their approach to work, about the demand for the data engineer profession in Europe, about projects in XING, etc.

- Kolya, what is your background and how was your career in data engineer and team lead?

- I became a Timlid about a year ago, but still try not to miss the opportunity to program a little when my hands reach. Before that, I was a developer for a long time and the one who is now called a date engineer .

image

In my last office, I was engaged in video advertising, built an analytical system for our clients. It was a startup that gradually grew until it burst. Business volumes grew, and with them - the amount of processed data, and at some point the tools that I used and the architectures that we used stopped working. As a result, I brought in new tools, new architectures, then lambda architecture was very fashionable.

Then there was a project for compiling profiles for hundreds of millions of users, also quite interesting. And in all these projects, I liked the challenge: there are some requirements for the response, for the consistency of the data, for something else. To meet these expectations and requirements, you need to rethink the current architecture very much. It was both a challenge and freedom of action at the same time, and that was exactly what attracted me.

- And how was your career path?

- I haven’t worked in such a large number of companies during my 11-year working life, XING is the fourth. And I started as a web developer, wrote a lot in Ruby in the Ruby on Rails framework. He worked in Moscow in an office that did a lot of interesting things related to video streams: both live broadcasts and interactive video. Also, this office was engaged in the construction of various video systems, organized the broadcast of the Olympics in Sochi, and then closed. All the companies I worked for have died. This, I hope, has nothing to do with me (laughs - author's comment).

- Does XING know about the fate of your past companies?

“XING is doing well.” Returning to Undev, I again liked the freedom to use all the technologies that I considered necessary. I managed to put on different “hats”: I was both a backend developer and a frontend, I was engaged in server administration, I wrote on a bunch of different languages. It was a very interesting experience.

After 5 years, I moved to St. Petersburg, moved to another company, which, in fact, was engaged in video advertising. I came there as a Ruby developer, but more and more interesting tasks appeared that required other approaches and other tools. At that time, I had a lot of time to read articles, blogs, books myself, which were devoted to the things that I worked with, and which solved some pain points of this work.

I acquired new skills, and after updating my resume on LinkedIn, I began to receive messages from recruiters. They offered vacancies in which the term "date engineer" appeared. Then I realized that the area in which I worked got its name, has its own specifics, and now these people are called “date engineers”.

And so I realized that now I am a date engineer, you can position yourself in the market in a slightly different way. I started watching vacancies and ended up at XING. XING is a German professional and social network that is a leader in the German-speaking world, but it is not represented in other languages.

- With such a strong developer background, why and at what point did you go to the Big Data Specialist program ?

- It happened when I worked at Virool, she was engaged in video advertising. At that moment, I had already built an analytical system that used Spark. Was in the middle of the road to building a data management platform. And I had questions, it was difficult to understand how to build architecture in certain cases in order to efficiently process data. There were many such questions, and at some point, one of my colleagues threw a link to this program into the chat. I said that I was interested, the office supported me, paid for the participation. At the program, classes about DMP were very important for me, I found there answers to many questions that tormented me. On the program, we did a lot of things with our hands, we read the theory to us, but all the same, you get the most benefit when you start applying knowledge to your tasks. And I was so lucky: there were no individuals at work, who would be engaged in machine learning, and I was able to put my new knowledge into practice on the task of predicting banner clicks. It was a very interesting project, it lasted a very long time, and I dealt with it for the rest of the time: it was necessary to develop models, somehow improve them, then put them into production with my own hands. Since the company had a startup atmosphere, there were not very many employees, and there was no end to the work, I did all this.

- You said that many different companies were knocking on you, Russian and foreign. Why XING? Why move to Germany? Have you considered other countries?

- I'll start from the end. I considered, of course, other countries. The last company had the opportunity to work in San Francisco. My wife and I lived there for several months, looked. In the end, they decided that San Francisco is not very suitable for family life. Maybe it’s very good for the young and courageous, but it seemed to us that this city is not very suitable for us. But I didn’t want to work somewhere in Russia in order to get a different experience. It was my goal - to work in an international company, to see how things are done differently. And I wanted it to be not very far from Russia.

We began to look at Europe, namely England and Germany, two economic locomotives. The rest of the countries are not so widely known for their IT companies. In England, I was a little confused by stereotypes about bad weather at a subconscious level, so I looked more towards Germany. And there is some irony in the fact that I now live in Hamburg, and this is the same Petersburg in Germany in terms of weather, it is quite rainy here. But since I had lived in St. Petersburg for the last 4 years, after it any weather seems good, wherever you move.

I looked at different companies in Germany. At first I thought that with my store of knowledge in advertising, it would be easier for me to apply them in an advertising company. But there were no companies where the match with the team and the tasks that they had would turn out. I did not see offices interesting for myself or I was not satisfied with people and their attitude.

- How did you look for work? Did you post a resume in addition to updating your profile on LinkedIn?

- There is such a HoneyPot service as dating service for developers. You place a resume, indicate your requirements and expectations, and companies begin to write to you. I talked with several companies there, but there was a feeling that the company was about to write, which is even better. It seems that this one, in which you have already gone through several stages, is nothing, but maybe there is still something better.

And at some point they wrote to me from XING, and not for the position of the engineer date, but for the position of the Ruby developer. I also thought that it was not interesting to me, then I read about the company, it was a co-organizer of competitions on recommendation systems for two years in a row, and I thought: “Well, okay, it might be interesting to talk with them.” At the first interview, I talked about my background, about what I know and can do, and they said: “we see that you have different expectations and a different profile, we have a vacancy, engineer date, let's switch to it.” So I continued to communicate with them, for several months this whole gimmick lasted from the interview, there were many stages. What caught me the most was the last stage - a meeting with the team.

- How was your meeting with the future team?

- Usually you go to a restaurant for lunch and talk there for an hour, eat, and in such a relaxed atmosphere you can understand what kind of people you want to work with them. And for me this was the moment when I decided what I want, it is interesting to me. We got together with my wife and child and moved. The company provides housing for a while until you find your own, but it is not only for you, the apartment had five rooms, and almost all were occupied. And life with a child who likes to wake up at 5 a.m. was a test for our neighbors. But, probably, this motivated them to quickly find their home. Then we also found our refuge.

image

- Tell us about the projects that you are doing with the team now.

- We have several areas, they are all connected with users, because we have a team whose focus is specifically on user data - recommendations and content production. There is another department that deals with vacancies, news, and we focus on user data. But at the same time, Xing is a rather large office, and there are about 60 teams that do different things, namely product teams, they saw different parts of the site.

There are two interesting projects in the recommendations section. We are now exploring the strength of connections between users based on various signals. Now there is just information about the presence of communication between users, but we do not have an understanding of how strong it is. There are some hypotheses that the student is now engaged in; she writes her master thesis about the strength of connections between users based on the network topology.

There is another project, which is also associated with recommendations, we are trying to understand why the user visited our site right now, what is his purpose right now. That is, such an emphasis is precisely on real time data, that is, to look at the session histories of this user at the real time and understand where he is going, what he needs, why he wants to add this person or that, and based on this, change our algorithm to provide him the best recommendation.

I can also say about content generation. Two new stories have recently appeared: the anniversary of the user's work in the company and when a new unfamiliar colleague appears in the company, the user. Here the interest is that the user indicates the company in free form, and he can specify anything, and sometimes it is difficult to compare different names and understand that this is the same company - either a typo or a division is called a little differently , and we want to understand that this is one and the same entity. If we take a company of 10 thousand employees, for example, then we do not want to show the appearance of a new person to everyone, but want to make a more personal recommendation based on interests and other things.

- Tell us about the data engineering tools that you use most often in your work and why. Which is your favorite?

- In our stack, with which we have to work more or less regularly, these are Nive, Spark, Kafka, Cassandra, Akka Streams, ElasticSearch. My favorites are Akka Streams and Kafka, because I like working with fresh data that are processed in real time, well, Kafka has helped me out more than once in my career, for that I really love her. From what I use most often as a manager, it is, rather, communication with people, which is very important, and many date engineers underestimate this tool.

- Most companies do not separate the data scientist and the data engineer, do not understand that these are two different profiles of a person, and try to look for a team of universal specialists who can do everything: collect data, process it, and prepare the model, and put it into production. Why do you think this is happening and is this approach justified?

- I think that this is normal, it comes from the resources available to the company. It’s very difficult for a small startup to hire the entire line of necessary specialists: QA, backend, frontend, mobile developers, data engineer, data scientists, anyone else. They just won’t pull, in startups they hire developers who are ready to do everything. When companies grow, then they can afford to have specialists who are focused on a specific area. In large offices, you can often see the separation into data engineer and data scientist. There is such a separation in XING, but at the same time, it is expected that data scientists are able to write production code, for which it is not a shame.

- Since you are now in a managerial position, looking for people to join your team, what hard and soft skills are important when choosing a candidate? And according to your observation, what is missing? Which requires more development if we are talking about those people with whom you spoke, hiring yourself as a team.

- As for hard skills: it is very difficult to find a person who matches your stack, for large companies the stack is very specific. For example, when I was hired, I hit the stack of this particular office very well, because I had Scala knowledge, which is used mainly for all tasks related to date engineering. At the same time, I had a Ruby backgroud, and in XING a lot of product stuff is written in Ruby. My background matched very well. But when hiring employees, I do not expect that they will have a perfect hit either, including on the data engineering stack.

It’s great if a person knows the fundamental things that are difficult to learn: to program well or to understand statistics. It is difficult, for example, to start writing well in Scala, it takes time, because the language is not easy; but just start writing in Hive, because it's SQL, and you just sit down and write, and when someone starts to swear that your requests are being laid by the server, you start to figure out what is going on there. We hire people who have something that matches our stack, and who have a very important trait for us - the ability to learn. It is important for my team that people are curious, so that even if they don’t know something, they can quickly master, figure it out, and somehow get this knowledge. Learning ability is one of the important skills that we look at when we hire.

- Kolya, tell us about your team.

- XING has an interesting structure: many teams have three managers each - agile project manager, team lead and product manager. Two managers - team lead and product - are constantly with the team. The product comes up with what features we can implement, and how to improve existing ones. Timlid is me, I am responsible for the technological solution and the development of people in the team. The Agile project manager looks at how the team works and gives advice on what can be improved in the processes or is trying to resolve some external conflicts. And besides these three managers, there are five developers: one QA engineer, one full stack developer, and three date engineers. Another person I did not mention, a student, is writing a Ph.D.

- What countries are your team members from?

- It is impossible to find specialists only in Germany, we have a very diverse company. Not as international as, say, most companies in Berlin, but still quite international. I have two colleagues from Ukraine, a guy and a girl from Germany and a girl from the Netherlands. Our agile project manager from Croatia. Inside the company, the official language is English, everyone communicates in it.

- The next question is in continuation of the previous one: do the Europeans differ from ours in your observations? You have worked in Russia for a long time, now you already have experience working with Europeans.

- I have some experience working with Europeans, but it is limited to one European company, so I can only talk about what I see. What is very noticeable: the amount of communication among developers - what we do and why. This implies a lot of synchronization and meetings, but everyone has a very good picture of what is happening, why and why.

- These meetings are triggered by your product?

- This is a company culture that goes from the top. We have a very open guide that regularly tells what is happening, what kind of news, and listens very carefully to the feedback about what people like, what they don't like, how to change it. There are tools to give this feedback to management - anonymously or openly. The feedback culture at all levels is very well organized. And this is the first thing that is very different from my previous experience in other companies.

The second thing: the balance between work and personal life is very clearly observed. By 6 o’clock the office is empty, people do not feel the need to sit in the evenings or at night in an attempt to meet the deadline, there are no rumors. In Russia, work often goes beyond the usual working day, in Germany people are very attentive to this and are not eager to stay after 18:00, everyone understands this, and is encouraged at all levels of management. They have a lot of their traditions: they love to sit in a cafe on Saturday morning, ordering brunch. They just enjoy life, they are interested out of work.

- About the office empty by 6 pm: is this due to the fact that the work is planned correctly, and there is no excess of tasks? Or, relatively speaking, tomorrow is the deadline, there are tasks that are not done, and the person gets up at 6 o’clock and leaves because he has a personal life?

- The deadlines (from what I saw) are not so tough that tomorrow there will definitely be something in production. Due to the feedback culture, if something does not succeed according to plan, it becomes obvious not at the last moment, but in a few weeks, and expectations can be adjusted.

- Are you satisfied with life in Germany?

- Yes, I am very pleased with life in Germany, I am glad that I moved there, that I moved to Hamburg. I knew about Berlin and spent a week there before starting active interviews in Germany. I didn’t do research in Hamburg, but it turned out to be a very good city, and I like it even more than Berlin. Now I compare many cities to which I come, compare with Hamburg, and most often Hamburg wins.

- Do you want to move somewhere professionally?

- I’m trying to understand how comfortable I feel as a manager, because, on the one hand, it seems that in this position you can have a greater influence on the development of the product and thus on the whole company. But, on the other hand, you find yourself further from the direct use of technology, sometimes I miss this. And so far it has not been possible to find a balance that would suit me, so that, on the one hand, I have enough time for managerial activity and for directly practicing something applied. I clearly understand that at the moment I do not want to grow higher in the hierarchy. Because then some kind of coordination of budgets, tablets in excel, and this will be the closest to data processing. So I don’t aspire there yet.

Also popular now: