Highload ++ 2014 through the eyes of a speaker who loves Processing Large-Scale Graph Data in a variety of bad ways

Hello!

You probably heard about the conference of developers of high-load systems Highload ++ or you participated or spoke or even stopped going to it, cursing it in social networks.

I have a very reverent attitude towards the Highload conference, because my first trip to it in 2012 stabilized the chaotic knowledge at that time in a sort of ordered chain, structure and provided the broadest overview of the current Highload technology market in Russia and abroad.

In 2013, I enjoyed watching an online broadcast in the evening from the Tamtek office, discussing with my colleagues the advantages of Rabbit MQ, the Dublgis attempts to cram everything in Neo4j, as well as the report of our colleague Denis Nelyubin on comparing the performance of NoSQL databases (Aerospike, Couchbase, Mongo, Cassandra ) on our test framework, in the development of which I myself participated a year earlier.

On March 14, 2014, on my birthday, I sent an application describing the report to Highload and began to wait. After six months, I received confirmation and hit the road, hoping not only to meet colleagues working in related fields, but also to raise my Big Data consciousness to a new level.

And the first day came ... and the gates opened ...

Long before the approach to the conference venue in the metro and at the stops, the admin-programmer audience began to concentrate, very different from the average Muscovite in their leisurely and relaxed manner.

The World Trade Center greeted me with the pathos of a hermes-like naked guy, which alluded to the high cost of the conference and promised a wide scope and fun inside.



After the participant defended in a perfectly sharded and balanced lineup, he could climb up to the main hall and join those waiting for the first big reports (keynote).



Those who arrived early managed to see a motivating video from the organizers.

Overall feel for the content

The morning began with an introductory report in which two lecturers, drawing attention, tried to clarify the key terms often used in the industry, using simple pictures and concepts.



On the sidelines on the first and second day it was really hot: for those who like to stare at short skirts - there were their stands, for those who like to talk with a developer from MongoDB - their own.



In general, such a useful thing as techtalks is slowly penetrating into Russia: small discussion platforms outside the main lecture space. And it’s nice that at Highload these spaces were meaningfully filled with smart and sociable engineers from different companies working on the cutting edge. The guys who were opposite each other, preaching Mongo DB and PostgreSQL especially tried.



Each speaker's speech was preceded by a cartoon drawn in real time. They all lie here . Here's an example of the process of drawing my “face”.



The reports went one by one: eminent companies in which the speakers work sounded more and more serious. However, most serious companies were in no hurry to disclose the secrets of their main services, with which they have income, but rather shared the architecture of some secondary solutions. On the one hand, it’s normal to not listen to Badoo architecture every year (it doesn’t change so fast), on the other hand, it starts to seem that a “cool” company does not always solve the problem in an optimal way, and even, on the contrary, can blindly close its eyes for some flaws.



It is possible that because of the constant high workloads at work, speakers often could not find enough expressive words in Russian except for the first ones that came to mind. Someone does it, but for the majority of young speakers who blindly copy the manner of performing the main stars, this turns out badly and looks out of place.

This year's trends

The main trends of this year were the complete dominance of PostgreSQL, the fall into the shadows of MySQL (it was not scolded only by the lazy), the all-floating Docker, as well as the dragon of the RTB world lifting its head. However, all the reports that in one way or another related to the RTB topic (based on their descriptions) were more about languages, databases and archiving algorithms, rather than opening the tidbit of architecture and mutual dance of services.

It is worth noting an unusually successful idea with a separate training track, ideally suited for experienced developers, whose projects have slowly become “heavy” recently, but the “relief” recipes on the Internet seem confusing and vague.

Also, a rather successful find was the arrangement of reports of similar subjects one after another in one room, this allowed for two or three reports to see the problem from different angles. The most successful and convex in my opinion were the family of reports on PostgreSQL, cartography, working with indexes and MongoDB.

In the cartographic block, the presentation of the cartographic service of our new national search engine Sputnik, based on the data of the Open Street Maps project and the js library of the Ukrainian developer Vladimir Agafonkin Lefleat, was pretty amused.



The speaker turned out to be a very serious person, well versed in how to render tiles and correctly “update the World once a month”.

Count, hefty ...

On the evening of the second day, I made a presentation “Thorny path to the Large-Scale Graph Processing”, where I talked for a long time and in detail about modern problems and tasks on large graphs, as well as their processing tools and ways to solve some problems on individual types of graphs (road , web, social).



Two consecutive reports on graphs, graph DBMSs, methods of processing them allowed me and the students to really dive deep into the subject area, so I think it’s a good idea to build reports on similar topics in one track.

If you are worried about the fate of large graphs, NoSQL databases, or you are just a person with a broad outlook and decent manners, then you can find me on VC, look through the blog or make friends in G + and twitter .

By the way, all presentations are already available at the link here.



At the conference there were many reports devoted to dry theory and complex mathematical models and algorithms, an insane amount of circuits and graphs that are difficult to fit tightly into one head. Those who came for the first time to such an event could really enjoy a high-quality training track, which explained the basic terms, basics, popular cases of highly loaded systems, for example, you could learn all about the operation of indexes of various databases.

It seemed to me that this year there was too much bias towards reports for DevOps and database administrators, at the same time, the evolution of the project from the first storm loads to well-thought-out solutions was poorly presented, after all, most of the participants go for such recipes.

Nevertheless, from the Highload conference and from all its participants, the spirit of practice breathed. He was saturated with all the conversations, he was invisibly present during handshakes and exchange of business cards. It was clear that many people would start new interesting projects after talking on the sidelines.



Conferences end and we always drive home with a slightly changed consciousness to further configure our servers and write new code. It’s just that the code will be a little different, and the settings a little better. And the response time is a little less ... For this, after all, all this is spinning.