October 2, 2012 at 10:59Autumn Technology Forum: Focus on Web Development
October 17, we will host the autumn Forum of Technologies Mail.Ru Group 2012 . 24 speakers were invited to the Autumn Forum, reports will be held in 3 streams.
We are holding the Technology Forum for the fourth time. In the spring we made a special version of the Forum for system administrators. This fall, as well as last year , the topic will be web development, in the broadest sense.
I would like to introduce you to topics and speakers, technologies and ideas. We invited several interesting western speakers to the Forum. For the first time we will tell a lot about what, live demonstrations and master classes are waiting for you. In this post, I would like to introduce you to the topics that will be raised this time, and with our speakers.
Ask questions in the comments - speakers will still have time to take them into account in their presentations. And prepare questions for the Forum, we love and appreciate them very much. Participation is free, but the number of places is limited,registration is required and will close soon.
Maxim BABICH , head of the project development department for Money Mail.Ru, will talk about how Mail.Ru Money is arranged inside, how to make such a system scalable and fault-tolerant, which atypical problems for other highly loaded services will have to be solved.
This knowledge is especially useful for those who develop projects related to electronic commerce (including payment aggregators, billing systems, etc.).
Money Mail.Ru - A payment system that currently processes hundreds of thousands of transactions per day. Earlier, almost no information was published about the Mail.Ru Money device. Over the past year, this service has grown and grown very seriously. If you are interested in any specific aspects of building a payment system, please unsubscribe in the comments - we will try to touch on these topics in the report.
Denis ANIKIN, Technical Director of Mail.Ru Mail, will give a presentation on the experience of launching HTTPS in Mail.Ru Mail. The well-known HTTP traffic encryption technology, used in browsers and servers for more than 15 years, has begun to be fully used relatively recently on loaded global projects and services. In these matters, there are a lot of subtleties associated with configuring SSL / nginx, adapting and rolling server software. Denis will be happy to share all this information in his report.
Denis knows a lot of technological subtleties and is responsible for the fourth in the world in attendance mail service.
At the last Technology Forum, Denis talked about how high loads are maintained in Mail.Ru Mail .
Alexey MASHANOV, the leading programmer of the social network My World, will tell you how to centrally collect logs from a cluster in almost real time. This problem is even relevant for a small group of servers working on one task, what to do when there are hundreds or thousands of these servers? There are a number of ready-made products for this, and Alexey will tell you why the Scribe log server was chosen from among them.
Scribe is a real-time log aggregator developed by Facebook and submitted to Open Source in 2008.
Alexey has been working at Mail.Ru Group since 2010, he is engaged in optimization, statistics, and improvement of the technical processes of the My World social network.
Boris WOLFSON - CTO HeadHunter. His report at the Technology Forum will be devoted to the balance between the quality and efficiency of the release of new "features." The need to maintain this balance is facing every development manager. Boris in his report will share the system approach adopted by Headhunter.
Ward Cunningham coined the term “technical debt” for this situation ( wiki , habrahabr) According to this metaphor, the quick and inaccurate addition of functionality leads us to technical debt, which the inventor of wiki technology compares with financial debt. Many teams, increasing technical debt, fall into the “debt hole” and, as a result, slow down the speed of their work. Boris will tell you why and in what situations a technical debt appears and how it can be purposefully reduced. This report will be especially interesting to technical managers.
Pavlo BARON , leading architect of the German consulting company Codecentric, will talk about how to properly build distributed systems to work with large amounts of data. Pavlo often speaks at IT conferences around the world, is the author of three books: Erlang / OTP, Pragmatic IT Architecture and Fragile Agile. The report itself will be associated with Big Data, but on the sidelines Pavlo will be happy to share his knowledge and experience in other matters, where he is a recognized expert - Erlang / OTP, Agile, Java.
Pavlo will speak in English, we will provide simultaneous translation. In the process, it turned out that our speaker had long-standing Ukrainian roots, and he was slightly familiar with the Russian language. So it is possible that the audience and Pavlo will begin to understand each other much faster than is usually the case with the participation of the translator.
Alexey VOROPAEV, the head of the ranking department of the project Search Mail.Ru, will reveal the topic of machine learning: what it is, how it works, and whether it can be applied in your projects. Alexey will talk about popular machine learning algorithms with and without a teacher, and also show examples of their application (including in the Mail.Ru Search project).
Before becoming the head of the ranking department, Alex was engaged in the development of a search server and indexer. Prior to Mail.Ru Group, Alexey worked on a distributed enterprise-level traffic analyzer at Perimetrix .
The report will be especially useful to programmers and analysts dealing with large volumes of data.
Sergie MANCILLA , lead developer of Ajax.org, will give a presentation on "Architecting Node.js JavaScript for Large-Scale Projects".
Sergi is a recognized JavaScript expert and leads the development team for the Cloud9 IDE in Amsterdam , one of Node.js.'s most famous online development environments.
Prior to that, Sergi worked as a senior developer at TomTom , where he led the team building the JavaScript framework for user interfaces.
Victor STARODUB , Mail.Ru Mail programmer, will talk about his experience in implementing the IMAP protocol in the mail server.
IMAP replaces POP3twenty years ago, but, despite this, it is a "tough nut" for large mail services. First of all, because the number of email clients, multiplied by the complexity of the standard, makes it necessary to take into account many exceptions in the server logic. Secondly, the IMAP principle itself implies a permanent connection between the client and server, unlike session POP3. In the case of millions of connections per hour, this feature turns into a problem. Victor will tell about solutions to this and other problems in his report.
Luca GARULLI , CEO of NuvolaBase Ltd , will talk about how to create a fail-safe replicated cluster database that can be easily scaled only by adding new nodes to the cluster.
Luc - author of OrientDB - graph, document-oriented database of the NoSQL class, 100% implemented in Java. It supports an object-oriented database interface that runs on top of a document-oriented layer. Implemented a powerful SQL-based query language, cluster architecture, master-master replication, full support for ACID transactions, natively supports HTTP, RESTful and JSON protocols without using third-party components, supports server-side scripting. According to a number of tests on conventional equipment, OrientDB allows you to save 150,000 records per second. When testing performance, one server with OrientDB was able to replace 125 MySQL servers (of course, on certain requests).
Alexey ROMANENKO, the leading programmer of the Mail.Ru Search project, in his report will reveal the features of processing large amounts of data using parallel computing in Hadoop- based clusters .
The MapReduce programming model has recently become increasingly popular in the context of processing large amounts of data. It is great for tasks that can be divided into parts (by distributing them to different servers, and then combining the processing results). The report will discuss typical problems that arise when porting standard algorithms to a distributed environment, as well as methods for solving them and practical implementations using the Hadoop / MapReduce framework as an example.
Andrey PANGIN, the leading programmer of Odnoklassniki social network, is preparing a report on the experience of the social network in implementing its own solutions on the Java platform. Classmates are one of the largest public services in the world built on this technology. The report discusses the shortcomings of the existing mechanisms for network interaction, caching and serialization in Java and suggests alternative solutions based on undocumented features of the Java platform, native libraries, dynamic bytecode generation, and proprietary algorithms developed in Odnoklassniki.
Andrey is engaged in the software platform underlying the portal services, specializing in data storage and transmission systems. Prior to that, he worked at Sun Microsystems for 6 years, developing Java virtual machines, from CLDC HI to HotSpot JVM. Perhaps he is the same person who knows everything about Java. If your project works in Java or if you consider Java as a platform for a future large project, you will definitely find a lot of useful information in Alexey’s report.
Dmitry YUDIN , Leading Developer of the VoIP Lab of the Mail.Ru Group IM Division , will give a talk on the voice and video transmission problems that his team encountered while designing the VoIP engine that provides audio / video communications inside Mail.Ru Agent. Previous publications on Habré showed great interest in this topic. We hope that everyone who wishes to receive answers to their questions will be able to do this personally at the Technology Forum.
Before joining Mail.Ru Group, Dmitry Yudin worked on a similar task for about 10 years at Spirit DSP , which specialized in products for voice and video over IP channels.
Sergey SPIRIDONOV , head of the Allods Online server development team, will talk about how the Allods Online server is arranged, about the architecture and features of the development. A number of architectural decisions made during the development will be considered, their advantages and disadvantages will be analyzed.
Yaroslav RABOVOLYUK, the head of the information security department of Mail.Ru Group, at the Forum will talk about client-side vulnerabilities. Yaroslav will talk about typical attacks, countermeasures, programming and deployment practices to minimize the likelihood of vulnerabilities or to protect web applications from their influence.
At the last Forum, Yaroslav's reports were collected in full rooms (slides and video reports External control of system security , Practical use of Unix cryptographic tools ).
This year we have a whole stream devoted to frontend development.
Andrey SUMIN, development manager of frontend Mail.Ru Group, will give a report on the experience of connecting “themes” to the Mail interface. Everything related to personalizing the interface always requires good design and a balanced approach.
At the last forum, Andrei talked about how to organize the storage and use of Javascript files .
Konstantin LEBEDEV , Mail.Ru Mail programmer, will tell you how to make a multi-functional browser file downloader that allows you to create preview images, read ID3 tags and much more. This functionality is an important component of many projects, and even when using some ready-made libraries, ignorance of many details can lead to problems in security and performance.
Vladimir Krenin, deputy frontend development manager, will share her workflow automation experience. Task tracker, project management system - tools designed to facilitate teamwork. Many use them, but far from all - with pleasure. Vladimir will tell you how to organize a workflow without drowning in formalism: the principles of building a workflow, common mistakes, and implementation examples.
It will be interesting and useful, first of all, to team leaders and project managers, developers who doubt the appropriateness of implementing and using task trackers.
Roman MITASOV , a leading developer of a web messenger, will talk about the features of developing a web client for Mail.Ru Agent, about the pros and cons, "pitfalls" of client data storage, about the synchronization of the interface, about the window-to-window router, about how to keep one connection to all user windows.
In a report by Vadim MAKEEV , a web evangelist of Opera Software, we will hear about how to create sites that take into account the high resolution displays of mobile and not quite mobile devices. Measures, half measures and just tricks to make your site look perfect - in this report.
Vyacheslav EGOROV , Google’s Software Engineer, will reveal how V8, the JavaScript engine on which Chromium-based browsers and server-side Node.js are built, works Vyacheslav will also dwell in detail on how you can turn knowledge about the internal structure of the V8 to your advantage.
Ivan CHASHKIN, Mail.Ru Mail programmer, and Yegor DYDYKIN , frontend developer, will talk about Graceful degradation using the touch interface of the Mail.Ru mobile Mail as an example.
The story originates in the distant 1994, when the site content for "advanced" browsers was presented in its entirety, and development was focused on it, and for outdated, but at that time occupying a significant market share, in a simplified one. Today we have a similar situation - the army of mobile users is growing, the principles of organizing the user interface for them are increasingly at odds with the “classics”, and more and more site developers are forced to find a balance between convenience and laboriousness.
Igor ERMAKOV, director of Instant Messengers Mail.Ru Group, will devote his talk to typical problems of managing a large development team.
A year ago at the Forum Technology Igor I talked about how to develop the services, tools and web development technologies with Mail.Ru . We published the transcript of this report on Habré.
Konstantin OSIPOV , head of the Tarantool development team , will talk about algorithms for storing data on disk in Tarantool , an open NoSQL database developed by Mail.Ru Group.
Last time at the Forum, Konstantin talked more about Tarantool (slides and videos - "Tarantool: how to handle 1.5 billion requests per day? + decryption of the report on Habr , Tarantool: practice of system administration ) and about MySQL ( New features of MySQL 5.5: view of the system administrator ). Konstantin Osipov is a high-class programmer, specialist in database management systems. Mail.Ru Group is developing the high-performance data warehouse Tarantool / Box. He is one of the founders and active participants in the Moscow MySQL User Group, from 2003 to 2010 he was one of the main developers of the MySQL DBMS and participated in the creation of many features 4.1, 5.0, 5.1 and 5.5. Prior to Mail.Ru Group, Konstantin worked at Oracle, Sun, MySQL, RingRows, SpyLOG, InterPro ...
The story about the Forum 2011 is here, about spring, for system administrators - here . It will be even better and more interesting. Come, we are waiting for you! Participation is free, but the number of seats is limited, registration is required and will close soon.
Aliyev Rauf,
Director of Research and Education
Mail.Ru Group
We are holding the Technology Forum for the fourth time. In the spring we made a special version of the Forum for system administrators. This fall, as well as last year , the topic will be web development, in the broadest sense.
I would like to introduce you to topics and speakers, technologies and ideas. We invited several interesting western speakers to the Forum. For the first time we will tell a lot about what, live demonstrations and master classes are waiting for you. In this post, I would like to introduce you to the topics that will be raised this time, and with our speakers. Ask questions in the comments - speakers will still have time to take them into account in their presentations. And prepare questions for the Forum, we love and appreciate them very much. Participation is free, but the number of places is limited,registration is required and will close soon.
Maxim BABICH , head of the project development department for Money Mail.Ru, will talk about how Mail.Ru Money is arranged inside, how to make such a system scalable and fault-tolerant, which atypical problems for other highly loaded services will have to be solved. This knowledge is especially useful for those who develop projects related to electronic commerce (including payment aggregators, billing systems, etc.).
Money Mail.Ru - A payment system that currently processes hundreds of thousands of transactions per day. Earlier, almost no information was published about the Mail.Ru Money device. Over the past year, this service has grown and grown very seriously. If you are interested in any specific aspects of building a payment system, please unsubscribe in the comments - we will try to touch on these topics in the report.
Denis ANIKIN, Technical Director of Mail.Ru Mail, will give a presentation on the experience of launching HTTPS in Mail.Ru Mail. The well-known HTTP traffic encryption technology, used in browsers and servers for more than 15 years, has begun to be fully used relatively recently on loaded global projects and services. In these matters, there are a lot of subtleties associated with configuring SSL / nginx, adapting and rolling server software. Denis will be happy to share all this information in his report. Denis knows a lot of technological subtleties and is responsible for the fourth in the world in attendance mail service.
At the last Technology Forum, Denis talked about how high loads are maintained in Mail.Ru Mail .
Alexey MASHANOV, the leading programmer of the social network My World, will tell you how to centrally collect logs from a cluster in almost real time. This problem is even relevant for a small group of servers working on one task, what to do when there are hundreds or thousands of these servers? There are a number of ready-made products for this, and Alexey will tell you why the Scribe log server was chosen from among them. Scribe is a real-time log aggregator developed by Facebook and submitted to Open Source in 2008.
Alexey has been working at Mail.Ru Group since 2010, he is engaged in optimization, statistics, and improvement of the technical processes of the My World social network.
Boris WOLFSON - CTO HeadHunter. His report at the Technology Forum will be devoted to the balance between the quality and efficiency of the release of new "features." The need to maintain this balance is facing every development manager. Boris in his report will share the system approach adopted by Headhunter. Ward Cunningham coined the term “technical debt” for this situation ( wiki , habrahabr) According to this metaphor, the quick and inaccurate addition of functionality leads us to technical debt, which the inventor of wiki technology compares with financial debt. Many teams, increasing technical debt, fall into the “debt hole” and, as a result, slow down the speed of their work. Boris will tell you why and in what situations a technical debt appears and how it can be purposefully reduced. This report will be especially interesting to technical managers.
Pavlo BARON , leading architect of the German consulting company Codecentric, will talk about how to properly build distributed systems to work with large amounts of data. Pavlo often speaks at IT conferences around the world, is the author of three books: Erlang / OTP, Pragmatic IT Architecture and Fragile Agile. The report itself will be associated with Big Data, but on the sidelines Pavlo will be happy to share his knowledge and experience in other matters, where he is a recognized expert - Erlang / OTP, Agile, Java. Pavlo will speak in English, we will provide simultaneous translation. In the process, it turned out that our speaker had long-standing Ukrainian roots, and he was slightly familiar with the Russian language. So it is possible that the audience and Pavlo will begin to understand each other much faster than is usually the case with the participation of the translator.
Alexey VOROPAEV, the head of the ranking department of the project Search Mail.Ru, will reveal the topic of machine learning: what it is, how it works, and whether it can be applied in your projects. Alexey will talk about popular machine learning algorithms with and without a teacher, and also show examples of their application (including in the Mail.Ru Search project). Before becoming the head of the ranking department, Alex was engaged in the development of a search server and indexer. Prior to Mail.Ru Group, Alexey worked on a distributed enterprise-level traffic analyzer at Perimetrix .
The report will be especially useful to programmers and analysts dealing with large volumes of data.
Sergie MANCILLA , lead developer of Ajax.org, will give a presentation on "Architecting Node.js JavaScript for Large-Scale Projects". Sergi is a recognized JavaScript expert and leads the development team for the Cloud9 IDE in Amsterdam , one of Node.js.'s most famous online development environments.
Prior to that, Sergi worked as a senior developer at TomTom , where he led the team building the JavaScript framework for user interfaces.
Victor STARODUB , Mail.Ru Mail programmer, will talk about his experience in implementing the IMAP protocol in the mail server. IMAP replaces POP3twenty years ago, but, despite this, it is a "tough nut" for large mail services. First of all, because the number of email clients, multiplied by the complexity of the standard, makes it necessary to take into account many exceptions in the server logic. Secondly, the IMAP principle itself implies a permanent connection between the client and server, unlike session POP3. In the case of millions of connections per hour, this feature turns into a problem. Victor will tell about solutions to this and other problems in his report.
Luca GARULLI , CEO of NuvolaBase Ltd , will talk about how to create a fail-safe replicated cluster database that can be easily scaled only by adding new nodes to the cluster. Luc - author of OrientDB - graph, document-oriented database of the NoSQL class, 100% implemented in Java. It supports an object-oriented database interface that runs on top of a document-oriented layer. Implemented a powerful SQL-based query language, cluster architecture, master-master replication, full support for ACID transactions, natively supports HTTP, RESTful and JSON protocols without using third-party components, supports server-side scripting. According to a number of tests on conventional equipment, OrientDB allows you to save 150,000 records per second. When testing performance, one server with OrientDB was able to replace 125 MySQL servers (of course, on certain requests).
Alexey ROMANENKO, the leading programmer of the Mail.Ru Search project, in his report will reveal the features of processing large amounts of data using parallel computing in Hadoop- based clusters . The MapReduce programming model has recently become increasingly popular in the context of processing large amounts of data. It is great for tasks that can be divided into parts (by distributing them to different servers, and then combining the processing results). The report will discuss typical problems that arise when porting standard algorithms to a distributed environment, as well as methods for solving them and practical implementations using the Hadoop / MapReduce framework as an example.
Andrey PANGIN, the leading programmer of Odnoklassniki social network, is preparing a report on the experience of the social network in implementing its own solutions on the Java platform. Classmates are one of the largest public services in the world built on this technology. The report discusses the shortcomings of the existing mechanisms for network interaction, caching and serialization in Java and suggests alternative solutions based on undocumented features of the Java platform, native libraries, dynamic bytecode generation, and proprietary algorithms developed in Odnoklassniki.Andrey is engaged in the software platform underlying the portal services, specializing in data storage and transmission systems. Prior to that, he worked at Sun Microsystems for 6 years, developing Java virtual machines, from CLDC HI to HotSpot JVM. Perhaps he is the same person who knows everything about Java. If your project works in Java or if you consider Java as a platform for a future large project, you will definitely find a lot of useful information in Alexey’s report.
Dmitry YUDIN , Leading Developer of the VoIP Lab Before joining Mail.Ru Group, Dmitry Yudin worked on a similar task for about 10 years at Spirit DSP , which specialized in products for voice and video over IP channels.
Sergey SPIRIDONOV , head of the Allods Online server development team, will talk about how the Allods Online server is arranged, about the architecture and features of the development. A number of architectural decisions made during the development will be considered, their advantages and disadvantages will be analyzed. Yaroslav RABOVOLYUK, the head of the information security department of Mail.Ru Group, at the Forum will talk about client-side vulnerabilities. Yaroslav will talk about typical attacks, countermeasures, programming and deployment practices to minimize the likelihood of vulnerabilities or to protect web applications from their influence. At the last Forum, Yaroslav's reports were collected in full rooms (slides and video reports External control of system security , Practical use of Unix cryptographic tools ).
This year we have a whole stream devoted to frontend development.
Andrey SUMIN, development manager of frontend Mail.Ru Group, will give a report on the experience of connecting “themes” to the Mail interface. Everything related to personalizing the interface always requires good design and a balanced approach. At the last forum, Andrei talked about how to organize the storage and use of Javascript files .
Konstantin LEBEDEV , Mail.Ru Mail programmer, will tell you how to make a multi-functional browser file downloader that allows you to create preview images, read ID3 tags and much more. This functionality is an important component of many projects, and even when using some ready-made libraries, ignorance of many details can lead to problems in security and performance. Vladimir Krenin, deputy frontend development manager, will share her workflow automation experience. Task tracker, project management system - tools designed to facilitate teamwork. Many use them, but far from all - with pleasure. Vladimir will tell you how to organize a workflow without drowning in formalism: the principles of building a workflow, common mistakes, and implementation examples. It will be interesting and useful, first of all, to team leaders and project managers, developers who doubt the appropriateness of implementing and using task trackers.
Roman MITASOV , a leading developer of a web messenger, will talk about the features of developing a web client for Mail.Ru Agent, about the pros and cons, "pitfalls" of client data storage, about the synchronization of the interface, about the window-to-window router, about how to keep one connection to all user windows. In a report by Vadim MAKEEV , a web evangelist of Opera Software, we will hear about how to create sites that take into account the high resolution displays of mobile and not quite mobile devices. Measures, half measures and just tricks to make your site look perfect - in this report. Vyacheslav EGOROV , Google’s Software Engineer, will reveal how V8, the JavaScript engine on which Chromium-based browsers and server-side Node.js are built, works Vyacheslav will also dwell in detail on how you can turn knowledge about the internal structure of the V8 to your advantage. Ivan CHASHKIN, Mail.Ru Mail programmer, and Yegor DYDYKIN , frontend developer, will talk about Graceful degradation using the touch interface of the Mail.Ru mobile Mail as an example. The story originates in the distant 1994, when the site content for "advanced" browsers was presented in its entirety, and development was focused on it, and for outdated, but at that time occupying a significant market share, in a simplified one. Today we have a similar situation - the army of mobile users is growing, the principles of organizing the user interface for them are increasingly at odds with the “classics”, and more and more site developers are forced to find a balance between convenience and laboriousness.
Igor ERMAKOV, director of Instant Messengers Mail.Ru Group, will devote his talk to typical problems of managing a large development team. A year ago at the Forum Technology Igor I talked about how to develop the services, tools and web development technologies with Mail.Ru . We published the transcript of this report on Habré.
Konstantin OSIPOV , head of the Tarantool development team , will talk about algorithms for storing data on disk in Tarantool , an open NoSQL database developed by Mail.Ru Group. Last time at the Forum, Konstantin talked more about Tarantool (slides and videos - "Tarantool: how to handle 1.5 billion requests per day? + decryption of the report on Habr , Tarantool: practice of system administration ) and about MySQL ( New features of MySQL 5.5: view of the system administrator ). Konstantin Osipov is a high-class programmer, specialist in database management systems. Mail.Ru Group is developing the high-performance data warehouse Tarantool / Box. He is one of the founders and active participants in the Moscow MySQL User Group, from 2003 to 2010 he was one of the main developers of the MySQL DBMS and participated in the creation of many features 4.1, 5.0, 5.1 and 5.5. Prior to Mail.Ru Group, Konstantin worked at Oracle, Sun, MySQL, RingRows, SpyLOG, InterPro ...
The story about the Forum 2011 is here, about spring, for system administrators - here . It will be even better and more interesting. Come, we are waiting for you! Participation is free, but the number of seats is limited, registration is required and will close soon.
Aliyev Rauf,
Director of Research and Education
Mail.Ru Group