Friendship, thanks to which Google has grown to enormous size

Original author: James Somers
  • Transfer
By programming together with one computer, Jeff Dean and Sanjay Gemawat changed the company's course — and the entire Internet. In the illustration: the best Google programmers sometimes seem to be two hemispheres of one brain. Picture by David Planckert

One day in March 2000, six top Google engineers gathered in a makeshift conference room. There was a state of emergency: from October 1999, the crawlers stopped. Although users are still given search results, but they are out of date for five months. More was at stake than engineers had imagined. At the moment, Larry Page and Sergey Brin were negotiating the supply of Google search on the largest Internet portal Yahoo and promised to increase the search index tenfold to keep up with the World Wide Web, which doubled in size in the previous year. If the crawlers are not repaired, google.com gets stuck in the past, the deal with Yahoo can fail, and the company risks burning up the investment and sink into oblivion.

In the conference room at the stairs, engineers put doors on the box, built improvised tables and installed computers. A sad Craig Silverstein sat near the far wall: a 27-year-old thin young man with a thin voice. Craig was the first employee of Google: he joined the company when its main office was located in the living room of Bryn, and he personally rewrote most of the founders code. After four days and nights of work, they did not achieve anything with a Romanian system engineer named Bogdan Kokosel: “None of the tests performed had any meaning,” Silverstein recalled. “Nothing worked, and we didn’t know why.”

Silverstein barely noticed near Sanjay Gemavat, a modest 33-year-old brunet with thick eyebrows and graying temples, a MIT graduate. Sanjay was hired just a few months ago, in December. He came after his colleague, the slim and energetic thirty-year-old Jeff Dean of Digital Equipment Corporation. Jeff left DEC ten months before Sanjay. They were extremely friendly and preferred to write code together. In the “war room,” Jeff pushed a chair toward Sanjay's desk, leaving his empty one. Sanjay pounded on the keyboard, and Jeff sat back, straightening and dictating, like a producer in a news announcer's earpiece.

Jeff and Sanjay began to study the stalled index. They found that some words were missing: they entered the query [mailbox] and did not receive any results, while others are not listed in order. For several days they were looking for a mistake, plunging into the logic of the code. Section by section, checked everything. No bugs.

Programmers sometimes present their program as a layer structure that descends from the user interface down to more fundamental layers. At the bottom of this structure, software is found with iron: such a descent from platonic code abstractions to electricity and silicon physics. On the fifth day of work, Jeff and Sanjay began to suspect that the problem was not the logic of the code, but a physical cause. They converted the index file into binary code. They wanted to see what cars see.

A thick column with zeros and ones appeared on Sanjay’s monitor, each row representing an indexed word. Sanjay remarked: 0 is in place 0. When Jeff and Sanjay gathered all the distorted words, they saw a pattern: the same failure in each word. Damage to memory chips on servers.

Sanjay looked at Jeff. For several months, Google experienced more and more hardware failures. The problem was that as Google grew, the computing infrastructure also expanded. Computer equipment rarely fails, unless you have a lot of computers - then they break down all the time. The wires are worn out, the hard drives are falling apart, the motherboards are overheating. Many cars fail at once; others start to work slower. Strange natural factors come into play. When a supernova explodes in space, the blast wave creates high-energy particles that are scattered in all directions: scientists believe that there is a non-zero chance that one of the particles of cosmic radiation could get into a computer chip on Earth and change 0 to 1. The most reliable computer systems in the world at NASA financial firms and the like, use special equipment that protects from single-bit "flips". But Google still worked as a startup and bought cheap computers that did not have such protection. The company has reached the point of inflection. The compute cluster has grown to such a size that even unlikely hardware failures have become inevitable.

Together, Jeff and Sanjay wrote a code to compensate for the failure on damaged systems. Soon built a new index - and the emergency was successfully resolved. Silverstein was confused. He ingeniously debugged the code, but here the answer was hidden at the hardware level. Jeff and Sanjay went deeper.

Prior to the March fiasco, Google worked on the code that the company's founders wrote in Stanford graduate school. Page and Brin were not professional programmers. They were scientists who conducted an experiment in the field of search technology. When the crawler fell, he did not display any informative diagnostic message, except for the phrase "Damn, horse!"

The system BigFiles, written by Page and Brin, the first employees of the company jokingly called BugFiles. An extremely important indexing procedure took several days, and if I encountered a problem, I started from the very beginning. It was a completely non-scalable system.

We say that we are “searching the Internet”, but in reality this is not the case: we work with the search index on the search engine servers. When Google was still called BackRub in 1996, the whole index was placed on computers installed in the page of the Page hostel. In March 2000, not a single supercomputer could handle it. The only solution left for Google is to buy ordinary computers and add them to the giant cluster. Because half the cost of conventional computers is what Google considers "garbage" - disk drives, metal cases - the company ordered only bare motherboards and hard drives - and connected them. By that time, Google had already connected 1,500 such self-made servers to two-meter racks in a building in Santa Clara, California. Due to hardware failures, only 1,200 worked at any one time. Failures occurred unpredictably and continued to disrupt the system. To survive, it was necessary for Google to combine computers into a single viable organism, resistant to disruptions.

A couple of Jeff and Sanjay took over the job. Wayne Rosing, who designed Apple's predecessor of the Macintosh, came to Google in November 2000, gathering a team of one hundred engineers. Working ninety hours a week, they wrote code so that no HDD crash would disrupt the system. They added breakpoints to the Web traversal process so that the cradle restarts from the moment it crashes, not from the very beginning. Having developed new coding and compression schemes, they actually doubled the capabilities of the system. They were inexorable optimizers. For example, Google moved the most frequently used data to the outer side of the HDD plates, where the higher speed of rotation of the disk and, consequently, higher speed of reading and writing information, and the inner areas of the plates left empty. Jeff and Sanjay used this space to store pre-processed data for general search queries. For four days in 2001, they proved that the Google index can be stored in fast RAM instead of slow HDDs. The discovery completely changed the economy of the company. Page and Bryn knew that users were flocking to a service that instantly gave answers. The problem was that speed required computational power, and computational power cost money. Jeff and Sanjay managed to squeeze through the eye of a needle with the help of software optimizations. that users flock to the service, which instantly gives answers. The problem was that speed required computational power, and computational power cost money. Jeff and Sanjay managed to squeeze through the eye of a needle with the help of software optimizations. that users flock to the service, which instantly gives answers. The problem was that speed required computational power, and computational power cost money. Jeff and Sanjay managed to squeeze through the eye of a needle with the help of software optimizations.

Alan Eustace led the engineering team after Rosing retired in 2005. “Paradoxically, but to solve large-scale problems, it is necessary to know the smallest details,” said Eustace. Jeff and Sanjay understood the system to a bit level. One day, Jeff distributed a list of “Delay rates that every programmer should know.”. In fact, this is a list of numbers that almost no programmer thinks about, but these numbers are literally built into the brain of Jeff and Sanjay: for example, a call from the L1 cache takes 0.5 ns or that sequential reading of one megabyte from memory takes 250 µs. After a series of key software optimizations, the power of Google’s systems has been scaled to orders of magnitude. Meanwhile, in the extensive data processing centers, appliances began to take winding paths, following program-generated instructions for replacing hard drives, power supplies, and memory modules. Even when parts were wearing out and breaking down, the system flourished.

Today, Google’s engineers have a unified classification system that starts with level 1, this is tech support. Level 2 - college graduates. Level 3 employees often have a master’s degree. Getting level 4 takes several years or requires a Ph.D. The career of most employees stops at level 5. Engineers of level 6 are the top 10% of the staff, they are so talented that the success of the projects depends on them. Seventh level give employees a level 6 with a long track record. Level 8 chief engineers are responsible for specific products or parts of the infrastructure. About outstanding engineers of the 9th level speak with respect. The title of Google Fellow (level 10) is awarded for life usually to the world's leading experts in their fields. Jeff and Sanjay are the Senior Google Fellows, the first and only level 11 employees.

The Google Campus, located just off the highway a few minutes from the center of Mountain View, is a series of squat, unattractive buildings with tinted windows. One Monday morning in the summer of 2017, after a morning session of pair programming, Jeff and Sanjay went to lunch at the campus cafeteria called the Big Table, named after the system they helped develop in 2005 to unite countless computers into a virtually single device. Tall and thin Sanjay in burgundy-colored ancient Henley shirt, gray trousers and small glasses on a wire rim peered out a table on the terrace and quickly took it, opening an umbrella and crouching in the shade. Near the sun, he set another chair for Jeff, who arrived a minute later, a broad-shouldered sports man in a short-sleeved shirt and stylish sneakers.

Like lovers, Jeff and Sanjay tell stories together, complementing pieces of the overall picture. They began to recall their first projects.

“We wrote everything manually,” says Sanjay. His glasses darkened in the sun. - It was necessary to rewrite the code and then suddenly: “Oh, this is similar to what we wrote last month.”

“Or a slightly different pass in our indexing data,” Jeff added.

“Or a little different,” says Sanjay. “And so we found out ...

” “This is the essence,” said Jeff.

“... what is a general pattern,” Sanjay concludes.

Jeff bit off a slice of pizza. He has a sailor's fingers, knotty and hard. Sanjay looks very gentle in comparison to him. He wonders how they started practicing pair programming: “I don’t even know how we decided that this would be better.”

“We worked like this before Google,” said Jeff.

“But I don’t know why we decided that it’s better to have one computer and not two,” says Sanjay.

“My DEC lab was two blocks from his lab,” says Jeff. - And between them is an ice cream shop.

- So this is an ice cream shop! - Sanjay is delighted.

Bachelor Sanjay spends holidays with Jeff, his two daughters and wife Heidi. Jeff's daughters call him Uncle Sanjay, and the five of them often have dinner on Fridays. Sanjay and Victoria, Jeff's eldest daughter, took up baking. “I saw them grow,” says Sanjay proudly. After Google’s IPO in 2004, they moved to new homes that are six kilometers apart. Sanjay lives in a modest three-bedroom house in Old Mountain View, and Jeff himself designed his home near the center of Palo Alto, setting up a trampoline in the basement. While working on the house, he found that he likes to design the space, but lacks the patience for details that he calls the “Sanjay-oriented aspects” of the architecture: the details of the beams, the fixtures, the calculation of the loads that prevent the great project from falling apart.

“I don’t know why others don’t work that way,” says Sanjay about pair programming.

“We need to find a partner who is compatible with you in a way of thinking so that you complement each other,” said Jeff.

They got up from the table and went to look for soft ice cream, walking along the Big Table among scurrying Googlers. As they walked, Jeff outlined his strategy for soft ice cream: “I twist it. I think this approach adds stability, ”he said. Satisfied and goal-oriented Sanjay spun the chocolate-vanilla mixture into his glass.

In the book “Circles of cooperation: the dynamics of friendship and creativity”In 2001, sociologist Michael P. Farrell studied close creative groups: the French Impressionists, Sigmund Freud and his contemporaries. “Most fragile ideas that initiated a new vision arise not when the whole group is together, and not when everyone works alone, but when they collaborated and interacted in pairs,” he wrote. Monet and Renoir worked side by side in the summer of 1869. They created a style that became impressionism; six years of working together gave birth to cubism, Pablo Picasso and Georges Braque often signed only the reverse side of the canvases to hide from each other the fact of the completion of the work (“The work was not completed until we both felt this was true,” Picasso recalled later). In the book “Double Power: Search for the Essence of Innovation in Creative Couples”writer Joshua Wolf Schenk quotes a 1971 interview in which John Lennon said that either he or Paul McCartney “wrote a couple of light notes like“ I read the news today ”or the like. “One of us got stuck until the other one came,” Lennon said, “I will sing half, and he has the inspiration to write another part, and vice versa.” Anyone can get into a creative dead end, but hardly two people at the same time.

At the stage of “theoretical construction” of a new science or art, it is important to expand the horizons without getting stuck in a dead end. Francois Jacob, who, together with Jacques Monodom, made a breakthrough in the study of the regulatory function of genes, noted that by the middle of the 20th century, most molecular biology research was carried out in pairs: “Together it is easier than one to invent theories and build models,” Jacob wrote. - When two minds work on a problem, ideas appear more often and faster. They rebound from partner to partner. They twist together like branches on a tree. And false illusions are nipped in the bud. ” Over the past 35 years, about half of the Nobel Prizes in the field of physiology and medicine have been awarded to pairs of scientists.

After years of working together, partners sometimes develop their own language, as the twins do. They imitate each other in habits and clothes, they have a general sense of humor. It becomes impossible to evaluate the specific contribution of each to working together. But such close partnerships are rarely found in software development. Although developers sometimes talk about “pair programming”, when two programmers share one computer, one “behind the wheel” and the other “navigator”, they usually represent only working relationships, like pilots in an airplane. Jeff and Sanjay, on the contrary, sometimes seem to be two hemispheres of one brain. In their most famous scientific works listed up to a dozen co-authors. However, Bill Cochran, one of their managers, recalls: "They were so prolific and so effective in a pair

In 1966, researchers at System Development Corporation found that the best programmers were more than ten times more efficient than the worst. Since then, the debate continues about the existence of the so-called "10-fold programmer." This idea emphasizes individuality, while large software projects create a team. In programming, little is achieved in isolation. Ironically, many programmers view the results of Jeff-Sanjay’s collaborative work as proof that a 10-fold programmer exists.

Jeff was born in Hawaii in July 1968. His father, Andy, was a researcher of tropical diseases; Virginia Lee's mother is a medical anthropologist who knew half a dozen languages. For fun, father and son assembled a computer from the IMSAI 8080 kit. They soldered the details, studying each part.

Jeff and his parents often moved. At thirteen, he missed the last three months of the eighth grade to help them in a refugee camp in western Somalia. In high school, he began writing a data collection program for epidemiologists called Epi Info; it has become a standard tool for researchers, translated into dozens of languages, and has a circulation of hundreds of thousands of copies (on the website of the Center for Disease Control and Prevention published a photograph of Jeff from his graduation ceremony). Heidi, whom Jeff met at the University of Minnesota college, only learned about the significance of this program years later. “He didn’t boast that much,” she says. “Everything had to be pulled out of it.” Their first date took place at the women's basketball game, where Jeff entertained the audience in a gopher costume.

Jeff's doctoral thesis is dedicated to compilers. “Compilers themselves are pretty boring,” says Alan Eustace, but on the other hand, “this is a very low level, close to hardware.” Describing Jeff, Sanjay rotates a finger at his temple: “When writing code, a certain model is formed in his head. What will be the performance of this code? It almost automatically calculates all border situations. ”

Sanjay first sat down at a computer at the age of seventeen, when he went to Cornell University. He was born in West Lafayette, Indiana, in 1966, but grew up in Kota, an industrial city in northern India. His father Mahipal was a botany professor, and his mother Shanta cared for Sanjay and his two older brothers and sisters. The family loved books: his uncle Ashok Mehta remembers buying “The Day of the Jackal” for childrenFrederick Forsythe with a torn binding, and the kids all together read a battered book, passing each other pages when finished. Sanjay's brother, Pankaj, became the youngest lecturer to ever get a position at the Harvard Business School (he is now a professor at New York University). Pankaj studied at the same school as Sanjay and was known as a Renaissance man: “I sort of lived in the shadow of my brother,” says Sanjay. As an adult, he retained this modesty. In 2016, when he was introduced to the American Academy of Arts and Sciences, he did not even tell his parents; they learned the news from a neighbor.

In the MIT graduate school, Sanjay made friends. However, he never met girls there, and now he does it “very, very rarely.” He says he did not refuse to start a family, it just happened. His close friends learned not to disturb him about this, and his parents had long understood that his son would remain a bachelor. Perhaps because it is so closed, a certain aura of mystery has formed around Google in Sangjay. He is considered a quiet but thorough engineer — one who thinks deeply and with extraordinary clarity. On the table he has a neat stack of Mead notebooks for almost twenty years, filled with neat lists and diagrams. He still keeps records by hand: he says it helps to think. At MIT, Barbara Liskov, an influential computer scientist, was his research advisor, which studied including the management of complex code bases. In her opinion, the best code is like good literature. It should have a carefully thought out structure, and every word should work. Such programming requires empathy with readers. And the code is considered not only as a means of achieving the goal, but also as a valuable artifact in itself. “I think he designs the systems best,” says Craig Silverstein. “Just look at its code: it is beautiful, like a work of art, like a sculpture with perfect proportions.” “I think he designs the systems best,” says Craig Silverstein. “Just look at its code: it is beautiful, like a work of art, like a sculpture with perfect proportions.” “I think he designs the systems best,” says Craig Silverstein. “Just look at its code: it is beautiful, like a work of art, like a sculpture with perfect proportions.”

In Google, Jeff is much more famous. About him create memes as about Chuck Norris. ("Chuck Norris counted to infinity ... twice"; "Jeff Dean's summary lists things he did not do: so shorter").

“When Jeff Dean develops a program, he first creates a binary, and then writes the source code as documentation.”

“Jeff Dean once failed the Turing test because he correctly set the 203rd Fibonacci number in less than a second.”

“One day in 2002, when the search back-end turned off, Jeff Dean manually answered user questions for two hours. During this period, the quality of search results has increased significantly. ”


From the article “ Jeff Dean from Google is Chuck Norris of our time ” - approx. per.

But for those who know both of them, Sanjay is equal to him in talent. “Jeff is great at inventing wild new ideas and prototypes,” said Wilson Sie, their longtime colleague. “Sanjay is building things conscientiously.” In life, Jeff is more sociable, Sanjay is more withdrawn. The code is the opposite. Jeff's programming is dazzling: he can quickly put forth startling ideas, but, as he does so quickly, with inspiration, he can leave readers behind. Sanjay code is more understandable.

“Some people write code that is too sparse,” explains Silverstein. There is very little information on one screen. You have to scroll it back and forth to understand. ” Others write too tight code: “You look at it and think:„ Wow. I don't want to figure it out. ” Sanjay somehow manages to find a balance. You look at its code and think: “Okay, I can understand it,” and still there is a lot of information on one page. ” Silverstein continues: “Whenever I want to add new functionality to the code of Sanjay, he seemed to have foreseen this in advance. I feel like Salieri. I see greatness, but I don’t understand how this is possible. ”

On Monday morning this spring, Jeff and Sanjay were standing in the kitchenette of building 40, which houses most of the Google AI division. Behind them is a board filled with matrix algebra formulas, on the table is an article about adversarial networks. Jeff in a faded T-shirt and jeans looks like a beach bum, which has become the way of correction. Sanjay in a sweater and gray trousers. Behind the bright windows are tall pines, behind them is a field. Everywhere, where Jeff works in Google, be sure to put espresso machines. La Marzocco is about one meter wide on the kitchen table. “We're late,” remarks Sanjay over the coffee grinder. On the clock 8:32.

After the cappuccino, they go to the computers. Jeff rolls a chair from his dirty table to Sanjay's immaculately clean table. He puts his foot on the bedside table, leaning back, while Sanjay studies the screen. Four windows are open: on the left - the browser and the terminal for launching analysis tools; on the right, two documents in the Emacs text editor: in one to-do list in a notebook, the other is filled with a colorful code. One of Sanjay's notebooks is next to the computer.

- OK, what did we do? - asks Sanjay.

“I think we analyzed the size of the TensorFlow Lite code,” says Jeff.

This is a new large machine learning project: Jeff and Sanjay are worried about bloated codebase. Like literary editors, they are looking for ways to reduce volume. To do this, they created a new tool, which itself needs to be optimized.

“I tried to find out how slow it is,” said Sanjay.

“Pretty slow,” Jeff leaned forward, still relaxed.

“This one was one hundred and twenty kilobytes,” says Sanjay, “and it goes about eight seconds.”

- One hundred and twenty thousand stack calls, not kilobytes.

- Well, kilobytes of text, well ... about as much.

- Oh, yes, I'm sorry.

“I don’t know what threshold to choose for a unit size,” Sanjay asks. - half a mega?

“It seems normal,” Jeff agrees. Sanjay begins to knock on the keyboard, and Jeff clings to the screen. “So, you write that if it is more than this, we will simply choose ...” he fell silent; Sanjay answered the code question.

When Sanjay drives the car, he puts his hands on ten and two and carefully looks ahead. The same discipline behind the keyboard. His legs were shoulder-width apart, his back was straight, as if he was working on his posture. Thin fingers gently run around the keys. The first programmers began to arrive at the office.

Soon they reached a small milestone, and Sanjay scored a team to run the test. He looked exhausted and checked the mail. The test has ended. He did not notice.

- Hey! - Jeff snapped his fingers and pointed to the screen. While talking to him gives jokes and puns, at the computer with Sanjay, he can become self-confident, rude and disapproving. Sanjay takes this for granted. When it seems to him that Jeff is moving too fast, he raises his hands from the keyboard and spreads his fingers, as if to say: “Stop”. This is the closest resemblance of a dispute: for twenty years they can not remember when they raised their voices.

Sanjay scrolled through the page, a new section of code appeared on the screen.

- From this you can make a routine procedure, is not it? Asked Jeff.



Mmmm ... Jeff snapped his knuckles:

“ It seems possible. ” Will do?

Sanjay is disarmed:

- No, I ...

- So, ignore the problem? - Jeff asks indignantly.

- No, I mean, we are just trying to understand what is happening. To make notes about this, right?

“Okay,” Jeff obviously was in a good mood. Together they dictated the note.

Approaching lunch. They worked for two hours with one ten-minute break, talking most of the time (the junior who watched them would be most impressed by the fact that they never stopped or stuck). Standard engineering practice is to give the revision code, but Jeff and Sanjay skip this step, putting a lgtm mark (looks good to me). In a sense, they are busy with little things. However, their code is executed on a Google scale. The kilobytes and microseconds they worry about are multiplied billions of times in data centers around the world — huge, noisy data centers where endless racks of servers are cooled with water tanks. On such days, Jeff, as you know, comes home and tells his daughters: "Today, Sanjay and I accelerated Google search by ten percent."

In 2003, four months later, Jeff and Sanjay made perhaps the biggest update in Google history. They did this with the MapReduce software. The idea came when they rewrote the Google crawler and indexer for the third time. They realized that every time they solve an important problem: coordination of work in a huge number of geographically distributed, separately unreliable computers. If you summarize the solution, the problem will be solved forever. At the same time, a tool will appear that any programmer on Google can use to manage machines in data centers, as if all of them are a single computer the size of a planet.

Jeff and Sanjay wrote MapReduce in a corner office overlooking the pond with ducks. This tool forever streamlined a breathtakingly complex process. Previously, each programmer had to independently figure out how to share and distribute data, assign jobs and take into account hardware failures. MapReduce provides a structured way to solve these problems. As the chef prepares the ingredients before starting the culinary work, MapReduce divides the tasks into two stages. First, the programmer gives each machine a "map" of the task (for example, count the number of references to a word on a page); then he writes instructions for "reducing" results from all machines (for example, by summing up). MapReduce handles distribution details and thus hides them.

The following year, Jeff and Sanjay rewrote Google's crawling and indexing system in terms of MapReduce tasks. Soon, other engineers realized how powerful this system was and started using MapReduce to process video and render tiles on Google Maps. MapReduce was so simple that new tasks appeared by themselves. Google has a daily load cycle: during the day, there is more traffic than at night, and MapReduce tasks began to absorb idle capacity. In a dream, the human brain processes the daily experience. Now Google has started processing its data at night.

There has long been evidence that Google is actually an AI company, which only pretends to be a search engine. In 2001, Noam Shazir, who shared the office with Jeff and Sanjay, decided to remake the spell checker system that Google licensed from another company. This system continued to make awkward mistakes. For example, the word TurboTax, she proposed to replace the turbot ax (flounder living in the North Atlantic).

The quality of the spell checker depends on the dictionary. Shazir realized that on the Internet, Google had access to the largest dictionary that ever existed in the history of mankind. He wrote a program that used the statistical properties of a text on the Internet to determine which words were most likely typed in error. The program understood that “pritany spears” and “brinsley spears” meant “Britney Spears”. When Shazir showed the program at the weekly meeting, many employees repeatedly tried to deceive her, but mostly without success. Shazir collaborated with Geoff Deane and an engineer named Georges Hariq to develop a similar advertising targeting system for web content. This targeting brought a whole avalanche of money that the company sent to its computing infrastructure. This was the beginning of the feedback loop: infrastructure growth improves Google’s computing power and intelligence; computational abilities become a source of profit, and profit allows you to build infrastructure. Such a feedback loop brought the company extreme and unprecedented market dominance.

Resourceful programmers used MapReduce to get different insights from Google data: this is how it became possible to transcribe users' voice messages, answer their questions, auto-complete queries and translate texts into more than one hundred languages. These systems were developed using relatively simple machine learning algorithms. However, “when you have a lot of data, very simple methods work incredibly well,” said Jeff. Because “data, data, data” - stored and processed with BigTable, MapReduce and their successors - is the company's main asset, Google’s global infrastructure has become more flexible and scalable. The idea of ​​distributed computing appeared long ago. The concepts of "cloud computing" and "big data" existed before the advent of Google. But by making the infrastructure intelligently managed for distributed programs by ordinary programmers, Jeff and Sanjay took Google to a new level. Users have felt that something has changed - the Google cloud is getting smarter.

In 2004, Jeff and Sanjay thought that the system would be useful for astronomers, geneticists and other scientists with a lot of data to process. Then they published the article "MapReduce: Simplified Data Processing in Large Clusters . " The document appeared as God of the car. Cheap equipment, the growth of web services and connected devices led to the flow of data, but few companies had software for processing such arrays of information. The two engineers who struggled to scale a small search engine called Nutch — Mike Cafarella and Doug Cutting — were so convinced of the importance of MapReduce that they decided to create a free clone of this system from scratch. In the end, they called their project Hadoop, after the name of the toy elephant, whom Kutting's son loved.

Gradually, Hadoop introduced half of the Fortune 50 companies. It became synonymous with big data. Facebook used Hadoop MapReduce to store and process user metadata - information about clicks, likes, ad views. At some point, Facebook had the largest cluster of Hadoop in the world. Hadoop MapReduce contributed to the success of LinkedIn and Netflix.

Former NSA technology director Randy Garrett remembers, as demonstrated technology to the director of the NSA, General Keith Alexander. Hadoop performed the analysis task 18,000 times faster than the previous system. This became the basis for a new approach to intelligence gathering, which some observers call the "total collection of everything."

Jeff has a restless nature: for him, the problem loses interest when he sees a solution. In 2011, cloud computing swept the world, and Jeff Dean began working with Andrew Eun, a computer science professor at Stanford, who led Google's secret project on neural network research. Jeff encountered neural networks back in his student years, but then they could not solve real problems. Eun told Jeff that the situation has changed. At Stanford, researchers have achieved some impressive results by giving the neural network access to large amounts of data. Eun suggested that on a large scale neural networks can be not only useful, but also powerful.

Neural networks are very different from traditional computer programs. Their behavior is not determined by normal commands; instead, the network is “trained” using input data and feedback. Jeff's knowledge of neural networks has not advanced since his student years, and Heidi watched their bathroom fill with textbooks. Jeff started paying about a day a week to a project called Google Brain. Many at Google have doubted this technology. “A waste of talent,” recalls Alan Eustace, Jeff's manager at the time. Sanjay also could not understand the decision of a friend: “You are working on infrastructure, what have you forgotten there?”

Over the next seven years, the Google Brain team developed neural networks that showed the world's best result in machine translation, speech recognition and images. In the end, these neural networks replaced the most important Google algorithms for ranking search results and advertising targeting, and Google Brain became one of the fastest growing teams in the company. Claire Tsui, Google’s engineer since 2001, says that Jeff’s decision was a turning point for Google’s AI development: “Some believed it, others didn’t. Jeff has proven that it can work. ”

It turned out that the AI ​​critically needed the scale, and system engineer Jeff Dean provided it. As part of this project, he led the development of a program called TensorFlow - it was an attempt to create something like MapReduce, only for AI. TensorFlow simplifies the distribution of a neural network in a cluster of computers, turning them into one big brain. In 2015, when TensorFlow was released to the public, it became the de facto standard for working with AI. Recently, Executive Director Sundar Pichai announced that Google’s main business is Artificial Intelligence, and appointed Jeff to lead all AI initiatives.

Now, Jeff pays four days a week to managing Google Brain. He leads the work of 3,000 people. He travels to talks, holds weekly meetings to work on a new computer chip (a tensor processor designed specifically for neural networks) and helps in developing AutoML, a system that uses neural networks to design other neural networks. He has time to program with Sanjay only once a week.

History erases engineering feats. We remember the great explorers of the eighteenth century — James Cook, George Vancouver — but not John Harrison, a carpenter from Yorkshire who, after decades of work, made the clock reliable enough to measure longitude at sea.

Recently, Jeff and Sanjay enjoyed margaritas and enchiladas in their favorite Mexican restaurant, Palo Alto Sol. Jeff pulled out his phone and asked, “When did Gmail come out?” The phone answered: "April 1, 2004." Sanjay, who observed the etiquette at the table, did not seem to appreciate the rush of a friend, but Jeff was delighted. Now Google is able to talk, listen and answer questions using a stack of programs that are easily integrated and largely invisible, stretching from its phone to data centers around the world.

Today, their roles have diverged. Sanjay at Google is known as an “individual member” - a coder who works alone and does not control anyone. He is grateful for that. “I wouldn’t like to work like Jeff,” he admits. He is currently creating software that will make it easier for engineers to combine and manage dozens of programs — news, photos, prices — that begin to work as soon as a user enters text into the Google search box. Once a week, he meets with a group of "technology leaders" (Area Tech Leads) - the engineering "Jedi Council" at Google - to make technical decisions that affect the entire company. If Google were home, Jeff would build an extension, and Sanjay would strengthen the foundation, strengthen the beams, and tighten the bolts.

Meanwhile, during their pairing programming sessions on Monday, they started something new. This is an AI project: as Jeff says, trying to train the "giant" ML model to perform thousands or millions of different tasks. Jeff thought about this idea for many years; he recently decided that this was possible. He and Sanjay plan to build a prototype around which the team can grow. In the software world, the best way to manage a team is based on software code.

“I think they miss each other,” says Heidi, Jeff's wife. Friday dinners with the family appeared when their collaboration slowed down.

On Sunday’s March, Jeff and Sanjay met for a country outing. The weather is clear, although it is hot in the sun. Jeff arrived at the hiking trail on a blue Tesla Roadster with a Bernie 2016 bumper sticker. Sanjay soon followed him on a red Model S. Sanjay read a book in the morning, Jeff played football (the device on his wrist showed that he ran 11 kilometers. Two decades later after fixing that index, Jeff looks like a tanned, resilient endurance athlete. Sanjay doesn't seem to have aged at all.

The path is a 10-kilometer loop through dense forests. Jeff went first. In the forest, they remembered how quickly Google had grown. Sanjay recalled how, after the company's first consolidation, a plumber installed two toilets in the same toilet in the men's room. “I remember Jeff’s comment,” he said. “Two heads are better than one!” He laughed.

They descended from the forest to a dry open area. A vulture flew in the sky.

“The mountains are cooler than I thought,” Jeff admitted.

- And someone said that this is a flat trip.

“I think that's why there are no bike lanes here.”

They climbed back into the forest. Rising up the hill, Jeff saw a gap among the trees: “At some point there will be a good view,” he said.

The trail went to the top of the hill, high and wide, without a forest, with a panoramic view. Haze on the horizon did not close the Santa Cruz Mountains in the south and Mission Peak in the east. “Sanjay, here is your office!” Said Jeff. They stood together and looked across the valley.


Also popular now: