Richard Hamming: Chapter 26. Experts

“What led you to success may become ineffective in the future.”

Hi, Habr. Remember the awesome article “You and Your Work” (+219, 2244 bookmarked, 351k reads)?

So Hamming (yes, yes, self-checking and self-correcting Hamming codes ) has a whole book written based on his lectures. Let’s translate it, because the man is talking business.

This book is not just about IT, it is a book about the thinking style of incredibly cool people. “This is not just a charge of positive thinking; it describes conditions that increase the chances of doing a great job. ”

We have already translated 8 (out of 30) chapters.

Chapter 26. Experts

(Thanks for the translation, Mitroshin Evgeny, who responded to my call in the “previous chapter.”) Who wants to help with the translation, write in a personal email or e-mail magisterludi2016@yandex.ru

As noted in the fifth chapter, the number of problems facing us is growing exponentially , and as a solution, we deal with this through specialization. This statement over the years becomes only more true:

An expert is one who knows everything in something small, an erudite is one who knows little, but about everything.

In a dispute between an expert and a scholar, an expert usually wins, simply because he uses terms that are not understood by anyone and refers to his specific experience, which is often not related to the subject of the dispute. However, the opinion of experts must be taken into account. Since experts at the same time play a very important role and, sometimes, interfere with the development of the industry, their opinion should be evaluated from all sides. Often the expert underestimates the problem posed, and the erudite cannot cope with the task due to the lack of deep knowledge. Anyone who thinks he understands the task, but in fact not, is a real misfortune, compared with someone who knows that he does not understand anything in the question.

Kuhn (note: Thomas Samuel Kuhn, American physicist, historian and philosopher of science) in his book Structures of Scientific Revolutions considers the structure of scientific progress and introduces the concept of paradigm as a description of the normal state of science. He notes that in any single scientific field, there is almost always a set of generally accepted assumptions, often not discussed, that are presented to students, and those, in turn, easily take them for granted. There is also a similar set of opposing hypotheses. Workers of science, working in this vein, expand and complicate the sphere to impossibility, often ignoring the contradictions that may arise in their way.

From time to time, usually due to these forgotten or ignored contradictions,
an unexpected paradigm shift arises, as a result of which new judgments become dominant and new questions appear along with answers to old problems. Changes of this kind in the generally accepted paradigm of science usually portend significant progress in the development of the sphere. An example is the special theory of relativity and quantum mechanics in physics.

At first, changes are accepted by hostility by key people who put in a lot of effort while working on the study of the previous picture, but, usually that Kuhn and other researchers notice, new ideas prevail over old ones. Sometimes it takes more time to adopt a new paradigm than was allocated to the life of the discoverer of new ideas! For example, I previously mentioned the theory of continental drift, which was addressed by Thomas Dick in 1838 (note: here the author refers to the work of Celestial Scenery, or the Wonders of Heavens Displayed), and then Alfred Wegener in his book dated 1900. When my wife and I were children, we (independently of each other, because we weren’t familiar yet) read the book by Alfred Wegener and noticed that, indeed, the coastlines of Africa and South America are very similar to each other, Wegener's observation was also a powerful argument - he notes that not only the shape of the coasts is similar, but even some rock formations on two continents correspond to each other like two parts of a mosaic! Despite the fact that even in the eyes of the child this theory seemed obvious, experts of that time did not actively consider it, geologists did not take the theory seriously.

In favor of the theory of continental drift, there is another argument - the distribution of various forms of life among centuries. Similar forms were found in physically separated places, such that they inspired thoughts of “earthen bridges” that were once on the surface and then sank. Their estimated number and location seemed incredible to me. Biologists studying ancient life forms tried to explain their observations using the hypothesis of Pangea and Gondwan, the predecessors of modern continents, and did not particularly think about the mythical "earth bridges", however, geologists did not want to listen to anyone. The theory of continental drift was approved by oceanographers only after the Second World War, during the study of the bottom of the oceans and magnetic anomalies, during which the theory of spreading was confirmed.

Of course, geologists can now argue that they always somehow believed in drift and that it was only necessary to shed some light on the details in order for them to accept the theory that is now accepted as the most plausible. This is a typical example of a paradigm shift. At first, when for a long time, when not very, the hypothesis is not accepted (I don’t know how many theories as a whole did not survive in such a struggle, and I can’t know), however, when it still enters the minds, those who objected to it , begin to argue that, in general, they never completely removed it from the accounts. You probably heard a lot of similar examples, the Wright brothers heard before their historical flight that the device could not fly heavier than air, they also spoke about the impossibility of flying at supersonic speed, etc.

I really like one example, based on the assumption that it is impossible to pump water to a height of more than ten meters (approx: 33 feet). When a patent office worker rejected a patent application stating that it was possible, the inventor showed how he pumps water into a tank on the roof of his house, which was clearly more than ten meters from the ground. How did he do it? He used the design shown in Fig. 26.1, resorted to the help of standing waves, which were not known in the patent office (note: here I am not a specialist, I mean patent US3743446). The bureau considered this impossible, since it was written in books that those specialists trained in, no one thought about what principles lay in the foundations.

All evidence of impossibility is based on judgments that may or may not be applied in a particular situation.

When an expert is faced with something new, his expertise and the habit of looking at things remain with him. If something does not fit into his picture of the world, then it either goes unnoticed or is fitted into the picture. Therefore, truly innovative ideas are rarely born within the sphere, and, in general, it would not be very correct for experts to blame this, because it is more advisable to try to explain something using old and proven methods before finding new solutions.

Everything that is recognized as impossible is obviously based on some statements, and if one or several statements turn out to be incorrect, then the “impossibility” is called into question, however, experts are rarely scrupulous in checking basic assumptions before declaring it impossible. There is a rather old expression on this subject:

“If an expert says that something can be done, then he is most likely right. But if you are told that this cannot be done, then it’s better for you to ask someone else. ”

Kuhn and other historians of science focused on the study of significant changes, it seems to me that the principle is common to everyone, including the smaller ones.



For example, while working at Bell Telephone Laboratories, I naturally had to follow the frequency approaches to numerical analysis, and therefore apply it to the numerical methods I use for the problems that I was asked to solve.

If you use the functions that customers are familiar with, the conclusions from the solutions obtained can show that there are other things that they did not initially consider. I found the frequency approach very useful, but some of my close friends, not from Bell Telephone Laboratories, regularly taunted me at every meeting for many years. They simply adhered to a polynomial approach, although when asked they could not give any real reason for this - it was just how everything should be done, therefore, it was the right way to do something.

Poking in the direction of experts is not the purpose of this chapter, the reasons why I write it below:

• Firstly, you will repeatedly meet experts on your way and it will be nice if you are familiar with their type
• Secondly, after a while many of you will become experts and I hope that you will listen to all that has been said and will not impede progress, as your predecessors sometimes did
• Thirdly, it seems that the current pace of development is much faster than before, which means that paradigms will change more often and you will have to withstand more changes than I found
• Fourthly, I would like to think that I found the right words in order to protect many of you from becoming useless after the established paradigm in your area has changed.

Let me also touch upon one point that we have hardly discussed before. It seems that most of the great, innovative ideas are generated outside the scope of research than in the circle of its researchers. You remember the example of the theory of continental drift. Consider archeology - one of the central tasks that archeologists pose is to determine the age of the finds.

Previously, it was solved only with the help of complex and not always reliable methods of stratigraphic dating. Now in most cases the method of radiocarbon analysis is used. Where did he come from? From physics! It is unlikely that any of the experts in archeology could come up with this. Another colorful example, a step seeker (note: the predecessor of the switch, an element of an automatic telephone exchange, patent US447918) was invented by the owner of the funeral home, Elmon Strowger, who was annoyed by the behavior of the telephone operator, whose husband owned the rival office and calls addressed to Elmon were often redirected to competitors.

Such cases take place in most scientific fields, but, unfortunately, they are not often mentioned in textbooks. Einstein at the beginning of his career worked in the patent office - he could not find official work in the university environment. A little later, he was recognized and was given the opportunity to take prestigious posts, finding himself in Berlin, and then in Princeton.

Thus, an expert faces a dilemma. Hundreds of crazy ideas swarm outside the scope of his research, but among them there may be those who later assume a dominant position. What behavior strategy will be rational?

Many people prefer to ignore all new trends coming from outside, thus depriving themselves of the opportunity to adapt if the paradigm changes. Those who consider proposed innovative ideas can spend a lot of time chasing a phantom correct concept that will eventually bring something new. Obviously, the strategy you choose depends entirely on your attitude to these ideas. I want you to remain reasonable when you get to this crossroads. You don’t just have to go with the flow, you should see your goal and have a plan on how to reach this goal. It’s not necessary to immediately reject any, even possibly crazy, ideas as soon as they appear before you, especially if it was expressed by a person who is not a specialist in this field of research - after all, this may be the very thing which will change everything! On the other hand, this does not mean that you can and should take into account everything that is in the air. Until this moment, I talked about changes in scientific paradigms, but, it seems to me, this approach extends to any field of research. It seems that the reason is the same everywhere - experts are sometimes overly confident in themselves, they put a lot of effort into studying the current picture and are quite inert. I think you yourself can recall many such cases that have taken place in the history of science. that the reason is the same everywhere - experts are sometimes overly confident in themselves, they put a lot of effort into studying the current picture and are quite inert. I think you yourself can recall many such cases that have taken place in the history of science. that the reason is the same everywhere - experts are sometimes overly confident in themselves, they put a lot of effort into studying the current picture and are quite inert. I think you yourself can recall many such cases that have taken place in the history of science.

I touched on two main problems associated with expert opinion. The first problem is that they are extremely confident in their innocence. Second, they are not always able to assess where the scope of the judgments on which their knowledge and their applicability to new issues rests. Remember my story with fast Fourier transforms and the Cooley-Tukey algorithm, which could be called the Tukey-Hamming algorithm (note chapter 16). This is not the only example when I made such a mistake, forgetting that progress does not stand still and can radically change the view of things that previously seemed impossible. No matter how ashamed I was to mention this, I told you about this case in order to convey to you in all its glory the importance of the issue. How do you plan to avoid such a mess, when will your turn come? At one time, no one told me about such things, I hope you will not make mistakes like mine.

With a more active development of technologies in the future, such errors will occur more often, at least it seems to me. Experts live in their own closed theoretical world, no doubt they are often right, but deaf to the opinions of others. Despite all their merits, experts are the real scourge of our society, with their infinite confidence in their boundless knowledge and lack of due doubt in their infallibility. When you face a serious problem, remember my words from chapter 8 - “what will you accept as evidence of your wrong?” . Ask yourself this question more often, along with “why am I so sure that it works like that?”, Especially when it comes to the area where you are considered an expert.

Experts do not always openly resist change, as I describe above. I bring to my attention my experience at Bell Labs at the dawn of the computer age. All my leaders succeeded in the field of mathematics and in their golden years working with computers was not considered something prestigious, but rather as a suitable lesson for yesterday’s schoolchildren. The chefs knew how to handle mathematics, the computers in their eyes were much lower than a living mathematician, it was pointless to discuss this with them, I would say that as the machines developed they began to hate them a little. I had to work with computers despite the rejection (usually implicit) of my leadership - they always said that with the help of a machine I can hardly solve a particular problem, although, usually, by that time it had already been solved by me. However, I tried not to get involved in these disputes, I was more interested in doing something that they could not do, I was fascinated by the thought of what could create the union of man and machine. I am not going to calculate now how many times I have cited this argument against the sometimes very implicit attacks on computers in those days. And it’s in such an elevated place like Bell Labs.

I hope to influence your worldview so that you avoid these ailments of know-it-alls when you yourself become experts. All I ask is, please, often recall all the features that I mentioned and try to look at yourself from the side - are they typical for you? I made a promise to myself that when I climb higher, I will behave more carefully, as a result I refrained from participating in decision-making on current issues in the field of computer computing.



Of course, I will express my opinion if they ask me, but I do not want to be a burden for the new generation, as my predecessors sometimes did. For clarity, I tried to illustrate the pace of progress using Figure 26.II. In the beginning, in the region of 1935 and earlier the derivative was positive, those who knew their business at that time eventually gained success and these were my future leaders. When the computers got into business, the picture changed slightly, now the derivative is negative. This suggests that the previous working methods that previously led to success are no longer so widely applicable. Maybe there is a small amount of truth in this graph. In my opinion, progress has not really stopped, rather, it is accelerating little by little, which means that the advice below is useful to you:

What led you to success may become ineffective in the future.

Please keep this in mind when you are on the edge of progress and give the new generation a little more chance of success than your mentors have given you. Earlier, I told you (approx. In chapter 4) how one of my friends claimed that Hamming did not fully understand the error correction codes and I had to admit that he seemed to be right! I sincerely believe that a venerable expert is often a burden for progress. Again, Einstein, who took such an important step for the development of quantum mechanics, in his work related to the photoelectric effect (note here, probably, this refers to the 1905 article “On one heuristic point of view concerning the emergence and transformation of light”), and subsequently when quantum mechanics began to actively develop, opposed it. Physicists don’t really like to recall this incident, because it does not represent Einstein in the best light,

Now the last and most significant reason why I tell you all this. I have seen and see how many experts again and again find themselves overboard after the subject area of ​​their expertise is changing due to a paradigm shift. Take a look at the history of computer computing with my eyes. In chapter 4, I told you about three things that were repulsed by programmers: symbolic programming languages, high-level software, and FORTRAN, when it appeared. What happened to those who were against these innovations? In the end, no one needed them!

I have a close friend, a big fan of analog technology, who taught me a lot in the field of analog computers when I was a leader at Bell Labs. When digital technology began to develop, he was always not averse to paying attention to the advantages of analog technology (which it had in those days). Little by little, he began to lose his grip, and in the end he had to deal with several other things. When I retired and went to teach (I always wanted to do this, it seemed to me that the old guards of researchers only put sticks in the wheels of the young), he also retired.

However, I left the company with good memories of the time spent in it, which can not be said about him, judging by our conversations after. If you do not develop along with your professional environment, then this will await you. Keep your nose upwind. When I lived in California, I had occasion to communicate with former naval captains in whose stories often expressed a share of hostility to the career path. Could it be otherwise?

If you ignore the important (for you) increase, this will probably not reflect in the best way on the memories of your career later.

In these lectures, I resorted many times to such everyday stories. They describe many situations very well, I know many more other examples. I began to notice these “theories” for quite some time, time and experience have shown that some were true, while others were not. This is not the ultimate truth, but simply the conclusions from the many observations with which I try to prove the truth of my beliefs. Of course, you will say that I am looking only for confirmation of my guesses, however, as a scientist, I am used to looking for rebuttals, so some judgments were eventually rejected. If you think about what I have said, you will notice that the truth of most of these stories is very intuitive and more based on human traits than anything else. We, of course, are all people, but this does not give us the right not to try to step over our instincts. We threw a thin veil over our primitive instincts and this veil is what we call civilization. To be civilized also means "to think first, and then do." So I'm just trying to add a little self-criticism to you so that you become even more “civilized” and perhaps this will lead you to success.

At the beginning of the chapter, I talked about how to deal with experts, but at the end I gave you a couple of tips that will come in handy when you yourself become an expert. Do not make the mistakes that I made!

To be continued ...

Who wants to help with the translation - write in a personal email or e-mail magisterludi2016@yandex.ru