Source: "Conway's Law"

Transfer

Author's note 42 years after publication:

Perhaps most noteworthy in this article is that it was published thanks to the statement described in the third paragraph from the end. To ease the difficulty of overcoming the previous 45 paragraphs, I will voice it in a free form now:

Any organization designing a system (interpreted here more broadly than just an information system) will inevitably create a model that will repeat the communication structure of the organization itself.

It turns out that this principle is observed not only in the development of software, in the context of which it is often mentioned.

I suggest you familiarize yourself with the material, and then look back and look for it in other areas.

Currently, my favorite example is the complex social issues relating to poverty in America: access to the labor market, housing, education and health care. After reading the article, think about how the structures of our various governments influence their attitude to similar issues.

How do committees create new things?

Melvin Conway (Melvin E. Conway)
Original PDF .

Intellectual activity, as a result of which a whole is created from disparate parts, can be called a system design. Whether it is the preparation of a large military complex, recommendations for solving social problems or a list of computer programs. A typical goal of a design organization is to prepare a document containing a coherently structured amount of information. Let's call this information a system model. Typically, the initiator is the customer who wants to use the model to provide any further action. For example, a statesman intends to propose a bill to prevent the recurrence of a recent disaster, for this he hires a team to find out the causes of the disaster. Or the manufacturer requires a new product, and he appoints the responsible persons for determining the new product and planning its release.

The project organization may or may not be involved in implementing the designed system. Often, in government initiatives there are rules that prevent the group from acting in accordance with its own recommendations, while the opposite situation prevails in the private sector. It would be reasonable to assume that this condition will affect the design process of the system, along with other ideas of the developer about their own future. As we will see later, the incentives that exist in the traditional management environment may contradict the goals of the customer. [one]

Design Stages

The initial design stage refers to the structuring of the process itself rather than the development of the system [2]. Certain preliminary actions precede full development, namely:

Defining the boundaries of the project activity and the final result set by the customer and the existing realities.
Initial assessment of the structure of the system with the aim of a balanced formation of development teams.

In the future, we will explain in detail that the organizational structure of the team already affects the implementation of the project, directly or indirectly. So, even the inability to provide ideal conditions for the search of employees, the formation of the team will not be completely objective. And after forming the team, responsibilities will be delegated to subgroups, narrowing the field of study, which will simultaneously reduce the number of options for developing the system.

By assigning responsibilities, managers face the problem of coordination at the level of working groups, which negatively affects the consolidation of efforts of each group in order to create a uniform system. Despite the possible reduction in this role of the individual.

The system design process has the following stages:

The definition of boundaries in accordance with the basic rules.
Development of a preliminary system concept.
Organization of design work and delegation of tasks in accordance with the concept.
Coordination at the level of delegated tasks.
Combining subsystems in the system.

In a single design work, all stages may not be traced. It may happen that in the process of work a new, more perfect concept of the project will be found. This, of course, is not an entirely favorable moment, since the subsequent termination of work will be painful and costly. Of course, from the point of view of the historian, everything repeats.

In that case, why doesn’t there always be enough time to do everything right, but there is always enough time to redo something?

System design

Any sequential system consists of interconnected subsystems. The description of the internal structure of the system should reflect its connections with the external environment and the interrelation of its subsystems. Going down a level, you can say the same about the subsystem, considering it as a system. And so on, up to such a subsystem that will be extremely simple and indivisible.

Examples

The state transcontinental transport system consists of buses, trains, airplanes, taxis, parking lots, terminals, etc. This is a very heterogeneous system, which means its subsystems are quite diverse. Going down to the level below, to the level of the aircraft, for example, we will see its subsystems: frame, engine, power distribution, communication, payload. The engine system includes such subsystems as fuel, ignition, etc.

Not so obvious, but theory is also a system. It relates to the external environment, to observable events, which it must explain or, at least, not contradict them. It consists of parts-theories that relate to each other in a similar way. For example, during the investigation of the crash, a theory was created describing the trajectory of the aircraft, its messages, damage, and interaction with other objects at the time of the accident. Each of these items is itself a separate story and can also be divided into sub-items, up to individual pieces of information.

Scheme

In Fig. 1 system is depicted in the form of a scheme - similar to a rattle - with links (branches) in the form of lines and main nodes (nodes) in the form of circles. Each node symbolizes a subsystem that communicates with other subsystems that can be represented in a similar way. The term interface (interface), which is gaining popularity among developers, refers to the interaction of subsystems, indicated by lines.

The graphic image clearly demonstrates the same form of the two concepts that we consider: the system and the organization that designs it. Try replacing:

“System” to “committee”
"Subsystem" on "subcommittee"
"Interface" to "coordinator"

As in the case of systems, we see that design organizations can be considered with several levels of complexity. The federal government is a great example of a project organization with a level of complexity that can satisfy any engineer. This is a particularly interesting example showing the similarities between the two concepts we are considering, because the Federal Government is also a project organization (drafting laws, contracts, policies) and a projected organization (with the Constitution as primary project documentation).

Main interaction

Now we are ready for the main issue of this article. Is there a predictable relationship between the structure of the design organization and the system it designs? The correct answer is yes, and this connection is so simple that it is often permanent. Consider the following "proof."

Let us arbitrarily choose the system and organization that developed it and choose randomly the level of complexity of the designed system, depict it schematically (our choice is arbitrary, because if we observe interesting interrelations, we will be able to extend them to any design organization and level of complexity). In Fig. 2 shows (for clarity) a structure for which the following statement is true.

For any node x in the system, we can identify the working group of the project organization that developed it (we denote it by X). Thus,

summarizing this process, for any node of the system we have a rule for searching for the corresponding working group. Note that the 1: 1 ratio is not necessarily preserved here, i.e. Two subsystems can be designed by the same project organization.

Interestingly, we can make a similar statement regarding links (branches). Take any two nodes of the same system, x and y. They may or may not be connected (ie, they either interact with each other in any way important for the functioning of the system, or they may not). If there is a connection between them, then the two working groups X and Y, who developed the two data nodes, had to agree on the interface features in order to realize the possibility of communication between the nodes. If there is no connection between x and y, then the subsystems do not communicate with each other, and therefore the working groups did not interact with each other (since there is no connection between X and Y). [3]

What did we just show? In simple words, we have demonstrated that there is a close connection between the structure of the system and the structure of the organization that is engaged in its design. When each subsystem has its own separate working group, we can observe that the structures (schemes) of the working group and the subsystem are identical. In the case when several working groups develop one subsystem, the structure of the design organization looks like the structure of the system, but with a smaller number of subsystems (nodes in the diagram).

Such a structure-preserving relationship between two objects is called a homomorphism. Speaking in mathematical language, there is a homomorphism between the scheme of the system and the scheme of the projecting organization.

The system is a reflection of the organization designing it.

Many experienced system developers believe that one day someone can do a better system project. In other words, it is erroneous and incorrect to speak about a design task except in the context of place, time, level of knowledge and technology. This should be remembered by those who read the story of the work on the design of an object or remembers and analyzes their work.

The development of computer translators for programming languages such as FORTRAN and COBOL is an excellent example. In the mid-50s, when prototypes of these languages appeared, their compilers were even more cumbersome than the computers themselves, gigantic in those times that the commands had to execute. Today, these translators are only historical wonders that have nothing in common with modern compilers. (It is worth noting that a huge breakthrough in the development of compilers was associated with the emergence of new groups of people in the field, previously considered the domain of computer manufacturers - it was first a small cohesive university group of researchers, followed by independent software developers).

If we assume that for any system requirement there are a number of system projects that will meet this requirement, we should also ask ourselves whether the choice of the design organization influences the process of choosing a system project from this series. If we believe in our homomorphism, then we will agree with what is affected. Given that the organization does not have a flexible communication structure, this organization will stamp a copy of itself in any project product. The larger the organization, the less flexible it is and the more pronounced this property becomes.

Examples

In one research organization there were eight people who had to do the compilers COBOL and ALGOL. After the initial assessment of the complexity and possible deadlines, five of them were determined to work with COBOL, and three were assigned to work with ALGOL. As a result, the COBOL compiler worked in five passes, and the ALGOL compiler worked in three passes.

Two military services under the leadership of their commanders developed a weapons system for their needs. After, not without effort, they made a copy of their organizational structure (see Fig. 3a)

Pay attention to the operating system of the computer involved in this task. Having studied it in detail, we see that it consists of three parts: hardware, software, and applications (see Fig. 3b). Their subsystems correspond to their developers: computer manufacturer's engineers, its system programmers, and custom application developers (the rare case in which hardware and software specialists work together, rather than just having difficulty tolerating each other).

System management

The structure of large systems tends to decay as it progresses. This observation is particularly evident when considering the major military information systems of the past ten years. They are the most complex objects of all previously created by man. An activity called “system management” has arisen, including due to the tendency of systems to split into components. Let's look at the practicality of managing systems in the context of the main idea of this article.

Why do large systems break up into component parts? This process goes through three stages, the first two of which are controlled, and the third stage is a direct consequence of our homomorphism.

First, developers, realizing the size of the designed system, cannot resist the temptation to hire as many people as possible to create it.
Secondly, when applying the usual management model for the implementation of such a large-scale project, its communication structure falls into separate parts.
Thirdly, the disintegration processes occurring in the structure of the organization-designer, occur in the structure of the system, in accordance with the phenomenon of a homomorphism.

First, consider the situation with "overpopulation." The natural desire of a developer to delegate tasks when the level of complexity of a system approaches the limits of its understanding is quite understandable. This is a turning point in the development process. Either he fights for the simplification of the system and wins, or he loses control of it. The outcome is almost predictable, given the pressure of the time frame and budget.

If you fail to meet the deadlines, the manager will be exactly to blame if he did not use all the resources to prevent this. Therefore, he will try to influence the developer, who, perhaps, would prefer to deal with the complexity of the system, rather than delegate tasks to others. As a result, the number of resources involved will increase.

Or another example that demonstrates a similar case of the conflict of the manager and the integrity of the designed system. The manager must give the implementation a critical and difficult part of the work. He can choose one of two performers: a new small company, which is distinguished by creativity and low pay, or a reputable reputable office that has more “realistic” prices. He knows that if a bright young company does not cope with the task, he will be guilty of disrupting the project. If a well-known organization fails, it will only indicate that the task is indeed difficult.

What is the catch here? Basically, in the method of assessment and allocation of resources, which is traditionally used. Thus, the unit of the resource is the dollar, so all resources are expressed in dollar terms. If the resource is human labor, the unit of measurement will be the cost of one hour of work multiplied by the number of employees.

The fallacy of this approach is that the work of two people during the year is assessed as the work of hundreds of people during the week. But since the organizational structure of the work of two and one hundred will be different, given the homomorphism, it can be argued that they will create different systems, so you should not compare the cost of their work. We know from experience that two properly selected people, if they manage, will show the best result. Assumptions that may be true for peeling potatoes or laying bricks are erroneous when developing systems.

Parkinson's law [4] is of great importance in the redistribution of labor resources in project activities. As long as the manager’s reputation depends on the size of the budget, he will be interested in expanding his organization. This is not the appropriate motivation for solving problems in the development of systems. Once an organization exists, then of course it will be used. Perhaps the most compelling reason for the existence of such a number of poorly developed systems today lies in the presence of design organizations looking for work.

In the second place, the reason for the collapse of the system design process - the fragmentation of the communication structure of the project organization - begins to manifest itself with the beginning of the delegation of tasks. According to the elementary probability theory, the number of possible communication links is half the square of the number of employees in an organization. Even small organizations have to limit communication so that people can finish the job. Research in the field of improving communication technologies between developers would have a strong impact on system management.

In organizations of a military type, quantitative restrictions of the administrative structure are usually used. So, everyone should have a superior and about seven subordinates. Communication is carried out in accordance with the principle of subordination. Therefore, such organizations develop systems whose structure resembles their own organizational structure.

Conclusion

The main thesis of this article is that organizations engaged in the development and design of systems (here it is interpreted in a broad sense) are doomed to create projects that copy their communication structure. We have seen that this property has a major impact on the management of the design process. We identified an important criterion for the organizational structure of design organizations: resources aimed at solving a problem should be increased only in accordance with the communication capabilities of the organization's structure. This criterion creates difficulties, since the need for communication depends on the concept of the system at a certain time. Due to the fact that the assessment of the task at the initial stage is never optimal, the concept of the system may require changes, which, in turn, reminds of such an important quality for the company-designer,

It is necessary to find ways to reward managers of the development process so that they are interested in the economy and flexibility of their companies. A new philosophy of systems design and development is needed, which will not be based on the assumption that additional labor increases productivity. Development in this direction will draw attention to the need to address basic questions about the cost of resources and methods of communication. Without this, it is impossible to imagine progress in the development of systems.

Notes

[1] A much clearer discussion of the behavior of project companies can be found in John K. Galbright’s The New Industrial State). Pay particular attention to Chapter VI, "Technostructure."

[2] For a more detailed discussion of turning a project activity into a project in a functional environment, see C. Middleton's “How to Establish a Project Organization” (C.J Middleton, “How to Set Up a Project Organization,” 1967, p. 73).

[3] This statement can be viewed from different angles. It can be trivial, closely related to the definition of important negotiations. Or it may be the result of observing that one group of developers will never agree to changing their own design to suit another group, without ordering it.

[4] S.N. Parkinson's "Parkinson's Law" (C. Northcote Parkinson, Parkinson's, 1957)

Translation: Sergey Danshin

Still

About the project "Engelbart"

Google translation with “Augmenting Human Intellect:
A Conceptual Framework” here , can be supplemented.

The impossible task for the Engelbart project is to “reload the matrix”, “rebuild” the entire field of information technology, the Internet and computer hardware, taking into account all the errors of the first (current) version.

The next steps are translations and collecting key conceptual documents in one place and searching for like-minded people ( wake up, Neo! What you are looking for is also looking for you .) At gunpoint - Vannevar Bush, Joseph Liklider, Paul Otlet, Alan Key, Douglas Engelbart, Glushkov, Lebedev, Ershov, WikiPedia, Web Archive, Knol, Quora, Cybersyn, Xanadu, DARPA, IARPA.

more details -50 years later. The mother of all demos

If you really want to understand NLS, you must forget the reality of today. Forget everything you know about computers. Imagine that you still do not know what a computer is. Go back to 1962. And then read what he has in mind .
- Bret Victor, a few words about Douglas Engelbart

useful materials