Rust in 2019 and beyond: growth restrictions

https://graydon2.dreamwidth.org/263429.html
  • Transfer
As requested, here are my suggestions for the development of Rust in 2019 and beyond.

I must say that I speak only for myself, and I am not even a very active participant in the project. Moreover, these proposals largely apply to many projects. Rust - a special case, but it is he who now leads to some reflections.

I should also note that I am generally pleased with the development of Rust, and this proposal is made only for the sake of maintaining further well-being, in order to avoid some of the problems that I now observe from the sidelines.

TL; DR: It is important to recognize the problem and plan explicit mechanisms to limit the growth of two things:

  1. Mandatory general technical artifacts, especially the definition of the language itself.
  2. The burden on people involved in the discussion of these artifacts.

In particular, I want to draw attention to the impossibility and undesirability of unlimited growth in both directions. There are natural limits. As in all natural systems that have reached growth limits, it is better to prepare for this event and conduct it in a controlled, planned way.

Please note that I do not write about the growth limits of many other areas. For example, the package index, site size, or even the user community. It is unclear what threat Rust bears, so now we are talking only about two specific issues above.

Limiting factors and flight


Every natural system has limits to growth. That is why the Universe is not (for example) a single amoeba that expands at the speed of light. The system grows (and often the speed of expansion also grows!) Until it is confronted with limiting factors, and then gradually slows down until the overall size of the system reaches a plateau. In this case, typical growth patterns look approximately like a sigmoid or “S-shaped curve”, gradually approaching some asymptote. At least, if limiting factors occur gradually and in a controlled way.



When a system encounters a limit in an uncontrolled manner or suddenly, a phenomenon that looks more like a flight of a target or even a return may occur: the limit still exists, but its effect is felt rather in the form of a collapse or a crisis. S-shaped curve rises to a peak, followed by a collapse. I would like to avoid this.

Examples of good control


There are several forms of process management in the Rust project that essentially limit the rate of change and / or growth. I consider these measures very reasonable: partly thanks to them, the project is still successful. By analogy with them, I want to formulate the following recommendations. Current forms of management:

  • Bors queue skips program correctness changes.
  • Crater skips releases on correctness for the ecosystem.
  • Scheduled releases prefer to be released on time, even if the planned function is not ready. The decision is made in time, and all unprepared is cut off.

In addition, the project has created important social structures to limit the number of project participants.

  • Code of Conduct . Not everyone remembers, but he does not just postulate social justice and so on. It also sets limits on the signal-to-noise ratio in conversations, the exploitation of someone else’s attention and time, and pushes for trade-offs (after all, not every solution gives a zero amount).
  • RFC process . Rules about the form, content, timing, recruitment of participants, allowed and expected forms of discourse when discussing significant changes.
  • Management model . The division of responsibilities, hierarchical delegation, where necessary, the roles and expectations of participants and so on.

All this is, in fact, the recognition that in the absence of control, trouble and crises can occur: at least, chaos and dysfunction to a certain extent. If possible, control is automatic and completely impartial (to minimize malevolence and subjective assessment, the temptation to cut corners, etc.) If subjectivity is not avoided, at least, the norms and processes are clearly formulated in writing, in well-documented, well-known places .

Problem areas


Let us return to the two problem areas, where the project now does not have adequate mechanisms or policies to control Rust, which carries the risk of possible dysfunction or even a crisis. In both cases, it is not entirely clear how far the project is now from such a crisis. But in any case it is better to act on preemption than to be late.

  1. The language itself . His definition. This (unlike many parts of the project) is an obligatory general technical artifact. Everybody is interested in it, and every change potentially affects everyone. Moreover, everyone should study and understand its essential part: it is impossible to ignore uninteresting parts. Even what I want to ignore exists in the general context: documentation and training materials, sample tests and test material, internal components of the compiler, formal models, code bases, total service load, etc.

    Here the growth of the language as an artifact is limited at least following factors:

    • The ability for a novice to learn a language.
    • The ability of the average user to feel confident, to adapt to other people's code bases.
    • The ability of an expert or maintainer to know all (or most) changes.
    • The cost-benefit ratio of each new change in terms of new and current work. The number of people or usage examples that benefit from it. The costs are combinatorial in many dimensions of the project and the size of the language. They almost always increase.

    If you do not comply with these limits, you can face very serious risks:

    • Unreasonable changes in the language, including the impossibility of maintaining guarantees of critical security.
    • The reputation of excessive complexity, the loss of users. The risk of becoming the next C ++ or Haskell.
    • Poor-quality functions with incomplete definition, tests, documentation.
    • Unusable functions that have been spent on efforts needed elsewhere.
    • Crushing into dialects, isolated programs, value reduction.
  2. The load on people working on the language. Some parts of the project can be delegated, distributed among all available developers. These are not general technical artifacts. To some extent, many people (and more and more of them) have to participate in almost all changes. This means a lot of pressure on everyone in this group: they have to follow all discussions, and in fact both the number of changes and the number of discussion participants are growing.

    Some restrictions on the growth of this load for project participants:

    • Number of hours per day.
    • The number of paid hours per day.
    • Responsibility and interests outside the project.
    • Reserve mental energy to understand what is happening.
    • Trust the opinion of all those involved in the conversation.
    • Reserve of psychological and emotional energy for reading and discussion.
    • The presumption of good faith of everyone involved in the conversation.

    The risks of exceeding these limits are also potentially very serious:

    • Bad decisions made due to exhaustion or burnout.
    • Rising inequality: only the most privileged, accessible, energetic, well-paid, or otherwise well-organized participants can follow everything.
    • Narrowing discourse: from careful consideration to winning a dispute.
    • People play the fool, burn out, misbehave, leave the project.
    • Disappointment, accusation of bad faith, offense, conspiratorial thinking, forks.

    I want to emphasize that the described limitations and risks are completely not specific for specific people or even the Rust project as a whole. In varying degrees, they are found anywhere. Simply replacing current maintainers (for example) with those you like will not really solve the problem: the limitations will remain. The only solution is thoughtful management in the face of a collision with a limit. Take control.

Possible control options


This is the hard part where I will try to avoid clear wording. In the end, it is important to take control of your own free will, and not imposed from outside. I think that project participants should pause, reflect, collectively consider and establish some of the controls. Therefore, I will only offer several possible options, not very structured, but in a festive Christmas spirit: like a bunch of potentially interesting gifts to expand, see and decide, leave or exchange for something more interesting.

  1. Negatively defined space . Take out the process of discussing functions and concepts from the plans for the future development of a language. Allow (or encourage) RFCs that say “Rust will never have X” for some X value. So we get a one-time round to fairly discuss and consider objections to long-term change (“never” is quite a long time!). Then this discussion will end forever. It will not be the eternal source of protracted conflict. A few examples where you can formulate negatively defined spaces:

    • permanently remove certain expressive categories from the type system (for example, dependent types, HKT, etc.);
    • from syntax (for example, parameter keys, positional or named arguments);
    • from a set of item types (for example, anonymous record types);
    • from the set of obligations for the conclusion in the middle-end (for example, the synthesis of constants, implicit arguments).

    Set some hard limits: to avoid these objects, as well as people who have set the goal of "doing everything right."
  2. Development costs need to be explicitly stated. Taking a page from the list of changes to WebAssembly, make it clear that already at an early stage such an RFC will require appropriate investments in implementation, formalization, revision of documentation, revision of training materials, test writing, maintenance, etc. If it is not possible to cover the costs, at this stage set aside the changes "until a sponsor is found."
  3. Set goals for learning speed and volume of material . Try to work in the opposite direction: proceed from the amount of time and the number of pages required to learn a language or to become an expert in it - and then delete everything that goes beyond that. If “learn Rust in 21 days” does not fit, find a suitable interval. Three months? Six? Year? Think of languages ​​that “definitely takes too much time” to learn, and choose a smaller number. Is the 1000 page manual normal? 500? 300?
  4. Set other automatic limits : the number of lines of code in the compiler, the total load time, dollars per day for AWS instances, the number of rules in the grammar, in the type system, the percentage of test coverage, the percentage of documents that can be marked as “incomplete”, etc. Get creative, figure out meaningful things that can be measured, and then implement mechanisms to limit them.
  5. Personal time limit : approximately how many hours (or paid time) a person can really devote to the project without exhaustion or burnout, including participants with minimal rights. Set similar restrictions on groups, individual releases and on relevant work plans. Then remove or set aside everything that does not fit into the limit.
  6. Allow moderators to set limits on the frequency of messages or set periods of silence in specific discussions. Sometimes it seems that the discussion is too hot. To de-escalate a conflict, it is easier to set a general limit than to apply sanctions to individual participants.
  7. As with the moderators: create an additional inter-project team, which is engaged in budgeting and auditing of load levels in other teams . This can be effective: the audit team will help people say “no” when necessary, and not endure, as most team members do, agreeing to too much.

These are just some ideas in the general direction of growth restrictions. I am sure that you can come up with more realistic ways to control restrictions on the size of the language and personal load. Over the years, the Rust community has proven to be amazingly creative, thoughtful and self-critical. For this you should be commended. I hope this article will be received in the same spirit of constructive criticism.

Happy New Year and good luck!

Also popular now: