The principles of one Python developer

    In this publication, I would like to introduce to the court of a respected reader some principles that I am guided by in performing my duties as a Python developer.

    On the one hand, I wanted to share the accumulated experience that may be useful for beginner developers, and on the other hand to get feedback from more experienced developers, managers and specialists in related fields.

    The principles are conditionally grouped into three groups: decision-making principles; principles aimed at improving the quality of the code; principles aimed at improving code performance.

    • Making decisions
      • Any technical solution must be justified.
      • Responsibility for the decision always lies with one or those who made this decision
      • When making technical decisions, it is necessary to take into account their effect in time and their relevance to business needs.
      • One of the main criteria when making technical and other decisions should be their greatest effectiveness.
      • Feel free to deviate from the rules, methodologies, templates, and other restrictions if the effect of such a deviation exceeds possible losses (Special cases aren't special enough to break the rules, although practicality beats purity)
      • If necessary, make a working, but perhaps not the best, solution immediately, and later improve it (Now is better than never, although never is often better than * right * now)
      • If it is difficult to choose between two alternative technical solutions, then you need to choose any and move on with it when more information appears, then you can refactor if the solution is not optimal
      • Flexibility of technical solutions is highly desirable, and versatility is optional
    • Source Code Quality
      • The quality of the code should be optimized on the basis of the formed system of criteria, balanced with respect to costs in the short and long term
      • Write the best code right away if that doesn't increase its complexity and development time (Beautiful is better than ugly)
      • Self-documenting code takes precedence over well-commented (Beautiful is better than ugly)
      • Write TODO and FIXME in code
      • Giving names, variables, functions, methods, classes, and other objects of the source code exactly their purpose, despite the increase in the length of the names (Explicit is better than implicit)
      • Fewer lines and a lot of code is preferable, while maintaining the same code readability (Simple is better than complex)
      • Use code review as a tool for detecting errors, leveling out the development style, getting to know someone else’s code and team training
      • Reuse your own and others code
      • Use specialized libraries to solve specific problems, instead of developing your own similar code
    • Performance
      • Code development performance takes precedence over code execution performance.
      • Performance optimization of code execution should be justified by the corresponding need.
      • Code execution performance optimization should be performed by removing the most serious bottlenecks.
      • First of all, the most effective methods for optimizing the performance of code execution should be used.

    The following is a detailed explanation of each of these principles. For some principles, Zen of Python postulates are indicated in parentheses, which in my opinion are related to these principles, or parts thereof.

    Making decisions


    Any technical solution must be justified.
    Any technical solution should be justified: starting from the used syntactic constructions in the code and ending with the architectural solutions of the highest level. For me, the validity of a technical solution means the presence of arguments confirming its advantages, the inevitability of such a solution, or other expediency of this solution. The justification should be accompanied by the availability of knowledge about the shortcomings of this decision and a plan to overcome them, or a justification for their non-materiality. In addition, alternative solutions should be considered with justification for their lower attractiveness compared to the selected solution. The justification should take into account both technical and organizational aspects related to a specific technical solution. If different arguments conflict with each other,

    In non-technical, including organizational, issues, this principle should be applied similarly.

    This principle is opposed to the routine decision-making, as well as the rash adoption of the first decision made without proper analysis of the alternatives.

    Responsibility for the decision always lies with one or those who made this decision
    This principle is aimed at improving the quality of decisions and is opposed to unsystematic decision-making.

    When making technical decisions, it is necessary to take into account their effect in time and their relevance to business needs.
    It should be taken into account what impact this or that decision will have both in the short and long term, and whether this technical solution is useful for business in these periods.

    This principle is opposed to one-sided decision-making with the concentration of results only in the short term, or only in the long term, and also without taking into account the needs of the business and taking into account the impact on business of these decisions, concentrating solely on the technical advantages of the solution.

    One of the main criteria when making technical and other decisions should be their greatest effectiveness.
    By efficiency in this matter, I understand the ratio of the value of the result for business to the amount of costs required for this.

    This principle is opposed to an approach that takes into account only the value of the result obtained regardless of the required costs.

    Feel free to deviate from the rules, methodologies, templates, and other restrictions if the effect of such a deviation exceeds possible losses (Special cases aren't special enough to break the rules, although practicality beats purity)
    Rules, methodologies, templates and other restrictions are a generalization of previous experience in order to obtain a stable result of activity in certain conditions. However, in specific cases, the conditions may differ, therefore, it is necessary to consciously approach the use of accumulated experience and make your own corrections if necessary.

    This principle is opposed to the dogmatic following of the rules, methodologies, templates and other accepted restrictions without taking into account the surrounding reality.

    If necessary, you should make a working, but perhaps not the best, solution immediately, and later improve it (Now is better than never, although never is often better than * right * now)
    There are situations in which it is necessary to quickly implement a certain functionality or fix a defect, and at the same time there are alternative solutions: fast and good. The quick effect can be obtained immediately, but in the long run it can create difficulties and lead to additional costs. A good decision to make is long and costly in the short term, but in the long run it will pay off and bring much more benefit than required. In this case, a combination of both solutions will be effective, namely: we make a quick decision now and in a planned manner we make a good decision. In total, this will turn out to be slightly more costly than the implementation of only one of the solutions, but the effect of their combination will pay back the costs.

    This principle is opposed to trying to do everything at once perfectly.

    If it is difficult to choose between two alternative technical solutions, then you need to choose any and move on with it when more information appears, then you can refactor if the solution is not optimal
    No comments

    Flexibility of technical solutions is highly desirable, and versatility is optional
    Flexibility is the ability to adapt a technical solution to new conditions that may appear in the future. Universality is the actual adaptation of a technical solution to the possibility of using it for a set of conditions. Often, ensuring universality requires more labor than providing flexibility, since it takes into account a greater number of conditions with which the technical solution must be compatible. Also, ensuring versatility can be ineffective, as it requires adaptation to conditions that have not yet arrived and may never come. In addition, sometimes unification requires complicated generalizations. Therefore, the flexibility of technical solutions is highly desirable, and versatility is not required.

    This principle is opposed to an attempt to immediately make universal decisions, which subsequently turn out to be unclaimed due to changes in external conditions.

    Source Code Quality


    The quality of the code should be optimized on the basis of the formed system of criteria, balanced with respect to costs in the short and long term
    My basic prioritized set of optimization criteria is as follows (if necessary, it should be adjusted to specific conditions):
    1. High readability of the code - the easier it is to understand the code, the better (Readability counts)
    2. Low cyclomatic code complexity - the fewer branches, loops, conditional operations, function and method calls, the better (Simple is better than complex. Flat is better than nested)
    3. High code flexibility - the easier it is to make changes to the code, the better (this optimization criterion increases development costs in the short term, but reduces them in the long term)
    4. Small code - the less code, the better (Simple is better than complex)
    5. Code Execution Performance - The faster the code is executed, the better

    The prioritization of the criteria is as follows: first, first of all, labor costs are spent on providing criteria with a higher priority; secondly, if the criteria conflict with each other, then a solution is applied that meets the criterion with a higher priority.

    This principle is opposed to the choatic optimization of the quality of the code, as well as to the use of optimization criteria and their priorities that are not agreed between members of the development team.

    Write the best code right away if that doesn't increase its complexity and development time (Beautiful is better than ugly)
    Sometimes there are several functionally equivalent code constructs, but having different performance. As a rule, it is enough to find out once which design has more performance or memory requirements and use it in all similar cases. However, it should be borne in mind that some such constructions can degrade the readability of the code and increase the development time both at the time of writing the code and with its support and modification in the future. This should be taken into account in order to avoid the disadvantages of premature optimization.

    Self-documenting code takes precedence over well-commented (Beautiful is better than ugly)
    Sometimes comments help to understand complex code, but at the same time they are an indicator of low code readability. Therefore, in cases where there is a need to write a comment, you should consider what changes you can make to the code so that the need for comments disappears.

    Here are typical, in my opinion, cases where you can do without a comment:
    • The commentary clarifies the meaning of the data contained in the variables - it is corrected by renaming the variable (similarly applies to the names of classes, methods, functions, constants, etc.)
    • Highlighting a code block using a comment - corrected by taking the code block to a separate function or method
    • The explanation of the performed action at the level of language semantics is corrected by deleting the comment (example of a comment to be deleted: a = 10 # assign 10 to a)

    Cases when comments should be written:
    • Public API Documentation
    • Explanation of non-obvious reasons, the choice of an alternative technical solution
    • Explanation of an optimized and, as a result, difficult to understand code section
    • TODO and FIXME - microplanning in the context of source code

    Comments can not only help, but interfere with development because they require additional writing time, take up space, reducing the number of lines of code that are simultaneously visible on the screen, require synchronization when changing the code, and take extra time to read. Therefore, commenting should be reasonable.

    This principle is opposed to the excessive use of comments.

    Write TODO and FIXME in code
    Not everything and it does not always make sense to do at the time of writing a particular section of code, especially when it comes to refactoring and making changes, the relevance of which is a question. In such cases, it is wise to write a TODO or FIXME comment in the code that you can work on at a more appropriate time. It is advisable to immediately assign an approximate priority to this comment in order to simplify planning in the future.

    This principle is opposed to writing the perfect code without taking into account the real needs and costs of such code at a particular point in time.

    Giving names, variables, functions, methods, classes, and other objects of the source code exactly their purpose, despite the increase in the length of the names (Explicit is better than implicit)
    Saving the length of the names cannot be done to the detriment of the comprehensibility of the code.

    Fewer lines and a lot of code is preferable, while maintaining the same code readability (Simple is better than complex)
    There are studies from which it follows that the number of errors in the code on average is directly proportional to the length of the code and the number of lines of code in particular. This is explained as follows: firstly, the smaller part of the code is placed simultaneously on the screen, the more effort is required to memorize sections of the code that are currently not visible during development, and secondly, the longer the code, the more typos can be made in it and other human factor errors.

    Use code review as a tool for detecting errors, leveling out the development style, getting to know someone else’s code and team training
    In addition to its immediate purpose, code inspection is excellent for leveling out the development style, getting to know other people's code and team training. In the ideal case, only code that has passed code inspection should get into the general branch.

    Reuse your own and others code
    This principle follows from the DRY principle. In order to successfully apply this principle, it is necessary to have a good knowledge of the code base, and the use of code inspection by all members of the development team helps in this.

    Use specialized libraries to solve specific problems, instead of developing your own similar code
    If you need to solve a problem (for example, send email via smtp), you should familiarize yourself with the availability of ready-made libraries for this task and study the possibilities of their use. Refusal to use specialized solutions should be justified, since specialized solutions have several advantages over the new development.

    Specialized solutions have already been developed and tested. This means saving time on the development, testing and correction of defects. Sometimes it seems that it is easier to develop your own code that performs the necessary function, because to use a ready-made solution you need to spend time studying it and there is a risk that it will be wasted because it turns out that the solution is not suitable. However, such an impression is often misleading, firstly, due to an underestimation of the complexity of the task, and secondly, due to the presence of other advantages.

    A feature of specialized solutions is that they are specially designed to solve a specific problem. This means that they take into account features that are difficult to take into account when solving a specific task along the way with the main one. It should be understood that in addition to functionality, the solution must also satisfy non-functional requirements, such as requirements for performance, security, ease of maintenance, compatibility, and others. All these features require a global immersion in the task and, if developed, may be overlooked due to limited time.

    Often specialized solutions are public and therefore used by many developers. This means that they have already identified and eliminated the shortcomings that they encountered during practical use. Thus, specialized solutions allow you to use someone else's experience without additional labor.

    Public specialized solutions often have open source code. In fact, these are most Python libraries. Such solutions are constantly being developed by the developer community, so it becomes possible to improve the quality of their software without additional labor costs by updating the versions used.

    Often, various public solutions are easily integrated with each other (for example, Spyne and SQLAlchemy), which can significantly reduce development costs.

    There are a number of good reasons for not using a specialized solution. Examples of such reasons:
    • The specialized solution lacks the necessary functionality
    • The specialized solution is poorly documented, and access to the source code is impossible or limited, or the source code is very difficult to use instead of documentation
    • A specialized solution contains serious errors and is poorly supported, or there is no access to the source code or the possibility of independent error correction
    • Specialized solution officially discontinued
    • The current needs for the functionality of a specialized solution make up an extremely small fraction of all the functionality embedded in a specialized solution

    Performance


    Code development performance takes precedence over code execution performance.
    In modern conditions, the cost of human resources is usually higher than the cost of hardware, so for me the productivity of code development (expressed in the amount of developed functionality per unit time) takes precedence over the performance of code execution.

    This principle is opposed to premature optimization of code execution performance.

    Performance optimization of code execution should be justified by the corresponding need.
    The discussion consists in the formation of performance requirements, which are based on the practical or planned scale of the use of the information system, and the subsequent identification of the inconsistency of the information system with these requirements.

    This principle is opposed to premature optimization of code execution performance.

    Code execution performance optimization should be performed by removing the most serious bottlenecks.
    To do this, it is necessary to identify bottlenecks, assess the contribution of each bottleneck to lower productivity and optimize the bottleneck in the first place.

    This principle is opposed to the chaotic optimization of code execution performance.

    First of all, the most effective methods for optimizing the performance of code execution should be used.
    The effectiveness of optimization methods is the ratio of the share of productivity growth to the total cost of optimization work. For me, the following prioritized list of optimization methods is basic:
    1. Changing the data processing algorithm (can give an increase in productivity up to several orders of magnitude at relatively low development costs)
    2. Increase in hardware capacity (can give an increase in performance from tens of percent to several tens of times depending on the nature of performance degradation; this method may require changes to the architecture of the information system to make scaling possible)
    3. Updating the finished application and system software used in the information system (assumes low costs, with the exception of paid software, but this rarely gives a significant performance gain, since application and system software are usually initially optimized)
    4. Changing the development language in terms of bottlenecks, or a complete transition to a more productive programming language (high development costs with an increase in productivity from tens of percent to several tens of times depending on the nature of the functionality of the information system and the programming languages ​​used)

    The prioritization of the methods consists in the fact that, first of all, labor and other resources are spent on the use of higher priority methods, and the transition to the next method is carried out when the optimization possibilities are exhausted, or the effectiveness of a lower priority method is higher.

    This principle is opposed to the chaotic optimization of code execution performance.

    Conclusion

    I deliberately excluded some of the principles from the publication, as they relate more to managing people, communication between people and building development processes, than directly to writing the source code. I simply could not identify or formulate well some principles, since the translation of unconscious knowledge into conscious knowledge is a rather complicated process. Therefore, I have high hopes that the comments on the article will turn out to be no less interesting than the article itself and will allow me to write a sequel that will include lost information.

    Also popular now: