CI / CD patterns and anti-patterns. Part 2

Good afternoon. Today we are sharing with you the translation of the second part of the article “CI / CD Patterns and Anti-Patterns” , dedicated to the launch of a new stream on the course “DevOps Practices and Tools” . The first part of this article can be read here .

1.3 Patterns and antipatterns in testing

1.3.1 Test automation

• Pattern: Automate software validation and validation by including testing units, components, capacity, functionality, and deployment.
• Anti-patterns: Manual testing of units, components, deployment, etc.
• Unit- Automation of tests without dependencies.
• Component- Automation of tests with dependencies on other components, databases and file systems.
• Deployment - Automate tests to verify successful deployment and configuration. This is sometimes called “smoke” testing.
• Functional- Automation of tests to verify the behavior of software from the point of view of the user.
• Capacity- Automation of load and performance testing in conditions close to operational.

1.3.2 Isolation of Test Data

• Pattern: Use transactions for database-dependent tests (e.g. component testing) and roll back the transactions when completed. Use a small subset of data to test behavior effectively.
• Anti-patterns: Using a copy of production data for Commit Stage tests. Running tests on a common database.

1.3.3 Parallel Tests

• Pattern: In parallel, run several tests on hardware instances to reduce the time spent.
• Anti-patterns: Running tests on a single machine or instance. Running dependent tests that cannot be run in parallel.

1.3.4 System Stability

• Pattern: Use stubs to simulate external systems to reduce deployment complexity.
• Anti-patterns: Manual installation and configuration of interdependent systems for Commit Stage build and deployment.

1.3.5 End-to-End Testing Considered Harmful

Continuous Delivery is a set of holistic principles and practices aimed at reducing time to market. It is based on fast and reliable feedback thanks to tests. Continuous Delivery requires any changes to the code, configuration, data or infrastructure to pass a set of automated and exploratory tests in the Deployment Pipeline to assess operational readiness. Therefore, if the organization wants to achieve shorter deadlines, the test execution time should be low and the test results unequivocal.

For example, consider the Company Payments service, in which payments at the end of the year are sent to the subsequent Payments service.

The behavior of the Company Payment service can be checked during the assembly time by performing the following types of automatic tests:

• Unit tests: comparing the goal and implementation when checking individual code modules.
• Acceptance tests: comparison of implementation and requirements when checking the functional part of the system.
• End-to-end tests: comparison of implementation and requirements when checking the functional part of the system, including ownerless dependent services.

While unit and acceptance tests differ in purpose and scope, acceptance and end-to-end tests differ in volume only. Acceptance tests do not include orphan dependent services, therefore the acceptance test of the travel of the user of the Company Payments will use the Test System , consisting of the code of the latest version of the Company Payments and the Stub of Payments.

End-to-end tests include ownerless dependent services, therefore, the end-to-end travel test of the user of the Company Payments will use the Test System consisting of the most recent code of the Company Payments and the working version of the Payments.

If the testing strategy is compatible with Continuous Delivery, then it should have an appropriate ratio of unit, acceptance and end-to-end tests that balances the need for information and quick, unambiguous feedback. If testing does not bring new information, then defects go unnoticed. But if testing takes too much time, delivery will be slow and lost revenue will increase.

The end of the second part.

According to the established tradition, we are waiting for your comments and we invite you to an open day . The third part of the article is already available here .