We control the quality of the code using the SonarQube platform

In this article, we will consider the main features of SonarQube - a platform for continuous analysis and measurement of code quality, as well as discuss the advantages of code quality assessment techniques based on SonarQube metrics.

Introduction

Our company is developing a PVS-Studio static code analyzer. We are convinced that the selection of an appropriate development methodology and following it, the application of proven methods and tools significantly increase the likelihood that the code will be written correctly and the final product will meet the required quality standards. One such methodology is the use of static source code analysis. Static analysis, used together with other code metrics, allows you to evaluate the current state of the code base, the dynamics of this state, and possible risks during the implementation of the project.

First, I’ll briefly talk about the presentation of PVS-Studio analysis results. The results of our PVS-Studio static analyzer are saved in xml format. The list of messages about detected errors can be opened directly in the Microsoft Visual Studio window or in the standalone utility, and work with these errors using the code navigation, sorting, filtering, suppressing false alarms, etc.

The list of errors found in xml format can be converted into one of the formats that are easy to read, for example, html. Such reports with the results of the analysis can be sent to all interested project participants by mail. Another way to notify project participants is to send error lists to those developers who made them. To do this, users can use special utilities provided in the PVS-Studio distribution kit.

Speaking about the capabilities of PVS-Studio, we should mention the functionality for mass suppression of diagnostic messages. If static analysis was introduced in the later stages of the application life cycle when the code base size became large, code inspection can lead to the detection of a large number of errors. Perhaps at this point in time the team does not have the resources necessary to fix all the errors. In this case, developers can hide all messages issued on the current revision of the code and concentrate on errors found in newly written or modified code.

The dynamics of the number of errors found by PVS-Studio can be obtained using the Analysis Statistics functionality of the PVS-Studio plug-in for Microsoft Visual Studio or the Standalone utility:

The above methods of presenting the results of our static analyzer, however, exist on their own and are not related to other code metrics, such as the number of lines of code, cyclomatic complexity, the number of errors per 1000 lines of code, the level of coverage of the code by unit tests, duplication, and so on. The report on the errors found after the next analysis and the number of these errors does not answer the questions: do we have a lot of mistakes or not enough? How is the dynamics of the number of errors associated with the growth of the code base? Is code quality improving or getting worse? And, probably, the most important question of managers: when will everything work (how long will it take to fix errors)? These questions made us think about how to make PVS-Studio reports more valuable and informative.

Why is it useful to collect and control code metrics? It is impossible to improve what you are not measuring. Suppose some team does not collect code metrics. With the development of the project, the code base can become worse and worse, and for a long time no one will even notice it until at some point the amount of technical debt reaches such a size that supporting and adding new functionality will cost more and more. If a team constantly monitored code metrics, they would see the dynamics of the state of their project and, upon reaching certain thresholds, would start to sound an alarm.

Further, let's say the team realized the problem and convinced their manager that it was necessary to spend resources on improving the quality of the code. The manager, in turn, will ask the team some simple questions: how long will it take for the team to do this? What parts of the project need to be improved? How many bugs are you going to fix? By what criteria do you assess that the quality of the code has reached the required level, and can you return to the development of new functionality? Using metrics will answer all these questions. If, before starting work on improving the quality of the code, the team will record the current state of the code for each component of the product and determine threshold values ​​for each metric, the achievement of which will allow us to state with a certain degree of confidence that the product code has become high-quality, you can predict the completion date of work and stop when certain threshold values ​​are reached. So, for example, the team will be able to show the manager that after completing work to improve the quality of the code, the level of coverage with modular tests reached 90%, adherence to the coding standards adopted by the company reached 95%, and duplication of code was reduced to 5%, etc.

Why SonarQube?

SonarQube is an open source platform designed for continuous analysis and measurement of code quality. SonarQube provides the following features:

• Support for Java, C, C ++, C #, Objective-C, Swift, PHP, JavaScript, Python, etc.
• Provides reports on code duplication, compliance with coding standards, code coverage by unit tests, possible errors in the code, density of comments in the code, technical debt and more.
• Saves the history of metrics and builds graphs of changes in these metrics over time.
• Provides fully automated analysis: integrates with Maven, Ant, Gradle and common continuous integration systems.
• Allows integration with IDEs such as Visual Studio, IntelliJ IDEA and Eclipse using the SonarLint plugin.
• Provides integration with external tools: JIRA, Mantis, LDAP, Fortify, etc.
• You can extend existing functionality with third-party plugins.
• Implements SQALE methodology for evaluating technical debt.

Impressive list, right? You can familiarize yourself with the capabilities of SonarQube and try it in action by clicking on the link https://sonarqube.com/ . SonarSource provides this service for analyzing open source projects, and if you have an open source project on GitHub, you can upload it to https://sonarqube.com/ and use SonarQube reports to monitor the quality of your project code.

We carefully examined the capabilities of the SonarQube platform and decided that these features could be of interest to our customers. Therefore, we decided to develop a plug-in for importing PVS-Studio analysis results.

Around the same time, one of our customers expressed interest in introducing a centralized repository of various code metrics. The client develops a very large (over 10 million lines of code) and long-term (over 15 years of active development) project. Naturally, in such a project there is a lot of inherited code and related skeletons in the closet, and in this case, in my opinion, it is absolutely necessary to develop and implement a set of metrics to assess the status of the project code and the dynamics of changes in this state over time. Naturally, our client made the decision to collect and analyze metrics a long time ago and introduced various utilities for monitoring code quality indicators, such as: covering code with unit tests, test run results, duplicating code blocks, following accepted coding standards, density of comments in code, etc. In parallel with the collection of these metrics, static code analysis was performed daily. The use of a large number of utilities complicated the configuration of the continuous integration server, writing additional scripts to convert the results of each utility into a convenient form for presenting and combining all indicators into a single report. This approach required significant resources to create and maintain this reporting system.

Based on the needs of the client and our research on the capabilities of the SonarQube platform, we proposed to introduce this platform. Which was done. As part of the implementation task, a plug-in for SonarQube was implemented, which allows importing PVS-Studio analysis results into SonarQube. The deployment process of SonarQube and its integration with the existing environment (assembly system, continuous integration server, version control system) did not cause difficulties due to logical configuration mechanisms and a large number of detailed documentation. Widgets were configured to evaluate both the state of the client’s portfolio of projects as a whole and the status of each project individually, Quality Profiles and Quality Gates were set up (I will talk about these SonarQube mechanisms below) according to the client’s needs, automatic task assignment to performers,

As a result of the implementation of SonarQube, the client received a centralized system for storing and displaying code metrics, which allows assessing and predicting project risks. The transition from individual tools to a centralized code quality control system not only simplifies the deployment and support of this system, but also allows you to make a quantum leap in the field of project management, providing all interested participants with tools to monitor the status of the project and make informed decisions. After the test operation, the client decided to include SonarQube in the existing ALM (Application Lifecycle Management) toolkit.

It should be noted that SonarQube, thanks to its wide integration capabilities with other tools, can easily become an integral part of your ALM framework. Using plugins in SonarQube, you can add support for Git, SVN, Mercurial, Team Foundation Version Control, ClearCase, configure authorization via LDAP, GitHub, Bitbucket, Azure Active Directory, import the results of third-party analyzers. SonarLint plugins for IntelliJ IDEA, Eclipse, and Visual Studio let you analyze the code in real time in your favorite IDE using the rules defined in your SonarQube profile. Integration with Team Foundation Server and Visual Studio Team Services is also available. You can run code analysis and data import into SonarQube directly from the assembly process on these systems, or, for example, manage the state of assemblies in Team Foundation Server and Visual Studio Team Services using Quality Gates (build quality indicators), which are configured in SonarQube: if the code does not meet the Quality Gate requirements, the assembly will be considered failed. Thus, SonarQube developers strive to make their product as open as possible and allow development teams to integrate SonarQube into their environment.

For what projects and teams it is advisable to implement SonarQube? I believe that for relatively short-term projects (no more than 2 - 3 months), which are involved in small teams (no more than 5 people), investments in introducing SonarQube into the development process may not be justified. As a rule, such projects do not require large expenses for product support. For such projects, I would recommend limiting ourselves to individual tools for monitoring the status of the project code: a static analyzer, monitoring of code coverage by tests, compliance with coding standards, etc., which the team is used to using.

On large projects requiring significant resources, with a long life cycle, the introduction of the SonarQube platform in the development process is justified. Moreover, the introduction of SonarQube can be beneficial at any stage of the development of the project. The optimal strategy, in my opinion, is the implementation of SonarQube in the early stages of the development cycle, which will allow the team to analyze quality control reports from the very beginning and be sure that the set standards of code quality are respected. Implementing SonarQube at a later stage of development could potentially cost a lot of money to improve code quality. So, for example, it may turn out that the static analysis has detected a large number of potential errors, there is a large amount of technical debt, the code is not covered by tests, public APIs are not documented, etc. However,

For example, a team may agree that if a piece of code is changed as part of the development of new functionality, all technical debt in this code will be eliminated. Investments in eliminating the deficiencies found will make it possible to further reduce the cost of supporting the product and developing new functionality. Also, if a product has been on the market for a long time, and despite the fact that a code quality inspection has revealed a large number of problems, the behavior in the production environment is quite stable and there are currently not enough resources to improve the quality of the code base, you can postpone this investment. SonarQube provides an opportunity to focus on the problems that have appeared in the new code. This functionality is similar to the functionality of mass message suppression in PVS-Studio.

How SonarQube helps evaluate code quality

The SonarQube quality model is based on the implementation of the SQALE (Software Quality Assessment based on Lifecycle Expectations) methodology with certain additions. As you know, the SQALE methodology focuses mainly on the complexity of maintainability and does not take into account the risks of the project. For example, if a critical security problem was discovered today in a project, strict adherence to the SQALE methodology obliges you to eliminate all existing problems with reliability, the possibility of changes, the testability, etc., and only then return to the new one. critical issue. In fact, if potential problems exist in the code for a long time and do not manifest themselves as custom bug reports, it is much more important to focus on fixing new bugs.

With this in mind, the SonarQube developers modified the quality model based on SQALE to focus on the following important points:

• The quality model should be as easy to use as possible.
• Bugs and vulnerabilities should not be lost among maintainability issues.
• The presence of serious bugs and vulnerabilities in the project should lead to the fact that Quality Gate requirements are not met
• Code support issues are important too and cannot be ignored.
• Calculation of the cost of troubleshooting (using the SQALE analysis model) is important and should be performed

The standard Quality Gate SonarQube uses the following metric values ​​to determine that the code successfully passed validation:

• 0 new bugs
• 0 new vulnerabilities
• technical debt ratio on the new code <= 5%
• coverage of a new code not lower than 80%

The Sonar team has identified seven deadly sins of developers that increase technical debt:

• Bugs and potential bugs
• Violation of coding standards
• Code duplication
• Insufficient unit test coverage
• Poor difficulty distribution
• Spaghetti design
• Not enough or too many comments

The SonarQube platform is designed to help combat these seven sins.

Let's take a closer look at the main features of SonarQube.

Main page

On the SonarQube main page, you see a list of projects added to the system, with brief statistics for each project: build version, number of lines of code, number of bugs, vulnerabilities, and signs of “code with a choke,” date of the last analysis:

The contents of the main page can be customized to your goals using a large set of built-in widgets that allow you to visualize the status of the code of projects in SonarQube.

Project metrics

To get more detailed information about the status of the project, we will go to the project metrics page:

Information about the following code metrics is provided here: Reliability, Security, Maintainability, Coverage, Duplications, Size (Code Base Size), Complexity, Documentation (Documentation) code) and Issues.

Turning to the Reliability metric, we get information about the total number of detected bugs and new bugs discovered during the last analysis, the code reliability rating on a scale from A to E, where E is the worst rating, indicating that at least one was found blocker bug, as well as the time required to eliminate all errors found:

The SonarQube platform allows you to analyze code metrics from top to bottom, from the project level as a whole to individual modules and files. So, for example, if you click on the Reliability Rating, you will see a list of project files sorted by increasing reliability rating. This will allow you to focus on the most problematic sections of the code:

Then you can go to the file with the source code and to specific sections of the code in which errors are detected:

This top-down navigation is also available for other metrics.

The Security metric page contains information on the total number of vulnerabilities, new vulnerabilities, security rating (also on a scale from A to E), and the time it will take to fix the vulnerabilities:

The Maintainability page contains information on the technical debt in the project:

Thanks to the “top to bottom” navigation, you can go to the list of files sorted by the number of code detections with a dash:

and then directly to the code that needs attention:

The Coverage page provides information about code coverage with tests:

The Duplications page contains information about duplicating code in a project:

With this metric, you can easily find duplicate lines, blocks of code, and even entire files:

The Size page contains information about the size of the project: the number of lines of code, expressions, functions, classes, files and directories:

The Complexity page provides information on the total cyclomatic complexity of the project, as well as on the average complexity of functions and files:

The Documentation page provides information on comments in the code: the ratio of lines with comments to the total number of lines in the project, the number of lines with comments, the number of public APIs and the level of documentation for public APIs:

The last tab in the project metrics section - Issues - contains the total number of problems found in the code (the sum of the number of bugs, vulnerabilities and code smells), as well as the distribution of problems by state: open, rediscovered, confirmed, false positives and won't fix:

Error and code navigation

After analyzing the code metrics, let's see how SonarQube allows you to work with problems found in the code. To do this, go to the Issues section:

Here are all the problems found in the code with wide filtering capabilities, which allows you to focus on the most important problems. It should be noted that SonarQube allows you to save filter settings in order to reuse them.

By double-clicking on the error message, you can go to the code in which the problem was found. A detailed description of the error and recommendations on how to fix it are also available:

Please also note that, thanks to integration with version control systems, it is clear who and when made changes to the code that triggered the analyzer:

Integration with version control systems also allows you to automatically assign bugs in SonarQube to those developers who allowed them. You can also assign bugs to developers manually, change their type (bug, vulnerability or code smell), severity, tags, add comments. For greater ease of use, the function of mass change of bugs is available:

Rules, Quality Profiles and Quality Gates

Rules, Diagnostic Rules, Quality Profiles, and Quality Gates are key concepts of the SonarQube platform. Each plugin for SonarQube that performs static code analysis contains a repository with a description of the diagnostic rules that this plugin implements. Violations of these rules are used to determine the technical debt in the code and calculate the time to fix problems. For ease of use, the rules are combined into Quality Profiles. By default, SonarQube creates a default quality profile for each supported language, but you can create your own quality profiles with the set of diagnostic rules that may be useful to you. For example, to analyze critical projects whose code quality requirements are the most stringent, you can define a quality profile,

Quality Gate is an indicator of compliance (or inconsistency) of the project code with the specified threshold metric values. By default, all projects added to SonarQube use a standard quality gate, which defines the following metrics and their threshold values:

• New bugs = 0
• New vulnerabilities = 0
• Technical debt ratio on the new code <= 5%
• New Code Coverage> = 80%

Based on your own requirements for the quality of the source code, you can change the standard quality gate or create a new one by adding or removing those metrics and their threshold values ​​that are of interest to you.

PVS-Studio and SonarQube

To import the analysis results into SonarQube, we developed the sonar-pvs-studio-plugin plugin. Using the plugin allows you to add messages found by the PVS-Studio analyzer to the message base of the SonarQube server. The plugin contains a repository with a description of the diagnostics that our static analyzer performs. After you add our plugin to SonarQube, you will see a repository called PVS-Studio for C, C ++ and C #:

PVS-Studio diagnostic messages in the plugin repository are accompanied by detailed error descriptions with code examples and troubleshooting tips:

After static analysis of the project code and importing the results into SonarQube, using filters you can, for example, select all the uncorrected problems found by PVS-Studio:

To add PVS-Studio analysis results to SonarQube, just install the sonar-pvs-studio-plugin plug-in, add the PVS-Studio diagnostics from the plug-in repository to Quality Profile and pass the path to the PVS-Studio report file in the sonar.pvs-studio.reportPath property when starting the SonarQube scanner.

SonarQube developers recommend using SonarQube Scanner for MSBuild to analyze MSBuild projects . This scanner is a wrapper over the standard SonarQube scanner and facilitates the process of creating the sonar-project.properties scanner configuration file by automatically adding modules (projects in the solution) to it and writing down the paths to the source files that need to be analyzed.

However, we came across important limitations of the SonarQube Scanner for MSBuild from our point of view.

Firstly, when analyzing C / C ++ projects, this scanner will add to the list of files for analysis only those files that are added to the ClCompile and ClInclude properties of the project .vcxproj project file. If, for example, the header file is not explicitly included in the project and is included in the code of one of the source files, this file will be ignored, and the results of the analysis of this file will not be available in SonarQube.

Secondly, SonarQube Scanner for MSBuild does not add source files located higher in the directory tree for analysis than the directory in which the project file is located. Messages for such files will also be missing from SonarQube.

Based on these limitations, we recommend using the standard SonarQube scanner to import PVS-Studio analysis results. Using this scanner involves creating the sonar-project.properties configuration file manually. Using and setting up the scanner is described in the article Analyzing with SonarQube Scanner .

By default, the SonarQube scanner indexes the source files for analysis located in the directory tree below the solution file (.sln) or the project (.vcxproj / .csproj). To analyze projects with a complex structure, where the source files can be higher in the directory tree than the solution or project file, in the sonar.projectBaseDir property, you need to specify the highest common directory for all source files (in extreme cases, this may be the root of the disk), and in the sonar.sources property, then list the directories in which to search for source files for analysis (or full paths to source files).

The process of adding paths to the source files to the sonar.sources property for large projects can be quite time-consuming, and with all connected header files included, it can be non-trivial. To facilitate this task, we implemented a special operating mode of our analyzer, which allows you to automatically create configuration files for the SonarQube scanner.

When developing our static analyzer, we focus on finding errors in the code, and do not support the search for potential vulnerabilities and code smells, so when using our plugin for SonarQube, the Security and Maintainability metrics will not be filled. It should also be noted that the current version of our plugin does not implement the calculation of Duplications, Complexity and Documentation.

Conclusion

In this review, I tried to show how SonarQube allows you to implement and apply metrics-based code quality control practices. As Peter Drucker (or perhaps it was William Deming) said: if you can't measure it, you can't improve it. Regular collection of code metrics, analysis of their changes over time allows us to detect that technical debt is accumulating, code support is reduced, which leads to an increase in the cost of developing new functionality and an increase in the risks associated with the delivery of new versions of the product. I will quote from the book “Programmer-pragmatist. The journey from apprentice to master "Andrew Hunt and David Thomas:

"Do not leave “broken windows” (unsuccessful constructions, incorrect decisions or poor-quality program text) without attention. As soon as you find them, repair right away. If there is no time for proper repair, clog the window with boards. Surely you will be able to comment on the erroneous fragment or display the message “Under Development” or use dummy data. It is necessary to take at least the slightest action to prevent further destruction, and to show that you are in control of the situation.

We saw error-free, functional systems quickly deteriorate as soon as windows began to break. There are other factors that contribute to the corruption of programs, and we will touch on some of them further, but negligence accelerates corruption faster than any other factor.

You might think that no one will have time to get around the “broken windows” of the project and repair them. If you continue to think in this way, then you better plan to purchase a garbage container or move to another area of ​​the city. Do not let entropy defeat yourself. "

The SonarQube platform allows you to detect such “broken windows”. Using SonarQube, you will get access to consolidated reports on the status of the project code, watch how this state changes over time, you will be able to investigate the problems detected by the top-down analysis, from the value of the metric you are interested in for the whole project to a specific code section in the browser window. SonarQube also provides an estimate of the time it takes to troubleshoot code. Thus, if a large amount of technical debt has accumulated in your project, you can always tell how much time you need to eliminate it.

Of the interesting and useful features of SonarQube, I also want to note its wide capabilities for integration with other tools, which makes it part of your ALM framework, and the ability to expand existing functionality through the use of third-party plug-ins. And all this power and convenience are available under a free license for free. If you use the PVS-Studio static analyzer, our plugin will allow you to import the analysis results into SonarQube in order to use its capabilities to investigate problems with code quality.

If you are interested in the capabilities of the SonarQube platform, try it in your environment. Installing SonarQube is very simple and consists only in extracting the distribution files from the archive and running the executable file for your operating system.

You can also check your projects written in C / C ++ or C # using the PVS-Studio static analyzer .

useful links

• SonarQube Website
• SonarQube Documentation
• Online instance of SonarQube
• Plugins for SonarQube
• Integration of PVS-Studio analysis results into SonarQube

If you want to share this article with an English-speaking audience, then please use the link to the translation: Pavel Kusnetsov. Control source code quality using the SonarQube platform .