SonarCloud is a cloud-based code quality and security service. The main features of SonarCloud are:
- 23 languages: Java, JS, C#, C/C++, Objective-C, TypeScript, Python, ABAP, PLSQL, T-SQL and more.
- Thousands of rules to track down hard-to-find bugs and quality issues thanks to powerful static code analyzers.
- Cloud CI Integrations, with Travis, Azure DevOps, BitBucket, AppVeyor and more.
- Deep code analysis, to explore all source files, whether in branches or pull requests, to reach a green Quality Gate and promote the build.
- Fast and Scalable
What’s covered in this lab
In this lab, you will learn how to integrate Azure DevOps Services with SonarCloud:
- Setup an Azure DevOps project and CI build to integrate with SonarCloud
- Analyze SonarCloud reports
- Integrate static analysis into the Azure DevOps pull request process
Prerequisites for the lab
You will need a Microsoft account.
Using this account, sign in to Azure DevOps Services.
Create a new Azure DevOps project for this lab:
Every project in Azure DevOps belongs to an organization. You will be placed into an automatically created default organization on sign in, the name of which is based on your user name (in our example, the user Claudia Sonarova has been given the organization claudiasonarova).
Inside this organization, create a project called SonarExamples:
Unless you intend to sign up for a paid plan with SonarCloud (see below), make sure that you set your Azure DevOps project to be public. If you do intend to sign up for a paid plan, then you can use a private or public project.
Import the Sonar Scanning Examples repository from GitHub at https://github.com/SonarSource/sonar-scanning-examples.git.
Go to Repos in your project overview:
Then, click Import:
See here for detailed instructions on importing a repository.
The scanning examples repository contains sample projects for a number of build systems and languages including C# with MSBuild, and Maven and Gradle with Java.
Install the SonarCloud Azure DevOps extension in your Azure DevOps account
Navigate to the SonarCloud extension in the Visual Studio Marketplace and click Get it free to install it.
If you do not have the appropriate permissions to install an extension from the marketplace, a request will be sent to the account administrator to ask them to approve the installation.
The SonarCloud extension contains build tasks, build templates and a custom dashboard widget.
Using the same account as you used for Azure Devops, sign into SonarCloud.
In SonarCloud, create an organization and, within that, a new project. The organization and project you set up in SonarCloud will mirror the organization and project that you set up in Azure DevOps.
Once you sign in, click Create an organization on the welcome page:
Choose an organization key. Note that the key must be unique within the SonarCloud system. In our example, we will use
Make sure that the the green checkmark appears beside the Key field. This indicates that the key is unique across SonarCloud.
Next, you can choose your plan. For this example, we choose a free plan (that is, one for public repos only), but you can choose a paid plan if you intend to have private repos:
You have now created the SonarCloud organization that will mirror your Azure DevOps organization.
The next step is to create, within that organization, the SonarCloud project that will mirror the Azure DevOps project SonarExamples. As with the SonarCloud organization, the key of your project must be unique across the SonarCloud system. In our example we use
Make sure that the checkmarks are green, indicating that your key is unique and your name is valid (the name can be different from the key and need not be unique. Here we leave it the same as the key).
Click Set Up and you will see the following:
Following the link will lead you to this document.
The next step is to set up the Azure DevOps integration with SonarCloud.
Exercise 1: Set up a pipeline that integrates with SonarCloud
We will set up a new build pipeline that integrates with SonarCloud to analyze the SonarExamples code.
- In your new Azure DevOps project, go to Pipelines under the Pipelines tab, then click on Create Pipeline:
Here you have two options. You can can configure the pipeline with either the YAML editor, or with the classic editor.
With the classic editor, you can take advantage of the pre-defined templates that were installed as part of the SonarCloud Extension, above. With the YAML editor you need to use a separately provided YAML file (link provided below).
Select the location of your code. Azure DevOps lets you build and release code located not only in this Azure DevOps account but also in other, external, repositories. We will not be using this feature today. In our case we want to analyze code in the git repo that we imported earlier, right in the same account as this pipeline. So, we select Azure Repos Git:
On the next screen select the git repository that you imported earlier, SonarExamples:
Now select a YAML file template. We will be building and analyzing the .NET code in our example imported repository, so we will start by choosing the .NET Desktop YAML template:
The YAML editor will open with the template YAML file. In order to configure it correctly you will need to adjust it (or replace it) so that it looks like the following example file:
When you are done making the changes to the file, click Save and Run:
To configure the pipeline using the classic editor, select Use the classic editor on the Where is your code? page:
Select Azure Repos Git, choose SonarExamples under Repository and master under Default branch…, then click Continue:
The SonarCloud extension installed earlier provides SonarCloud-enabled custom build templates for Maven, Gradle, .NET Core and .NET Desktop applications. The templates are based on the standard Azure DevOps templates but with additional analysis-specific tasks and some pre-configured settings.
Select the .NET Desktop with SonarCloud template.
The template contains all of the necessary tasks and most of the required settings. However, You will need to provide a few additional settings.
Select Azure Pipelines in Agent pool, then choose vs2017-win2016 in Agent Specification:
Get the SonarCloud endpoint token. This is a token generated by SonarCloud that identifies your account on that system and allows other services, in this case, Azure DevOps, to connect to that account.
Go back to SonarCloud (if you are not logged in, log back in with the same account that you used at the start of this lab) and click My Account in the top right dropdown:
On the My Account page go to the Security tab. Here you can generate the endpoint token. Give the the token a name (like
azure_devops) and click Generate:
SonarCloud will generate a token. Copy the token immediately to the clipboard by clicking Copy, go back to Azure DevOps.Click the Prepare analysis on SonarCloud task:
Then click on the + New button:
Paste the endpoint token that you copied in to the field SonarCloud Token, give the endpoint a name and click Verify and save:
If the token was correctly copied, Azure DevOps should connect with your SonarCloud account and you should be able to click on the Organization drop-down and see the organization you created earlier and choose it from the dropdown (in our case
Now enter the key of the project that you created within that organization on SonarCloud (in our case
[Optional] Enable the Publish Quality Gate Result step
This step is not required unless you want to use the pre-deployment gate along with Release Pipelines. If this step is enabled, a summary of the analysis results will appear on the Extensions tab of the Build Summary page. However, this will delay the completion of the build until the processing on SonarCloud has finished.
Save and queue the build. Once the build has completed you should see something like this:
If you enabled the Publish Quality Gate Result step above the Build Summary will contain a summary of the analysis report.
- Either click on the Detailed SonarCloud Report link in the build summary to open the project in SonarCloud, or browse to SonarCloud and view the project.
We have now created a new organization on SonarCloud and configured an Azure DevOps build to perform analysis and push the results of the build to SonarCloud.
Exercise 2: Analyze SonarCloud Reports
Open the SonarExamples project in the SonarCloud Dashboard. Under Bugs and Vulnerabilities, we can see a bug has been found.
The page has other metrics such as Code Smells, Coverage, Duplications and Size. The following table briefly explains each of these terms.
|Bugs||An issue that represents something wrong in the code. If this has not broken yet, it will, and probably at the worst possible moment. This needs to be fixed|
|Vulnerabilities||A security-related issue which represents a potential backdoor for attackers|
|Code Smells||A maintainability-related issue in the code. Leaving it as-is means that at best maintainers will have a harder time than they should make changes to the code. At worst, they’ll be so confused by the state of the code that they’ll introduce additional errors as they make changes|
|Coverage||To determine what proportion of your project’s code is actually being tested by tests such as unit tests, code coverage is used. To guard effectively against bugs, these tests should exercise or ‘cover’ a large proportion of your code|
|Duplications||The duplications decoration shows which parts of the source code are duplicated|
|Size||Provides the count of lines of code within the project including the number of statements, Functions, Classes, Files and Directories|
Click on the Bugs count to see the details of the bug. The Issues page will appear:
Click on the bug to navigate to the code. You will see the error in line number 9 of Program.cs file as Change this condition so that it does not always evaluate to ‘true’; some subsequent code is never executed.:
We can also see which lines of code are not covered by tests:
Our sample project is very small and has no historical data. However, there are thousands of public projects on SonarCloud that have more interesting and realistic results.
[DONE TO HERE]
Exercise 3: Set up pull request integration
Configuring SonarCloud analysis to run when a pull request is created involves two steps:
- A SonarCloud project needs to be provided with an access token so it can add PR comments to Azure DevOps, and
- Branch Policy needs to be configured in Azure DevOps to trigger the PR build
Create a Personal Access Token in Azure DevOps.
Follow the instructions in this article to create a token with Code (read and write) scope.
SonarCloud will post comments to the pull request as if it is a user who owns the personal access token. The recommended practice is to create a separate “bot” Azure DevOps user for this so that it is clear which comments are from real developers and which are from SonarCloud.
Make sure to set the scope under Code to Read & write:
Click Create and copy the generated token:
You should treat Personal Access Tokens like passwords. It is recommended that you save them somewhere safe so that you can re-use them for future requests.
Configure SonarCloud to analyze pull requests
- Browse to the Sonar Examples dashboard in SonarCloud
Click on Administration > General Settings
- Select the Pull Requests tab
- Set the Provider drop-down to Azure DevOps Services and click Save
Paste in the Personal access token you copied earlier and click Save
Configure the branch policy for the project in Azure DevOps
- Navigate to the SonarExamples project in Azure DevOps
- Click on Repos, Branches to view the list of branches
Click on the settings link (“…”) for master and select Branch policies
Click the + beside Build Validation to add a new build policy:
- Select the build definition we created earlier from the Build definition drop-down
- Set the Display name to SonarCloud analysis
Azure DevOps is now configured to trigger a SonarCloud analysis when any pull request targeting the master branch is created.
Create a new pull request
Now we will make a change to a file and create a new request so that we can check that the pull request triggers an analysis.
- Navigate to the code file Program.cs at sonarqube-scanner-msbuild/CSharpProject/SomeConsoleApplication/Program.cs and click Edit
Add an empty method to the code as shown in the following screen shot, then click Commit…
In the dialog that appears,
- Change the branch name from master to branch1
- Check the Create a pull request checkbox
Click Commit, then click Create on the next screen to submit the pull request
If the pull request integration is correctly configured the UI will show that an analysis build is in progress.
Review the results of the Pull Request analysis
The results show that the analysis builds completed successfully, but that the new code in the PR failed the Code Quality check. Comment has been posted to the PR for the new issue that was discovered.
Note that the only issues in code that was changed or added in the pull request are reported - pre-existing issues in Program.cs and other files are ignored.
Block pull requests if the Code Quality check failed
At this point, it is still possible to complete the pull request and commit the changes even though the Code Quality check has failed. However, it is simple to configure Azure DevOps to block the commit unless the Code Quality check passes:
- Go to the Branch Policy page of the master branch (since the master branch is the one you will want your pull requests to merge into, this is where you have to adjust the policy).
Click Add status policy
- Select SonarCloud/quality gate from the Status to check drop-down
- Set the Policy requirement to Required
Users will now be unable to merge the pull request until the Code Quality check is successful (the Code Quality check succeeds when all issues have been either fixed or marked as confirmed or resolved in SonarCloud).
Exercise 4: Check the SonarCloud Quality Gate status in a Continuous Deployment scenario (In Preview)
Disclaimer: This feature is in preview, and may not reflect its final version. Please look at the notes at the end of this exercise for more information.
Starting from version 1.8.0 of the SonarCloud extension for Azure DevOps, a pre-deployment gate is available for your release pipeline. It allows you to check the status of the SonarCloud Quality Gate for the artifact you want to deploy and block the deployment if the Quality Gate failed.
- Enable the Publish Quality Gate Result in your build pipeline
- Have an artifact built and published in this pipeline to feed the release pipeline
- Click on Pipelines on the left pane, then click on Releases
Click on New Pipeline
On the template selection, choose the template you want. We will choose Empty job for this exercise.
- Close for now the Stage properties
Click on the pre-deployment conditions on Stage 1
- Click on Enabled beside Gates
Click on + Add, then select SonarCloud Quality Gate status check
In order to have the fastest result possible for this exercise, we recommend you to set the evaluations options as per this screenshot (See how Gate evaluation flow works, as per Microsoft Documentation)
- That’s it. You can close this panel.
- Click now on Add an artifact on the left.
- Currently, only build artifacts are supported. Choose the project and the source (build pipeline) of your artifact, its alias should match the name of the artifact published in the build pipeline.
Click on Add
- You can now save your pipeline.
- Go back to the build pipeline section, trigger a build of the pipeline that creates the artifact.
Once the build is completed and succeeded, you can create a new release by either clicking on the Release button on the build page or set up an automatic release trigger based on a branch filter.
- Go to the release by either clicking on the link if a manual release has been triggered from the build (on the top of the Build page), or going to the Releases page.
After few minutes (as set up on the point 8 of this exercise), your Quality Gate check should have been performed (at least twice to get a ‘go/nogo’ for the stage), and if it’s green, it should look like this:
Otherwise, if it’s failed, then read important notes below to find out how it happened and how to get a green Quality Gate.
Important notes about this feature
- The Publish Quality Gate Result task in your build pipeline has to be enabled in order for the release gate to work.
- If the Quality Gate is in the failed state, it will not be possible to get the pre-deployment gate passing as this status will remain in its initial state. You will have to execute another build with either the current issues corrected in SonarCloud or with another commit for fixing them.
- Please note also that current behavior of the pre-deployment gates in Release Pipelines is to check the status every 5 minutes, for a duration of 1 day by default. However, if a Quality Gate for a build has failed it will remain failed so there is no point in re-checking the status. Knowing this, you can set the timeout after which gates fail to a maximum of 6 minutes so the gate will be evaluated only twice as described above, or just cancel the release itself.
- Only the primary build artifact related Quality Gate of the release will be checked.
- During a build, if multiple analyses are performed, all of the related Quality Gates are checked. If one of them has the status either WARN, ERROR or NONE, then the Quality Gate status on the Release Pipeline will be failed.
With the SonarCloud extension for Azure DevOps Services, you can embed automated testing in your CI/CD pipeline to automate the measurement of your technical debt including code semantics, testing coverage, vulnerabilities. etc. You can also integrate the analysis into the Azure DevOps pull request process so that issues are discovered before they are merged.