Git Product home page Git Product logo

cnescatlab / sonar-scanner Goto Github PK

View Code? Open in Web Editor NEW
4.0 3.0 4.0 191 KB

Docker environment containing open source code analysis tools configured by CNES and dedicated to Continuous Integration.

Home Page: https://hub.docker.com/r/lequal/sonar-scanner

License: GNU General Public License v3.0

Dockerfile 14.31% Shell 5.50% Java 0.32% Fortran 3.08% Python 75.80% C 0.98%
sonarqube docker cnes software-factory ci

sonar-scanner's Introduction

CNES sonar-scanner-catlab image

Docker environment containing open source code analysis tools configured by CNES and dedicated to Continuous Integration.

This image is a pre-configured sonar-scanner image derived from Docker-CAT. It contains the same tools for code analysis and it is available on Docker Hub at lequal/sonar-scanner.

SonarQube itself is an opensource project on GitHub: SonarSource/sonarqube.

For versions and changelog: GitHub Releases.

ℹī¸ If you only need a containerized sonar-scanner, you better use the official image from SonarSource available on Docker Hub: sonarsource/sonar-scanner-cli. The official image is smaller because it does not embed any other tool.

Features

Compared to the official sonarsource/sonar-scanner-cli image, this image provides additional features.

Additional features are:

This image is made to be used in conjunction with a pre-configured SonarQube server image that embeds all necessary plugins and configuration: cnescatlab/sonarqube. It is, however, not mandatory to use it.

User guide

  1. Write a sonar-project.properties at the root of your project
  2. Execute the sonar-scanner on the project by running this image from the root of the project
    $ docker run \
            --rm \
            -u "$(id -u):$(id -g)" \
            -e SONAR_HOST_URL="url of your SonarQube instance" \
            -v "$(pwd):/usr/src" \
            lequal/sonar-scanner
    This docker command is equivalent to sonar-scanner -Dsonar.host.url="url of your SonarQube instance".
    • If the SonarQube server is running in a container on the same computer, you will need to connect both containers (server and client) to the same bridge so that they can communicate. To do so:
      $ docker network create -d bridge sonarbridge
      $ docker network connect sonarbridge "name of your sonarqube container"
      # add the following option to the command line when running the lequal/sonar-scanner
      --net sonarbridge

This image suffers from the same limitations as the official SonarQube sonarsource/sonar-scanner-cli image.

  • If you need to analyze .NET projects, you must use the SonarScanner for MSBuild.
  • If you want to save the sonar-scanner cache, you must create the directory to bind mount in the container before running it. For more information, see SonarQube documentation.

How to use embedded tools

Not only does this image provide a sonar-scanner, but also a set of open source code analysis tools. All available tools are listed below. They can be used from the image by changing the arguments of the container when running one.

# Example with shellcheck
$ docker run \
        --rm \
        -u "$(id -u):$(id -g)" \
        -v "$(pwd):/usr/src" \
        lequal/sonar-scanner \
        shellcheck --color always -s bash -f checkstyle my-script.bash
# where my-script.bash is a file in the current working directory

For information on how to use these tools, refer to their official documentation.

How to use embedded CNES pylintrc

There are 3 pylintrc embedded in the image under /opt/python:

  • pylintrc_RNC2015_A_B
  • pylintrc_RNC2015_C
  • pylintrc_RNC2015_D

To use one of these files when running pylint from within the container:

# pylint with a CNES pylintrc
$ docker run \
        --rm \
        -u "$(id -u):$(id -g)" \
        -v "$(pwd):/usr/src" \
        lequal/sonar-scanner \
        pylint --rcfile=/opt/python/pylintrc_RNC2015_A_B my-script.py
# where my-script.py is a python module in the current working directory

To import pylint results in SonarQube see the official documentation. (Summed up: Run pylint with the following template: pylint <module_or_package> --rcfile=<pylintrc> -r n --msg-template="{path}:{line}: [{msg_id}({symbol}), {obj}] {msg}" > pylint-report.txt. Activate at least one pylint rule in the Quality Profile the project uses for Python.)

How to use other pylintrcs

You may want to use the embedded pylint with a pylintrc of yours . In this case, the easiest way to do so is to put a pylintrc file along with the sources.

To then use it:

# pylint with a custom pylintrc
$ docker run \
        --rm \
        -u "$(id -u):$(id -g)" \
        -v "$(pwd):/usr/src" \
        lequal/sonar-scanner \
        pylint --rcfile=/usr/src/custom_pylintrc my-script.py
# where my-script.py is a python module in the current working directory
# and custom_pylintrc is a pylintrc in the current working directory

On the other hand, if you want to use a CNES pylintrc for your project you can download it directly from github. They are stored on this repository under pylintrc.d.

Examples usage in CI

This image was made for CI, hence here are some examples. Make sur to use the right URL for your SonarQube instance instead of my-sonarqube.com.

These examples still need to be tested.

Jenkins

Here are 2 examples of a declarative Jenkinsfile and a scripted Jenkinsfile that call this image in a stage to analyze a project.

// Declarative pipeline
def sonarqubeURL = 'https://my-sonarqube.com'

pipeline {
    agent any

    stages {
        stage('Sonar scan') {
            steps {
                sh  """
                    docker run --rm \
                        -u "\$(id -u):\$(id -g)" \
                        -e SONAR_HOST_URL="${sonarqubeURL}" \
                        -v "\$(pwd):/usr/src" \
                        lequal/sonar-scanner
                    """
            }
        }
    }
}
// Scripted pipeline
def sonarqubeURL = 'https://my-sonarqube.com'

node {
    checkout scm

    stage('Sonar scan') {
        sh  """
            docker run --rm \
                  -u "\$(id -u):\$(id -g)" \
                  -e SONAR_HOST_URL="${sonarqubeURL}" \
                  -v "\$(pwd):/usr/src" \
                  lequal/sonar-scanner
            """
    }
}

GitHub Actions

Here is a GitHub Actions job of a GitHub Actions workflow that call this image to analyze a project.

jobs:
  sonar-scanning:
    name: Run CNES sonar-scanner
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Cache sonar-scanner data
        uses: actions/cache@v2
        with:
          path: .sonarcache
          key: sonar-scanner-cache
      - run: |
          mkdir -p .sonarcache
          docker run --rm \
                    -u "$(id -u):$(id -g)" \
                    -e SONAR_HOST_URL="https://my-sonarqube.com" \
                    -v "$(pwd):/usr/src" \
                    -v ".sonarcache:/opt/sonar-scanner/.sonar/cache" \
                    lequal/sonar-scanner

Travis CI

Here is a Travis CI script step, in a .travis.yml, to analyze a project with this image.

cache:
  directories:
    - /home/travis/.sonarcache

script:
  - mkdir -p /home/travis/.sonarcache
  - docker run --rm \
    -u "$(id -u):$(id -g)" \
    -e SONAR_HOST_URL="https://my-sonarqube.com" \
    -v "$(pwd):/usr/src" \
    -v "/home/travis/.sonarcache:/opt/sonar-scanner/.sonar/cache" \
    lequal/sonar-scanner

GitLab-CI

Here is GitLab-CI job, in a .gitlab-ci.yml, to analyze a project with this image.

sonar-scanning:
  stage: test
  cache:
    key: sonar-scanner-job
    paths:
      - .sonarcache
  script:
    - mkdir -p .sonarcache
    - docker run --rm \
      -u "$(id -u):$(id -g)" \
      -e SONAR_HOST_URL="https://my-sonarqube.com" \
      -v "$(pwd):/usr/src" \
      -v ".sonarcache:/opt/sonar-scanner/.sonar/cache" \
      lequal/sonar-scanner

Analysis tools included

Tool Version Default report file
sonar-scanner 5.0.1.3006
ShellCheck 0.8.0
pylint 3.1.0 pylint-report.txt
CNES pylint extension 7.0.0
CppCheck 2.14.0 cppcheck-report.xml
Infer 1.1.0

Developer's guide

Note about branch naming: if a new feature needs modifications to be made both on the server image and this one, it is strongly advised to give the same name to the branches on both repositories because the CI workflow of this image will try to use the server image built from the same branch.

How to build the image

It is a normal docker image. Thus, it can be built with the following commands.

# from the root of the project
$ docker build -t lequal/sonar-scanner .

To then run a container with this image see the user guide.

To run the tests and create your own ones see the test documentation.

How to contribute

If you experienced a problem with the image please open an issue. Inside this issue please explain us how to reproduce this issue and paste the log.

If you want to do a PR, please put inside of it the reason of this pull request. If this pull request fixes an issue please insert the number of the issue or explain inside of the PR how to reproduce this issue.

All details are available in CONTRIBUTING.

Bugs and feature requests: issues

To contribute to the project, read this about CATLab's workflows for Docker images.

License

Licensed under the GNU General Public License, Version 3.0

This project is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.

sonar-scanner's People

Contributors

begarco avatar diegorodriguez31 avatar github-actions[bot] avatar louisjdmartin avatar topin2001 avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

sonar-scanner's Issues

Node.js unavailable

Describe the bug

During lequal/sonar-scanner execution, if .css files are found, sonar-scanner fails to launch 'node -v'

To reproduce

  1. create a project containing .css files
  2. declare the project in sonarqube instance
  3. export variables SONAR_HOST_URL, SONAR_TOKEN, SONAR_PROJECT_KEY
  4. run :
docker run --rm -u "$(id -u):$(id -g)" -e SONAR_HOST_URL="${SONAR_HOST_URL}" -v "$(pwd):/usr/src" -e SONAR_LOGIN="${SONAR_TOKEN}" -e SONAR_SCANNER_OPTS="-Dsonar.projectKey=${SONAR_PROJECT_KEY}" lequal/sonar-scanner

Expected behavior

Node.js shall be installed and java should successfully launch node -v when scanning *.css files.

Screenshots & log

INFO: Sensor CSS Rules [javascript]
ERROR: Error when running: 'node -v'. Is Node.js available during analysis?
org.sonarsource.nodejs.NodeCommandException: Error when running: 'node -v'. Is Node.js available during analysis?
        at org.sonarsource.nodejs.NodeCommand.start(NodeCommand.java:79)
        at org.sonarsource.nodejs.NodeCommandBuilderImpl.getVersion(NodeCommandBuilderImpl.java:203)
        at org.sonarsource.nodejs.NodeCommandBuilderImpl.checkNodeCompatibility(NodeCommandBuilderImpl.java:169)
        at org.sonarsource.nodejs.NodeCommandBuilderImpl.build(NodeCommandBuilderImpl.java:143)
        at org.sonar.plugins.javascript.eslint.EslintBridgeServerImpl.initNodeCommand(EslintBridgeServerImpl.java:201)
        at org.sonar.plugins.javascript.eslint.EslintBridgeServerImpl.startServer(EslintBridgeServerImpl.java:142)
        at org.sonar.plugins.javascript.eslint.EslintBridgeServerImpl.startServerLazily(EslintBridgeServerImpl.java:233)
        at org.sonar.plugins.javascript.eslint.AbstractEslintSensor.execute(AbstractEslintSensor.java:68)
        at org.sonar.plugins.javascript.eslint.CssRuleSensor.execute(CssRuleSensor.java:89)
        at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:64)
        at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:88)
        at org.sonar.scanner.sensor.ModuleSensorsExecutor.lambda$execute$1(ModuleSensorsExecutor.java:61)
        at org.sonar.scanner.sensor.ModuleSensorsExecutor.withModuleStrategy(ModuleSensorsExecutor.java:79)
        at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:61)
        at org.sonar.scanner.scan.SpringModuleScanContainer.doAfterStart(SpringModuleScanContainer.java:82)
        at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:188)
        at org.sonar.core.platform.SpringComponentContainer.execute(SpringComponentContainer.java:167)
        at org.sonar.scanner.scan.SpringProjectScanContainer.scan(SpringProjectScanContainer.java:403)
        at org.sonar.scanner.scan.SpringProjectScanContainer.scanRecursively(SpringProjectScanContainer.java:399)
        at org.sonar.scanner.scan.SpringProjectScanContainer.doAfterStart(SpringProjectScanContainer.java:368)
        at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:188)
        at org.sonar.core.platform.SpringComponentContainer.execute(SpringComponentContainer.java:167)
        at org.sonar.scanner.bootstrap.SpringGlobalContainer.doAfterStart(SpringGlobalContainer.java:137)
        at org.sonar.core.platform.SpringComponentContainer.startComponents(SpringComponentContainer.java:188)
        at org.sonar.core.platform.SpringComponentContainer.execute(SpringComponentContainer.java:167)
        at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:72)
        at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:66)
        at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
        at jdk.proxy1/jdk.proxy1.$Proxy0.execute(Unknown Source)
        at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
        at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
        at org.sonarsource.scanner.cli.Main.execute(Main.java:126)
        at org.sonarsource.scanner.cli.Main.execute(Main.java:81)
        at org.sonarsource.scanner.cli.Main.main(Main.java:62)
Caused by: java.io.IOException: Cannot run program "node": error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
        at org.sonarsource.nodejs.ProcessWrapperImpl.startProcess(ProcessWrapperImpl.java:39)
        at org.sonarsource.nodejs.NodeCommand.start(NodeCommand.java:77)
        ... 38 common frames omitted
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
        ... 41 common frames omitted

User environment

os: linux
image : lequal/sonar-scanner:latest

function get_number_of_issues() from file test_cnes_sonar_scanner.py is not working in case of an issue number superior to 100

function get_number_of_issues() from file test_cnes_sonar_scanner.py is not working in case of an issue number superior to 100.

def get_number_of_issues():
            """
            Factor out the resquest to get the number of issues of a project on SonarQube

            :returns: the number of issues
            """
            output = requests.get(f"{cls.SONARQUBE_LOCAL_URL}/api/issues/search?componentKeys={project_key}",
                        auth=("admin", cls.SONARQUBE_ADMIN_PASSWORD)).json()['issues']
            issues = [ issue for issue in output if issue['status'] in ('OPEN', 'TO_REVIEW') ]
            return len(issues)

This function does not handle paging of API/issues/search result.
In the case of the VHDL project the return of the query is:
image

Therefore the query result should be iterate over every 8 pages to return the good number of issues.

In fact, as it is a new project, all the issues should be open . So there won't need iteration around issues to check status.
I think you can simply query to get the total value from paging field.

Originally posted by @LeFl0w in #18 (comment)

Add hadolint support

Is your feature request related to a problem? Please describe.

A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

This image cannot handle Docker projects.

Describe the solution you'd like

A clear and concise description of what you want to happen.

Last release of hadolint should be installed and available: https://github.com/hadolint/hadolint

Python support

Is your feature request related to a problem? Please describe.

A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Python language should be supported.

Describe the solution you'd like

A clear and concise description of what you want to happen.

The same tools as in Docker-CAT should be available.

Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you've considered.

Support for Shell and Fortran

Is your feature request related to a problem? Please describe.

A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Shell and Fortran languages should be supported, like in Docker-CAT.

Describe the solution you'd like

A clear and concise description of what you want to happen.

This image should be able to analyze shell and fortran projects.

Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you've considered.

Wait for the CI workflow to complete before running the CD workflow

Describe the bug

A clear and concise description of what the bug is.

The CD workflow should be executed after the CI workflow if the CI succeeded.

To reproduce

Steps to reproduce the behavior.

  1. See that when merging the la PR of a milestone, CI and CD workflows are run at the same time.

Expected behavior

A clear and concise description of what you expected to happen.

CD should be executed after CI.

Screenshots & log

If applicable, add screenshots and logs to help explain your problem. Do not forget to obfuscate them if they contain sensitive data..

User environment

Please complete the following information.

  • OS: GitHub runners
  • Project version: 0.1.0

Basic image

Create a basic image

The goal is to have a working, tested and documented image with minimal features.

C/C++ support

Is your feature request related to a problem? Please describe.

A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

C/C++ languages should be supported.

Describe the solution you'd like

A clear and concise description of what you want to happen.

The same tools as in Docker-CAT should be available.

Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you've considered.

Pylint default report filename

Is your feature request related to a problem? Please describe.

A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Like CppCheck, Vera++ and RATS, Pylint could have a default filename for its report.

Describe the solution you'd like

A clear and concise description of what you want to happen.

Following the naming convention of the other tools, the default filename for pylint could be pylint-report.txt.

Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you've considered.

Always specify -Dsonar.python.pylint.reportPath=pylint-report.txt when invoking sonar-scanner.

Automerge

Is your feature request related to a problem? Please describe.

A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

There should be a workflow to automatically merge PR that are reviewed and checked.

Describe the solution you'd like

A clear and concise description of what you want to happen.

A GitHub workflow.

Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you've considered.

  • Doing it manually

docker build fail

Describe the bug

The docker build of the image fail due to a missing version of g++ and libzarith-ocaml-dev in the repository.

To reproduce

Steps to reproduce the behavior.

  1. execute command : docker build -t lequal/sonar-scanner .

Expected behavior

The creation of the image shoud ran without any problem.

Possible fix

The error comes from Dockerfile line 201 and 208.
Removing version dependy could fix the issue . From :

apt-get install -y --no-install-recommends \
....
            libzarith-ocaml=1.9.1-* \
...
            gcc=4:10.1.0-* 

To:

apt-get install -y libzarith-ocaml g\+\+\

Screenshots & log

Get:10 http://deb.debian.org/debian buster-updates/main amd64 Packages [7868 B]
Fetched 16.3 MB in 31s (520 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
E: Version '4:10.1.0-*' for 'g++' was not found
E: Version '1.9.1-*' for 'libzarith-ocaml-dev' was not found
The command '/bin/sh -c echo 'deb http://ftp.fr.debian.org/debian/ bullseye main contrib non-free' >> /etc/apt/sources.list     && apt-get update     && apt-get install -y --no-install-recom

User environment

  • OS: windows 10

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤ī¸ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.