Monday, 12 October 2020

More about SonarQube and scanning Java source code ....

 As per previous posts : -

Tinkering with SonarQube for code-scanning shell scripts ...

Getting to grips with Maven - in five minutes ...

I've been tinkering further with SonarQube (SQ) to scan projects with Java files, both source .java AND compiled .class files.

I was trying to mitigate an issue where SQ or, to be more specific, the FindBugs plugin was complaining about uncompiled source ... in this instance, I've got a project that contains a single .java source file which, for various not-so-interesting reasons, has not been compiled.

This is what I did ...

Run SQ container

docker run -d --name sonarqube -e SONAR_ES_BOOTSTRAP_CHECKS_DISABLE=true -p 9000:9000 sonarqube:latest

Access SQ via browser

http://192.168.1.100:9000/about

Install FindBugs v4.0.1 plugin

http://192.168.1.100:9000/admin/marketplace?search=findbugs

Create Project

mkdir ~/DaveSQJava

Create Java source

vi ~/DaveSQJava/HelloWorld.java

public class HelloWorld

{

    public static void main(String[] args)

    {

        for (int i = 0; i < 5; i++) {

            System.out.println("Hello, World");

        }

    }

}

Scan Project

cd ~/DaveSQJava

sonar-scanner \ -Dsonar.projectKey=DaveSQJava \ -Dsonar.sources=. \ -Dsonar.host.url=http://192.168.1.100:9000 \ -Dsonar.login=2b7d7e9cd8d35baa9d9d5b8f11011bff703e4696

which fails with: -

ERROR: Error during SonarScanner execution

java.lang.IllegalStateException: Can not execute Findbugs

    at org.sonar.plugins.findbugs.FindbugsExecutor.execute(FindbugsExecutor.java:188)

    at org.sonar.plugins.findbugs.FindbugsSensor.execute(FindbugsSensor.java:114)

    at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:48)

    at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:85)

    at org.sonar.scanner.sensor.ModuleSensorsExecutor.lambda$execute$1(ModuleSensorsExecutor.java:59)

    at org.sonar.scanner.sensor.ModuleSensorsExecutor.withModuleStrategy(ModuleSensorsExecutor.java:77)

    at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:59)

    at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:82)

    at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)

    at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)

    at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:393)

    at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:389)

    at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:358)

    at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)

    at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)

    at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:144)

    at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)

    at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)

    at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:72)

    at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:66)

    at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)

    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

    at java.base/java.lang.reflect.Method.invoke(Unknown Source)

    at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)

    at com.sun.proxy.$Proxy0.execute(Unknown Source)

    at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)

    at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)

    at org.sonarsource.scanner.cli.Main.execute(Main.java:112)

    at org.sonarsource.scanner.cli.Main.execute(Main.java:75)

    at org.sonarsource.scanner.cli.Main.main(Main.java:61)

Caused by: java.lang.IllegalStateException: One (sub)project contains Java source files that are not compiled (/root/DaveSQJava).

    at org.sonar.plugins.findbugs.FindbugsConfiguration.getFindbugsProject(FindbugsConfiguration.java:123)

    at org.sonar.plugins.findbugs.FindbugsExecutor.execute(FindbugsExecutor.java:119)

    ... 31 more

ERROR:

ERROR: Re-run SonarScanner using the -X switch to enable full debug logging.

However, I found some inspiration here: -

sonar-findbugs 3.6 fails when analyzing module with non-compiled JSPs #148

which led me down a series of rabbit holes until I found a configuration option within the FindBugs plugin within the SQ web UI itself: -


Working a hunch, I decided to try sonar.findbugs.allowuncompiledcode as a command-line switch .....

What could possibly go wrong ?

Add -Dsonar.findbugs.allowuncompiledcode switch

sonar-scanner \ -Dsonar.projectKey=DaveSQJava \ -Dsonar.sources=. \ -Dsonar.host.url=http://192.168.1.100:9000 \ -Dsonar.login=2b7d7e9cd8d35baa9d9d5b8f11011bff703e4696 -Dsonar.findbugs.allowuncompiledcode

which....

WORKED !!

INFO: ------------------------------------------------------------------------

INFO: EXECUTION SUCCESS

INFO: ------------------------------------------------------------------------

INFO: Total time: 10.553s

INFO: Final Memory: 17M/60M

INFO: ------------------------------------------------------------------------

I then add a .jsp file into the mix ....

vi ~/DaveSQJava/HelloWorld.jsp

<%@ page language="java" contentType="text/html; charset=ISO-8859-1"

    pageEncoding="ISO-8859-1"%>

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<html>

<head>

<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">

<title>JSP - Hello World Tutorial - Programmer Gate</title>

</head>

<body>

<%= "Hello World!" %>

</body>

</html>

and re-ran the scan: -

sonar-scanner   -Dsonar.projectKey=DaveSQJava   -Dsonar.sources=.   -Dsonar.host.url=http://158.85.5.109:9000   -Dsonar.login=2b7d7e9cd8d35baa9d9d5b8f11011bff703e4696 -Dsonar.findbugs.allowuncompiledcode

INFO: Sensor FindBugs Sensor [findbugs]
WARN: Findbugs needs sources to be compiled. Please build project before executing sonar or check the location of compiled classes to make it possible for Findbugs to analyse your (sub)project (/root/DaveSQJava).
WARN: JSP files were found in the current (sub)project (/root/DaveSQJava) but FindBugs requires their precompiled form. For more information on how to configure JSP precompilation : https://github.com/find-sec-bugs/find-sec-bugs/wiki/JSP-precompilation
INFO: Findbugs analysis skipped for this project.
INFO: Sensor FindBugs Sensor [findbugs] (done) | time=1628ms
INFO: Sensor SurefireSensor [java]
INFO: parsing [/root/DaveSQJava/target/surefire-reports]
INFO: Sensor SurefireSensor [java] (done) | time=3ms
INFO: Sensor JavaXmlSensor [java]
INFO: Sensor JavaXmlSensor [java] (done) | time=2ms
INFO: Sensor HTML [web]
INFO: Sensor HTML [web] (done) | time=159ms
INFO: Sensor VB.NET Properties [vbnet]
INFO: Sensor VB.NET Properties [vbnet] (done) | time=1ms
INFO: ------------- Run sensors on project
INFO: Sensor Zero Coverage Sensor
INFO: Sensor Zero Coverage Sensor (done) | time=11ms
INFO: Sensor Java CPD Block Indexer
INFO: Sensor Java CPD Block Indexer (done) | time=17ms
INFO: SCM Publisher No SCM system was detected. You can use the 'sonar.scm.provider' property to explicitly specify it.
INFO: CPD Executor 1 file had no CPD blocks
INFO: CPD Executor Calculating CPD for 1 file
INFO: CPD Executor CPD calculation finished (done) | time=120ms
INFO: Analysis report generated in 96ms, dir size=84 KB
INFO: Analysis report compressed in 28ms, zip size=13 KB
INFO: Analysis report uploaded in 38ms
INFO: ANALYSIS SUCCESSFUL, you can browse http://158.85.5.109:9000/dashboard?id=DaveSQJava
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at http://158.85.5.109:9000/api/ce/task?id=AXUcx9l_NFsnENiRRAYU
INFO: Analysis total time: 9.051 s
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 10.724s
INFO: Final Memory: 8M/34M
INFO: ------------------------------------------------------------------------

Even better, the scan actually did scan / review the .jsp ( Java Server Pages ) source, and found some bugs ...



Just to confirm, my project has NO compiled code therein ...

pwd

/root/DaveSQJava

ls -R -al

.:

total 20

drwxr-xr-x  3 root root 4096 Oct 12 12:27 .

drwx------ 15 root root 4096 Oct 12 12:27 ..

drwxr-xr-x  3 root root 4096 Oct 12 12:27 .scannerwork

-rw-r--r--  1 root root  150 Oct 12 10:33 HelloWorld.java

-rw-r--r--  1 root root  404 Oct 12 12:27 HelloWorld.jsp


./.scannerwork:

total 16

drwxr-xr-x 3 root root 4096 Oct 12 12:27 .

drwxr-xr-x 3 root root 4096 Oct 12 12:27 ..

-rw-r--r-- 1 root root    0 Oct 12 10:34 .sonar_lock

-rw-r--r-- 1 root root    0 Oct 12 12:27 class-mapping.csv

drwxr-xr-x 2 root root 4096 Oct 12 12:27 findbugs

-rw-r--r-- 1 root root  246 Oct 12 12:27 report-task.txt


./.scannerwork/findbugs:

total 8

drwxr-xr-x 2 root root 4096 Oct 12 12:27 .

drwxr-xr-x 3 root root 4096 Oct 12 12:27 ..

Job's a good 'un ....

No comments:

Yay, VMware Fusion and macOS Big Sur - no longer "NAT good friends" - forgive the double negative and the terrible pun ...

After macOS 11 Big Sur was released in 2020, VMware updated their Fusion product to v12 and, sadly, managed to break Network Address Trans...