Monday, 12 October 2020

More about SonarQube and scanning Java source code ....

 As per previous posts : -

Tinkering with SonarQube for code-scanning shell scripts ...

Getting to grips with Maven - in five minutes ...

I've been tinkering further with SonarQube (SQ) to scan projects with Java files, both source .java AND compiled .class files.

I was trying to mitigate an issue where SQ or, to be more specific, the FindBugs plugin was complaining about uncompiled source ... in this instance, I've got a project that contains a single .java source file which, for various not-so-interesting reasons, has not been compiled.

This is what I did ...

Run SQ container

docker run -d --name sonarqube -e SONAR_ES_BOOTSTRAP_CHECKS_DISABLE=true -p 9000:9000 sonarqube:latest

Access SQ via browser

http://192.168.1.100:9000/about

Install FindBugs v4.0.1 plugin

http://192.168.1.100:9000/admin/marketplace?search=findbugs

Create Project

mkdir ~/DaveSQJava

Create Java source

vi ~/DaveSQJava/HelloWorld.java

public class HelloWorld

{

    public static void main(String[] args)

    {

        for (int i = 0; i < 5; i++) {

            System.out.println("Hello, World");

        }

    }

}

Scan Project

cd ~/DaveSQJava

sonar-scanner \ -Dsonar.projectKey=DaveSQJava \ -Dsonar.sources=. \ -Dsonar.host.url=http://192.168.1.100:9000 \ -Dsonar.login=2b7d7e9cd8d35baa9d9d5b8f11011bff703e4696

which fails with: -

ERROR: Error during SonarScanner execution

java.lang.IllegalStateException: Can not execute Findbugs

    at org.sonar.plugins.findbugs.FindbugsExecutor.execute(FindbugsExecutor.java:188)

    at org.sonar.plugins.findbugs.FindbugsSensor.execute(FindbugsSensor.java:114)

    at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:48)

    at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:85)

    at org.sonar.scanner.sensor.ModuleSensorsExecutor.lambda$execute$1(ModuleSensorsExecutor.java:59)

    at org.sonar.scanner.sensor.ModuleSensorsExecutor.withModuleStrategy(ModuleSensorsExecutor.java:77)

    at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:59)

    at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:82)

    at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)

    at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)

    at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:393)

    at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:389)

    at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:358)

    at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)

    at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)

    at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:144)

    at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)

    at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)

    at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:72)

    at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:66)

    at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)

    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

    at java.base/java.lang.reflect.Method.invoke(Unknown Source)

    at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)

    at com.sun.proxy.$Proxy0.execute(Unknown Source)

    at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)

    at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)

    at org.sonarsource.scanner.cli.Main.execute(Main.java:112)

    at org.sonarsource.scanner.cli.Main.execute(Main.java:75)

    at org.sonarsource.scanner.cli.Main.main(Main.java:61)

Caused by: java.lang.IllegalStateException: One (sub)project contains Java source files that are not compiled (/root/DaveSQJava).

    at org.sonar.plugins.findbugs.FindbugsConfiguration.getFindbugsProject(FindbugsConfiguration.java:123)

    at org.sonar.plugins.findbugs.FindbugsExecutor.execute(FindbugsExecutor.java:119)

    ... 31 more

ERROR:

ERROR: Re-run SonarScanner using the -X switch to enable full debug logging.

However, I found some inspiration here: -

sonar-findbugs 3.6 fails when analyzing module with non-compiled JSPs #148

which led me down a series of rabbit holes until I found a configuration option within the FindBugs plugin within the SQ web UI itself: -


Working a hunch, I decided to try sonar.findbugs.allowuncompiledcode as a command-line switch .....

What could possibly go wrong ?

Add -Dsonar.findbugs.allowuncompiledcode switch

sonar-scanner \ -Dsonar.projectKey=DaveSQJava \ -Dsonar.sources=. \ -Dsonar.host.url=http://192.168.1.100:9000 \ -Dsonar.login=2b7d7e9cd8d35baa9d9d5b8f11011bff703e4696 -Dsonar.findbugs.allowuncompiledcode

which....

WORKED !!

INFO: ------------------------------------------------------------------------

INFO: EXECUTION SUCCESS

INFO: ------------------------------------------------------------------------

INFO: Total time: 10.553s

INFO: Final Memory: 17M/60M

INFO: ------------------------------------------------------------------------

I then add a .jsp file into the mix ....

vi ~/DaveSQJava/HelloWorld.jsp

<%@ page language="java" contentType="text/html; charset=ISO-8859-1"

    pageEncoding="ISO-8859-1"%>

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<html>

<head>

<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">

<title>JSP - Hello World Tutorial - Programmer Gate</title>

</head>

<body>

<%= "Hello World!" %>

</body>

</html>

and re-ran the scan: -

sonar-scanner   -Dsonar.projectKey=DaveSQJava   -Dsonar.sources=.   -Dsonar.host.url=http://158.85.5.109:9000   -Dsonar.login=2b7d7e9cd8d35baa9d9d5b8f11011bff703e4696 -Dsonar.findbugs.allowuncompiledcode

INFO: Sensor FindBugs Sensor [findbugs]
WARN: Findbugs needs sources to be compiled. Please build project before executing sonar or check the location of compiled classes to make it possible for Findbugs to analyse your (sub)project (/root/DaveSQJava).
WARN: JSP files were found in the current (sub)project (/root/DaveSQJava) but FindBugs requires their precompiled form. For more information on how to configure JSP precompilation : https://github.com/find-sec-bugs/find-sec-bugs/wiki/JSP-precompilation
INFO: Findbugs analysis skipped for this project.
INFO: Sensor FindBugs Sensor [findbugs] (done) | time=1628ms
INFO: Sensor SurefireSensor [java]
INFO: parsing [/root/DaveSQJava/target/surefire-reports]
INFO: Sensor SurefireSensor [java] (done) | time=3ms
INFO: Sensor JavaXmlSensor [java]
INFO: Sensor JavaXmlSensor [java] (done) | time=2ms
INFO: Sensor HTML [web]
INFO: Sensor HTML [web] (done) | time=159ms
INFO: Sensor VB.NET Properties [vbnet]
INFO: Sensor VB.NET Properties [vbnet] (done) | time=1ms
INFO: ------------- Run sensors on project
INFO: Sensor Zero Coverage Sensor
INFO: Sensor Zero Coverage Sensor (done) | time=11ms
INFO: Sensor Java CPD Block Indexer
INFO: Sensor Java CPD Block Indexer (done) | time=17ms
INFO: SCM Publisher No SCM system was detected. You can use the 'sonar.scm.provider' property to explicitly specify it.
INFO: CPD Executor 1 file had no CPD blocks
INFO: CPD Executor Calculating CPD for 1 file
INFO: CPD Executor CPD calculation finished (done) | time=120ms
INFO: Analysis report generated in 96ms, dir size=84 KB
INFO: Analysis report compressed in 28ms, zip size=13 KB
INFO: Analysis report uploaded in 38ms
INFO: ANALYSIS SUCCESSFUL, you can browse http://158.85.5.109:9000/dashboard?id=DaveSQJava
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at http://158.85.5.109:9000/api/ce/task?id=AXUcx9l_NFsnENiRRAYU
INFO: Analysis total time: 9.051 s
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 10.724s
INFO: Final Memory: 8M/34M
INFO: ------------------------------------------------------------------------

Even better, the scan actually did scan / review the .jsp ( Java Server Pages ) source, and found some bugs ...



Just to confirm, my project has NO compiled code therein ...

pwd

/root/DaveSQJava

ls -R -al

.:

total 20

drwxr-xr-x  3 root root 4096 Oct 12 12:27 .

drwx------ 15 root root 4096 Oct 12 12:27 ..

drwxr-xr-x  3 root root 4096 Oct 12 12:27 .scannerwork

-rw-r--r--  1 root root  150 Oct 12 10:33 HelloWorld.java

-rw-r--r--  1 root root  404 Oct 12 12:27 HelloWorld.jsp


./.scannerwork:

total 16

drwxr-xr-x 3 root root 4096 Oct 12 12:27 .

drwxr-xr-x 3 root root 4096 Oct 12 12:27 ..

-rw-r--r-- 1 root root    0 Oct 12 10:34 .sonar_lock

-rw-r--r-- 1 root root    0 Oct 12 12:27 class-mapping.csv

drwxr-xr-x 2 root root 4096 Oct 12 12:27 findbugs

-rw-r--r-- 1 root root  246 Oct 12 12:27 report-task.txt


./.scannerwork/findbugs:

total 8

drwxr-xr-x 2 root root 4096 Oct 12 12:27 .

drwxr-xr-x 3 root root 4096 Oct 12 12:27 ..

Job's a good 'un ....

No comments:

Note to self - use kubectl to query images in a pod or deployment

In both cases, we use JSON ... For a deployment, we can do this: - kubectl get deployment foobar --namespace snafu --output jsonpath="{...