VMware Fusion - Toggling back and forth...

 A friend had asked how one can switch out of full-screen whilst running a VM within VMware Fusion on the Mac.

I had a quick dig about, and found this: -

Return to Another View from Full Screen View

which says, in part: -

To see the Mac desktop and the virtual machine desktop at the same time, or to see a guest application in a window on the Mac desktop, switch to Unity view or Single Window view.

To have your virtual machine appear in a single window on the Mac desktop, exit to Single Window view. To display guest application windows directly on your Mac desktop without seeing the virtual machine's desktop, exit to Unity view.

Procedure

    To exit Full Screen to Single Window view, in the Full Screen title bar, select View > Single Window, or use the keyboard shortcut Command key+Control+F.

    To exit Full Screen to Unity view, in the Full Screen title bar, select View > Unity, or use the keyboard shortcut Command key+Shift+U.

Whilst I'm not making use of Unity, the first tip - [command] [control] [f] - did the job nicely. For the record, I'm using VMware Fusion 12.0.0 on macOS Catalina 10.15.7, waiting for macOS Big Sur 11 to arrive .....

Thanks, VMware 

More about SonarQube and scanning Java source code ....

 As per previous posts : -

Tinkering with SonarQube for code-scanning shell scripts ...

Getting to grips with Maven - in five minutes ...

I've been tinkering further with SonarQube (SQ) to scan projects with Java files, both source .java AND compiled .class files.

I was trying to mitigate an issue where SQ or, to be more specific, the FindBugs plugin was complaining about uncompiled source ... in this instance, I've got a project that contains a single .java source file which, for various not-so-interesting reasons, has not been compiled.

This is what I did ...

Run SQ container

docker run -d --name sonarqube -e SONAR_ES_BOOTSTRAP_CHECKS_DISABLE=true -p 9000:9000 sonarqube:latest

Access SQ via browser

http://192.168.1.100:9000/about

Install FindBugs v4.0.1 plugin

http://192.168.1.100:9000/admin/marketplace?search=findbugs

Create Project

mkdir ~/DaveSQJava

Create Java source

vi ~/DaveSQJava/HelloWorld.java

public class HelloWorld

{

    public static void main(String[] args)

    {

        for (int i = 0; i < 5; i++) {

            System.out.println("Hello, World");

        }

    }

}

Scan Project

cd ~/DaveSQJava

sonar-scanner \ -Dsonar.projectKey=DaveSQJava \ -Dsonar.sources=. \ -Dsonar.host.url=http://192.168.1.100:9000 \ -Dsonar.login=2b7d7e9cd8d35baa9d9d5b8f11011bff703e4696

which fails with: -

ERROR: Error during SonarScanner execution

java.lang.IllegalStateException: Can not execute Findbugs

    at org.sonar.plugins.findbugs.FindbugsExecutor.execute(FindbugsExecutor.java:188)

    at org.sonar.plugins.findbugs.FindbugsSensor.execute(FindbugsSensor.java:114)

    at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:48)

    at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:85)

    at org.sonar.scanner.sensor.ModuleSensorsExecutor.lambda$execute$1(ModuleSensorsExecutor.java:59)

    at org.sonar.scanner.sensor.ModuleSensorsExecutor.withModuleStrategy(ModuleSensorsExecutor.java:77)

    at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:59)

    at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:82)

    at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)

    at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)

    at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:393)

    at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:389)

    at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:358)

    at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)

    at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)

    at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:144)

    at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)

    at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)

    at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:72)

    at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:66)

    at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)

    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

    at java.base/java.lang.reflect.Method.invoke(Unknown Source)

    at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)

    at com.sun.proxy.$Proxy0.execute(Unknown Source)

    at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)

    at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)

    at org.sonarsource.scanner.cli.Main.execute(Main.java:112)

    at org.sonarsource.scanner.cli.Main.execute(Main.java:75)

    at org.sonarsource.scanner.cli.Main.main(Main.java:61)

Caused by: java.lang.IllegalStateException: One (sub)project contains Java source files that are not compiled (/root/DaveSQJava).

    at org.sonar.plugins.findbugs.FindbugsConfiguration.getFindbugsProject(FindbugsConfiguration.java:123)

    at org.sonar.plugins.findbugs.FindbugsExecutor.execute(FindbugsExecutor.java:119)

    ... 31 more

ERROR:

ERROR: Re-run SonarScanner using the -X switch to enable full debug logging.

However, I found some inspiration here: -

sonar-findbugs 3.6 fails when analyzing module with non-compiled JSPs #148

which led me down a series of rabbit holes until I found a configuration option within the FindBugs plugin within the SQ web UI itself: -

Working a hunch, I decided to try sonar.findbugs.allowuncompiledcode as a command-line switch .....

What could possibly go wrong ?

Add -Dsonar.findbugs.allowuncompiledcode switch

sonar-scanner \ -Dsonar.projectKey=DaveSQJava \ -Dsonar.sources=. \ -Dsonar.host.url=http://192.168.1.100:9000 \ -Dsonar.login=2b7d7e9cd8d35baa9d9d5b8f11011bff703e4696 -Dsonar.findbugs.allowuncompiledcode

which....

WORKED !!

INFO: ------------------------------------------------------------------------

INFO: EXECUTION SUCCESS

INFO: ------------------------------------------------------------------------

INFO: Total time: 10.553s

INFO: Final Memory: 17M/60M

INFO: ------------------------------------------------------------------------

I then add a .jsp file into the mix ....

vi ~/DaveSQJava/HelloWorld.jsp

<%@ page language="java" contentType="text/html; charset=ISO-8859-1"

    pageEncoding="ISO-8859-1"%>

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<html>

<head>

<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">

<title>JSP - Hello World Tutorial - Programmer Gate</title>

</head>

<body>

<%= "Hello World!" %>

</body>

</html>

and re-ran the scan: -

sonar-scanner   -Dsonar.projectKey=DaveSQJava   -Dsonar.sources=.   -Dsonar.host.url=http://158.85.5.109:9000   -Dsonar.login=2b7d7e9cd8d35baa9d9d5b8f11011bff703e4696 -Dsonar.findbugs.allowuncompiledcode

INFO: Sensor FindBugs Sensor [findbugs]

WARN: Findbugs needs sources to be compiled. Please build project before executing sonar or check the location of compiled classes to make it possible for Findbugs to analyse your (sub)project (/root/DaveSQJava).

WARN: JSP files were found in the current (sub)project (/root/DaveSQJava) but FindBugs requires their precompiled form. For more information on how to configure JSP precompilation : https://github.com/find-sec-bugs/find-sec-bugs/wiki/JSP-precompilation

INFO: Findbugs analysis skipped for this project.

INFO: Sensor FindBugs Sensor [findbugs] (done) | time=1628ms

INFO: Sensor SurefireSensor [java]

INFO: parsing [/root/DaveSQJava/target/surefire-reports]

INFO: Sensor SurefireSensor [java] (done) | time=3ms

INFO: Sensor JavaXmlSensor [java]

INFO: Sensor JavaXmlSensor [java] (done) | time=2ms

INFO: Sensor HTML [web]

INFO: Sensor HTML [web] (done) | time=159ms

INFO: Sensor VB.NET Properties [vbnet]

INFO: Sensor VB.NET Properties [vbnet] (done) | time=1ms

INFO: ------------- Run sensors on project

INFO: Sensor Zero Coverage Sensor

INFO: Sensor Zero Coverage Sensor (done) | time=11ms

INFO: Sensor Java CPD Block Indexer

INFO: Sensor Java CPD Block Indexer (done) | time=17ms

INFO: SCM Publisher No SCM system was detected. You can use the 'sonar.scm.provider' property to explicitly specify it.

INFO: CPD Executor 1 file had no CPD blocks

INFO: CPD Executor Calculating CPD for 1 file

INFO: CPD Executor CPD calculation finished (done) | time=120ms

INFO: Analysis report generated in 96ms, dir size=84 KB

INFO: Analysis report compressed in 28ms, zip size=13 KB

INFO: Analysis report uploaded in 38ms

INFO: ANALYSIS SUCCESSFUL, you can browse http://158.85.5.109:9000/dashboard?id=DaveSQJava

INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report

INFO: More about the report processing at http://158.85.5.109:9000/api/ce/task?id=AXUcx9l_NFsnENiRRAYU

INFO: Analysis total time: 9.051 s

INFO: ------------------------------------------------------------------------

INFO: EXECUTION SUCCESS

INFO: ------------------------------------------------------------------------

INFO: Total time: 10.724s

INFO: Final Memory: 8M/34M

INFO: ------------------------------------------------------------------------

Even better, the scan actually did scan / review the .jsp ( Java Server Pages ) source, and found some bugs ...

Just to confirm, my project has NO compiled code therein ...

pwd

/root/DaveSQJava

ls -R -al

.:

total 20

drwxr-xr-x  3 root root 4096 Oct 12 12:27 .

drwx------ 15 root root 4096 Oct 12 12:27 ..

drwxr-xr-x  3 root root 4096 Oct 12 12:27 .scannerwork

-rw-r--r--  1 root root  150 Oct 12 10:33 HelloWorld.java

-rw-r--r--  1 root root  404 Oct 12 12:27 HelloWorld.jsp

./.scannerwork:

total 16

drwxr-xr-x 3 root root 4096 Oct 12 12:27 .

drwxr-xr-x 3 root root 4096 Oct 12 12:27 ..

-rw-r--r-- 1 root root    0 Oct 12 10:34 .sonar_lock

-rw-r--r-- 1 root root    0 Oct 12 12:27 class-mapping.csv

drwxr-xr-x 2 root root 4096 Oct 12 12:27 findbugs

-rw-r--r-- 1 root root  246 Oct 12 12:27 report-task.txt

./.scannerwork/findbugs:

total 8

drwxr-xr-x 2 root root 4096 Oct 12 12:27 .

drwxr-xr-x 3 root root 4096 Oct 12 12:27 ..

Job's a good 'un ....

Talking with tree - on the Mac

 Now back in the day, I remember a version of DOS or Windows that introduced the tree command that allowed one to produce a pseudo-graphical listing of a file-system.

This nice picture from Wikipedia shows the concept: -

Well, I'd missed that on macOS ...

Now I miss it no longer ....

'twas a simple installation: -

brew install tree

and now: -

which tree

/usr/local/bin/tree

tree --version

tree v1.8.0 (c) 1996 - 2018 by Steve Baker, Thomas Moore, Francesc Rocher, Florian Sesser, Kyosuke Tokoro 

tree --help

usage: tree [-acdfghilnpqrstuvxACDFJQNSUX] [-H baseHREF] [-T title ]

[-L level [-R]] [-P pattern] [-I pattern] [-o filename] [--version]

[--help] [--inodes] [--device] [--noreport] [--nolinks] [--dirsfirst]

[--charset charset] [--filelimit[=]#] [--si] [--timefmt[=]<f>]

[--sort[=]<name>] [--matchdirs] [--ignore-case] [--fromfile] [--]

[<directory list>]

  ------- Listing options -------

  -a            All files are listed.

  -d            List directories only.

  -l            Follow symbolic links like directories.

  -f            Print the full path prefix for each file.

  -x            Stay on current filesystem only.

  -L level      Descend only level directories deep.

  -R            Rerun tree when max dir level reached.

  -P pattern    List only those files that match the pattern given.

  -I pattern    Do not list files that match the given pattern.

  --ignore-case Ignore case when pattern matching.

  --matchdirs   Include directory names in -P pattern matching.

  --noreport    Turn off file/directory count at end of tree listing.

  --charset X   Use charset X for terminal/HTML and indentation line output.

  --filelimit # Do not descend dirs with more than # files in them.

  --timefmt <f> Print and format time according to the format <f>.

  -o filename   Output to file instead of stdout.

  ------- File options -------

  -q            Print non-printable characters as '?'.

  -N            Print non-printable characters as is.

  -Q            Quote filenames with double quotes.

  -p            Print the protections for each file.

  -u            Displays file owner or UID number.

  -g            Displays file group owner or GID number.

  -s            Print the size in bytes of each file.

  -h            Print the size in a more human readable way.

  --si          Like -h, but use in SI units (powers of 1000).

  -D            Print the date of last modification or (-c) status change.

  -F            Appends '/', '=', '*', '@', '|' or '>' as per ls -F.

  --inodes      Print inode number of each file.

  --device      Print device ID number to which each file belongs.

  ------- Sorting options -------

  -v            Sort files alphanumerically by version.

  -t            Sort files by last modification time.

  -c            Sort files by last status change time.

  -U            Leave files unsorted.

  -r            Reverse the order of the sort.

  --dirsfirst   List directories before files (-U disables).

  --sort X      Select sort: name,version,size,mtime,ctime.

  ------- Graphics options -------

  -i            Don't print indentation lines.

  -A            Print ANSI lines graphic indentation lines.

  -S            Print with CP437 (console) graphics indentation lines.

  -n            Turn colorization off always (-C overrides).

  -C            Turn colorization on always.

  ------- XML/HTML/JSON options -------

  -X            Prints out an XML representation of the tree.

  -J            Prints out an JSON representation of the tree.

  -H baseHREF   Prints out HTML format with baseHREF as top directory.

  -T string     Replace the default HTML title and H1 header with string.

  --nolinks     Turn off hyperlinks in HTML output.

  ------- Input options -------

  --fromfile    Reads paths from files (.=stdin)

  ------- Miscellaneous options -------

  --version     Print version and exit.

  --help        Print usage and this help message and exit.

  --            Options processing terminator.

I can even get a tree in JSON format ....

tree -J

[{"type":"directory","name": ".","contents":[

    {"type":"file","name":"pom.xml"},

    {"type":"directory","name":"src","contents":[

      {"type":"directory","name":"main","contents":[

        {"type":"directory","name":"java","contents":[

          {"type":"directory","name":"com","contents":[

            {"type":"directory","name":"dave","contents":[

              {"type":"directory","name":"app","contents":[

                {"type":"file","name":"App.java"}

              ]}

            ]}

          ]}

        ]}

      ]},

      {"type":"directory","name":"test","contents":[

        {"type":"directory","name":"java","contents":[

          {"type":"directory","name":"com","contents":[

            {"type":"directory","name":"dave","contents":[

              {"type":"directory","name":"app","contents":[

                {"type":"file","name":"AppTest.java"}

              ]}

            ]}

          ]}

        ]}

      ]}

    ]},

    {"type":"directory","name":"target","contents":[

      {"type":"directory","name":"classes","contents":[

        {"type":"directory","name":"com","contents":[

          {"type":"directory","name":"dave","contents":[

            {"type":"directory","name":"app","contents":[

              {"type":"file","name":"App.class"}

            ]}

          ]}

        ]}

      ]},

      {"type":"file","name":"daves-app-1.0-SNAPSHOT.jar"},

      {"type":"directory","name":"generated-sources","contents":[

        {"type":"directory","name":"annotations","contents":[

        ]}

      ]},

      {"type":"directory","name":"generated-test-sources","contents":[

        {"type":"directory","name":"test-annotations","contents":[

        ]}

      ]},

      {"type":"directory","name":"maven-archiver","contents":[

        {"type":"file","name":"pom.properties"}

      ]},

      {"type":"directory","name":"maven-status","contents":[

        {"type":"directory","name":"maven-compiler-plugin","contents":[

          {"type":"directory","name":"compile","contents":[

            {"type":"directory","name":"default-compile","contents":[

              {"type":"file","name":"createdFiles.lst"},

              {"type":"file","name":"inputFiles.lst"}

            ]}

          ]},

          {"type":"directory","name":"testCompile","contents":[

            {"type":"directory","name":"default-testCompile","contents":[

              {"type":"file","name":"createdFiles.lst"},

              {"type":"file","name":"inputFiles.lst"}

            ]}

          ]}

        ]}

      ]},

      {"type":"directory","name":"surefire-reports","contents":[

        {"type":"file","name":"TEST-com.dave.app.AppTest.xml"},

        {"type":"file","name":"com.dave.app.AppTest.txt"}

      ]},

      {"type":"directory","name":"test-classes","contents":[

        {"type":"directory","name":"com","contents":[

          {"type":"directory","name":"dave","contents":[

            {"type":"directory","name":"app","contents":[

              {"type":"file","name":"AppTest.class"}

            ]}

          ]}

        ]}

      ]}

    ]}

  ]},

  {"type":"report","directories":32,"files":13}

]

Do I need that ? No, but who cares !! I CAN!!

Colour me happy 😂😂😂

Getting to grips with Maven - in five minutes ...

As part of my investigation into SonarQube, as per: -

Tinkering with SonarQube for code-scanning shell scripts ... 

I wanted to validate SQ's ability to scan Java projects.

SQ "assumes" that you're either using Maven or Gradle: -

so I thought I'd play with Maven, having used it for other Java work recently.

Thankfully, the Apache Maven project has a really useful tutorial on their site: -

Maven in 5 Minutes

which I followed, both on a Linux virtual server and on my Mac.

Here's a whiz-through the tutorial on my Mac ( with a bit of editorialising on my part ) : -

Validate Maven version

mvn --version

Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)

Maven home: /usr/local/Cellar/maven/3.6.3_1/libexec

Java version: 1.8.0_251, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/jdk1.8.0_251.jdk/Contents/Home/jre

Default locale: en_GB, platform encoding: UTF-8

OS name: "mac os x", version: "10.15.7", arch: "x86_64", family: "mac"

Creating a Project

mvn archetype:generate -DgroupId=com.dave.app -DartifactId=daves-app -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4 -DinteractiveMode=false

[INFO] Scanning for projects...

....

[INFO] ----------------------------------------------------------------------------

[INFO] Using following parameters for creating project from Archetype: maven-archetype-quickstart:1.4

[INFO] ----------------------------------------------------------------------------

[INFO] Parameter: groupId, Value: com.dave.app

[INFO] Parameter: artifactId, Value: daves-app

[INFO] Parameter: version, Value: 1.0-SNAPSHOT

[INFO] Parameter: package, Value: com.dave.app

[INFO] Parameter: packageInPathFormat, Value: com/dave/app

[INFO] Parameter: version, Value: 1.0-SNAPSHOT

[INFO] Parameter: package, Value: com.dave.app

[INFO] Parameter: groupId, Value: com.dave.app

[INFO] Parameter: artifactId, Value: daves-app

[INFO] Project created from Archetype in dir: /Users/hayd/daves-app

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time:  56.777 s

[INFO] Finished at: 2020-10-08T07:53:00+01:00

[INFO] ------------------------------------------------------------------------

This created a new project folder, with subdirectories: -

tree daves-app/

daves-app/

├── pom.xml

└── src

    ├── main

    │   └── java

    │       └── com

    │           └── dave

    │               └── app

    │                   └── App.java

    └── test

        └── java

            └── com

                └── dave

                    └── app

                        └── AppTest.java

11 directories, 3 files

including pom.xml : -

cat daves-app/pom.xml

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

  <modelVersion>4.0.0</modelVersion>

  <groupId>com.dave.app</groupId>

  <artifactId>daves-app</artifactId>

  <version>1.0-SNAPSHOT</version>

  <name>daves-app</name>

  <!-- FIXME change it to the project's website -->

  <url>http://www.example.com</url>

  <properties>

    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

    <maven.compiler.source>1.7</maven.compiler.source>

    <maven.compiler.target>1.7</maven.compiler.target>

  </properties>

  <dependencies>

    <dependency>

      <groupId>junit</groupId>

      <artifactId>junit</artifactId>

      <version>4.11</version>

      <scope>test</scope>

    </dependency>

  </dependencies>

...

and, of course, the Java app: -

cat ./src/main/java/com/dave/app/App.java

package com.dave.app;

/**

 * Hello world!

 *

 */

public class App 

{

    public static void main( String[] args )

    {

        System.out.println( "Hello World!" );

    }

}

mvn package

[INFO] Scanning for projects...

[INFO] 

[INFO] -----------------------< com.dave.app:daves-app >-----------------------

[INFO] Building daves-app 1.0-SNAPSHOT

[INFO] --------------------------------[ jar ]---------------------------------

...

[INFO] 

[INFO] -------------------------------------------------------

[INFO]  T E S T S

[INFO] -------------------------------------------------------

[INFO] Running com.dave.app.AppTest

[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.023 s - in com.dave.app.AppTest

[INFO] 

[INFO] Results:

[INFO] 

[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] 

[INFO] 

[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ daves-app ---

...

[INFO] Building jar: /Users/hayd/daves-app/target/daves-app-1.0-SNAPSHOT.jar

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time:  27.681 s

[INFO] Finished at: 2020-10-08T08:00:09+01:00

[INFO] ------------------------------------------------------------------------

mvn validate

[INFO] Scanning for projects...

[INFO] 

[INFO] -----------------------< com.dave.app:daves-app >-----------------------

[INFO] Building daves-app 1.0-SNAPSHOT

[INFO] --------------------------------[ jar ]---------------------------------

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time:  0.079 s

[INFO] Finished at: 2020-10-08T08:16:49+01:00

[INFO] ------------------------------------------------------------------------

find . -name `ls -R | grep \.jar`

./target/daves-app-1.0-SNAPSHOT.jar

ls -al target/daves-app-1.0-SNAPSHOT.jar 

-rw-r--r--  1 hayd  staff  2687  8 Oct 08:00 target/daves-app-1.0-SNAPSHOT.jar

tree

.

├── pom.xml

├── src

│   ├── main

│   │   └── java

│   │       └── com

│   │           └── dave

│   │               └── app

│   │                   └── App.java

│   └── test

│       └── java

│           └── com

│               └── dave

│                   └── app

│                       └── AppTest.java

└── target

    ├── classes

    │   └── com

    │       └── dave

    │           └── app

    │               └── App.class

    ├── daves-app-1.0-SNAPSHOT.jar

    ├── generated-sources

    │   └── annotations

    ├── generated-test-sources

    │   └── test-annotations

    ├── maven-archiver

    │   └── pom.properties

    ├── maven-status

    │   └── maven-compiler-plugin

    │       ├── compile

    │       │   └── default-compile

    │       │       ├── createdFiles.lst

    │       │       └── inputFiles.lst

    │       └── testCompile

    │           └── default-testCompile

    │               ├── createdFiles.lst

    │               └── inputFiles.lst

    ├── surefire-reports

    │   ├── TEST-com.dave.app.AppTest.xml

    │   └── com.dave.app.AppTest.txt

    └── test-classes

        └── com

            └── dave

                └── app

                    └── AppTest.class

32 directories, 13 files

Run the App

java -cp target/daves-app-1.0-SNAPSHOT.jar com.dave.app.App

Hello World!

Change the App

vi ./src/main/java/com/dave/app/App.java

package com.dave.app;

/**

 * Hello world!

 *

 */

public class App

{

    public static void main( String[] args )

    {

        System.out.println( "Hello World!, you rule :-)" );

    }

}

Rebuild the App

mvn package

[INFO] Scanning for projects...

[INFO] 

[INFO] -----------------------< com.dave.app:daves-app >-----------------------

[INFO] Building daves-app 1.0-SNAPSHOT

[INFO] --------------------------------[ jar ]---------------------------------

[INFO] 

[INFO] --- maven-resources-plugin:3.0.2:resources (default-resources) @ daves-app ---

[INFO] Using 'UTF-8' encoding to copy filtered resources.

[INFO] skip non existing resourceDirectory /Users/hayd/daves-app/src/main/resources

[INFO] 

[INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ daves-app ---

[INFO] Changes detected - recompiling the module!

[INFO] Compiling 1 source file to /Users/hayd/daves-app/target/classes

[INFO] 

[INFO] --- maven-resources-plugin:3.0.2:testResources (default-testResources) @ daves-app ---

[INFO] Using 'UTF-8' encoding to copy filtered resources.

[INFO] skip non existing resourceDirectory /Users/hayd/daves-app/src/test/resources

[INFO] 

[INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) @ daves-app ---

[INFO] Nothing to compile - all classes are up to date

[INFO] 

[INFO] --- maven-surefire-plugin:2.22.1:test (default-test) @ daves-app ---

[INFO] 

[INFO] -------------------------------------------------------

[INFO]  T E S T S

[INFO] -------------------------------------------------------

[INFO] Running com.dave.app.AppTest

[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.02 s - in com.dave.app.AppTest

[INFO] 

[INFO] Results:

[INFO] 

[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

[INFO] 

[INFO] 

[INFO] --- maven-jar-plugin:3.0.2:jar (default-jar) @ daves-app ---

[INFO] Building jar: /Users/hayd/daves-app/target/daves-app-1.0-SNAPSHOT.jar

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time:  2.203 s

[INFO] Finished at: 2020-10-08T09:00:13+01:00

[INFO] ------------------------------------------------------------------------

Run the App

java -cp target/daves-app-1.0-SNAPSHOT.jar com.dave.app.App

Hello World!, you rule :-)

Thanks for Apache Maven for their tutorial: -

Maven in 5 Minutes

Right, back to SonarQube ......

Tinkering with SonarQube for code-scanning shell scripts ...

I'm having a very quick tinker with a tool called SonarQube for code quality scanning.

One of my colleagues had asked whether SQ can scan scripts e.g. Bash, which made me go "Hmmmm" and start to play ...

As you'd expect, I started with a Docker container: -

docker pull sonarqube

docker run -d --name sonarqube -e SONAR_ES_BOOTSTRAP_CHECKS_DISABLE=true -p 9000:9000 sonarqube:latest

which is the bare minimum for getting SQ running, with the internal port 9000 mapped to the host port 9000.

I then hit my Linux virtual server on that port: -

http://10.0.0.10:9000/about

and logged in.

I cribbed the above from the Getting Started guide on the SQ website, and got to a point where I had a project setup, ready for scanning ....

I then downloaded the SQ scanning tool ( for my Linux box ) : -

wget https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-4.5.0.2216-linux.zip

and extracted it: -

mkdir -p /sonarqube

cd /sonarqube

unzip ~/sonar-scanner-cli-4.5.0.2216-linux.zip 

and added SQ to my PATH: -

export PATH=$PATH:/usr/local/go/bin:$GOPATH/bin:/root/sonarqube/sonar-scanner-4.5.0.2216-linux/bin

I had previously created a dummy "app" comprising a Dockerfile, a Go module and a Bash script: -

cd ~/Dave

ls -al

total 4032

drwxr-xr-x  3 root root    4096 Oct  6 13:23 .

drwx------ 12 root root    4096 Oct  6 13:21 ..

-rwxr-xr-x  1 root root 2068291 Oct  4 16:34 Dave

-rw-r--r--  1 root root     121 Oct  4 16:36 Dockerfile

-rwxr-xr-x  1 root root 2034794 Oct  4 16:44 hello

-rw-r--r--  1 root root      76 Oct  4 16:34 hello.go

-rwxr-xr-x  1 root root      32 Oct  6 13:21 hello.sh

so I was then able to run a scan: -

sonar-scanner   -Dsonar.projectKey=dave_test   -Dsonar.sources=.   -Dsonar.host.url=http://10.0.0.10:9000   -Dsonar.login=hah82889fhqwhabe9173283

and a scan magically appeared in the SQ web UI: -

However, I did notice this message down on the right-hand side: -

Quality Profile: Use 'ShellCheck' (Shell)

which led me to ShellCheck and sonar-shellcheck and yet more shellcheck ..

I installed the Plugin on the SQ server: -

but was still seeing the same "warning"

Only then did I realise that I was missing something on the "client" side i.e. from where I'm running the actual SQ scan.

I installed the requisite binary: -

apt-get install shellcheck

which shellcheck

shellcheck --version

ShellCheck - shell script analysis tool

version: 0.4.6

license: GNU General Public License, version 3

website: http://www.shellcheck.net

and re-ran the scan, which reported, in part: -

INFO: 1 source files to be analyzed

INFO: Load project repositories

INFO: Load project repositories (done) | time=23ms

INFO: 1/1 source files have been analyzed

INFO: Sensor SonarGo [go] (done) | time=418ms

INFO: Sensor ShellCheck Sensor [shellcheck]

INFO: Sensor ShellCheck Sensor [shellcheck] (done) | time=257ms

INFO: Sensor JavaXmlSensor [java]

INFO: Sensor JavaXmlSensor [java] (done) | time=2ms

INFO: Sensor HTML [web]

INFO: Sensor HTML [web] (done) | time=5ms

INFO: ------------- Run sensors on project

INFO: Sensor Zero Coverage Sensor

INFO: Sensor Zero Coverage Sensor (done) | time=11ms

INFO: SCM Publisher No SCM system was detected. You can use the 'sonar.scm.provider' property to explicitly specify it.

INFO: CPD Executor 1 file had no CPD blocks

INFO: CPD Executor Calculating CPD for 0 files

INFO: CPD Executor CPD calculation finished (done) | time=0ms

INFO: Analysis report generated in 132ms, dir size=100 KB

INFO: Analysis report compressed in 25ms, zip size=12 KB

INFO: Analysis report uploaded in 27ms

and now my scan looks lovely: -

Yay!

Docker Content Trust and the case of the strange errors e.g. "read/write on closed pipe"

 Whilst trying to build a Docker image on one of my IBM Z build servers yesterday, I hit a weird series of errors, including : -

ERRO[0000] Can't add file /root/dave/hello to tar: io: read/write on closed pipe 

ERRO[0000] Can't close tar writer: io: read/write on closed pipe 

and: -

error during connect: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.40/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&nocache=1&rm=1&shmsize=0&target=&ulimits=null&version=1: Error: error contacting notary server: unauthorized: The login credentials are not valid, or your IBM Cloud account is not active.

Initially, I thought that the first error only appeared to occur with a Dockerfile that copied a binary object ( actually a compiled binary written in Go ) into the image.

After some fun n' games, I realised what was going on ....

I've reproduced the situation below: -

cd dave

ls -al

total 19016

drwxr-x---  2 root root     4096 Oct  3 13:31 .

drwx------ 15 root root     4096 Oct  3 13:30 ..

-rw-r-----  1 root root       38 Oct  3 13:30 Dockerfile

-rw-r--r--  1 root root       13 Oct  1 12:28 greeting.txt

-rwxr-x---  1 root root 19452575 Oct  3 13:31 hello

cat Dockerfile 

FROM alpine:3.11

COPY greeting.txt .

Note that there's NO mention of the binary file ( hello ) in the Dockerfile and yet .....

docker build --no-cache -f Dockerfile .

Sending build context to Docker daemon 

ERRO[0000] Can't add file /root/dave/hello to tar: io: read/write on closed pipe 

ERRO[0000] Can't close tar writer: io: read/write on closed pipe 

error during connect: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.40/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&nocache=1&rm=1&shmsize=0&target=&ulimits=null&version=1: Error: error contacting notary server: unauthorized: The login credentials are not valid, or your IBM Cloud account is not active.

Given that I knew that I wasn't logged into Docker Hub or the IBM Cloud Container Registry (ICCR) instance that I've been using for many of my builds .....

I even checked this: -

cat ~/.docker/config.json | json_pp 

{

   "HttpHeaders" : {

      "User-Agent" : "Docker-Client/19.03.6 (linux)"

   },

   "auths" : {}

}

And then it struck me ...... with the force of Mjolnir ....

Note that the message related to my "IBM Cloud account" ?

And also note that the error says "error contacting notary server" ?

And yet I wasn't logged into IBM Cloud .....

However, I had been earlier AND was working with Docker Content Trust (DCT).

And I'd set two environment variables in my current Bash session: -

set | grep DOCKER

DOCKER_CONTENT_TRUST=1

DOCKER_CONTENT_TRUST_SERVER=https://de.icr.io:4443

but I wasn't logged into Docker Hub or, in this case, ICCR .....

So I'd told the Docker Client to set DCT and even told it where the Notary server was ....

And yet .....

So I disabled DCT: -

export DOCKER_CONTENT_TRUST=

and re-ran the build: -

docker build --no-cache -f Dockerfile .

Sending build context to Docker daemon  19.46MB

Step 1/2 : FROM alpine:3.11

3.11: Pulling from library/alpine

7184c046fdf1: Pull complete 

Digest: sha256:9a839e63dad54c3a6d1834e29692c8492d93f90c59c978c1ed79109ea4fb9a54

Status: Downloaded newer image for alpine:3.11

 ---> 4b858171dd2c

Step 2/2 : COPY greeting.txt .

 ---> 164461a814d3

Successfully built 164461a814d3

The moral of the story ? If you're using DCT, remember to log in to your Registry and thus Notary.

Otherwise ... DON'T !!!!!

"Permission denied" - Docker and permissions and PostgreSQL

Long story very short, I've been wrangling with a knotty "Permission denied" issue with a PostgreSQL container, with regard to some SQL and Bash scripts that the Dockerfile copies into the /docker-entrypoint-initdb.d directory at build time.

After some tinkering and thought, I realised what was going on ....

In short, the COPY command within the Dockerfile will, of course, inherit the permissions of the source files and, unless you choose to override with a chmod command, that's all she wrote ....

I proved this via a very simple test case: -

I started with this: -

ls -al

total 16

drwxr-x---  2 root root 4096 Oct  1 12:52 .

drwx------ 15 root root 4096 Oct  1 12:52 ..

-rw-r-----  1 root root   94 Oct  1 12:32 Dockerfile

-rw-------  1 root root   13 Oct  1 12:28 greeting.txt

Note the permissions of the greeting.txt file are set to chmod 640 ( rw-r----- ). That means owner ( root ) has read, group ( root ). has read and everybody/public has NOTHING.

And here's the Dockerfile: -

FROM alpine

RUN mkdir /msg

COPY greeting.txt /msg

RUN ls -al /msg

RUN cat /msg/greeting.txt

When I built the image: -

docker build --no-cache -t dave:latest -f Dockerfile .

notice that the permission of that file are retained: -

Sending build context to Docker daemon  3.072kB

Step 1/5 : FROM alpine

 ---> 0d9ffb5b0ee7

Step 2/5 : RUN mkdir /msg

 ---> Running in 9b3a38e9a0df

Removing intermediate container 9b3a38e9a0df

 ---> 23b61ff3ebc9

Step 3/5 : COPY greeting.txt /msg

 ---> d0d76efa3d8a

Step 4/5 : RUN ls -al /msg

 ---> Running in 96b2db84132a

total 12

drwxr-xr-x    1 root     root          4096 Oct  1 13:03 .

drwxr-xr-x    1 root     root          4096 Oct  1 13:03 ..

-rw-------    1 root     root            13 Oct  1 12:28 greeting.txt

Removing intermediate container 96b2db84132a

 ---> 000ca956de06

Step 5/5 : RUN cat /msg/greeting.txt

 ---> Running in f5b63433f174

Hello World!

Removing intermediate container f5b63433f174

 ---> 8ccdc3916112

Successfully built 8ccdc3916112

Successfully tagged dave:latest

In other words, if the permissions start at 640 they stay at 640.

If I change the permissions on my build machine: -

chmod 644 greeting.txt

ls -al

total 16

drwxr-x---  2 root root 4096 Oct  1 12:52 .

drwx------ 15 root root 4096 Oct  1 12:52 ..

-rw-r-----  1 root root   94 Oct  1 12:32 Dockerfile

-rw-r--r--  1 root root   13 Oct  1 12:28 greeting.txt

Note the permissions of the greeting.txt file are set to chmod 644 ( rw-r--r-- ).

That means owner ( root ) has read, group ( root ). has read and everybody/public has read.

When I re-run the build: -

docker build --no-cache -t dave:latest -f Dockerfile .

Sending build context to Docker daemon  3.072kB

Step 1/5 : FROM alpine

 ---> 0d9ffb5b0ee7

Step 2/5 : RUN mkdir /msg

 ---> Running in 0bd5097bde82

Removing intermediate container 0bd5097bde82

 ---> e05cbcd1c9a4

Step 3/5 : COPY greeting.txt /msg

 ---> 4281fac6e501

Step 4/5 : RUN ls -al /msg

 ---> Running in a0da87ffd1b4

total 12

drwxr-xr-x    1 root     root          4096 Oct  1 13:11 .

drwxr-xr-x    1 root     root          4096 Oct  1 13:11 ..

-rw-r--r--    1 root     root            13 Oct  1 12:28 greeting.txt

Removing intermediate container a0da87ffd1b4

 ---> 33973f2c3e98

Step 5/5 : RUN cat /msg/greeting.txt

 ---> Running in 8e9d1c3cf3e6

Hello World!

Removing intermediate container 8e9d1c3cf3e6

 ---> 27b1dd46c329

Successfully built 27b1dd46c329

Successfully tagged dave:latest

So, again, the permissions of the file on the host build machine are inherited/preserved inside the built image.

So, for me, the fix is to either: -

(a) change the permissions in the GitHub repo from which I'm building the image

(b) add a sneaky chmod into my Dockerfile.

For PostgreSQL, this was breaking things as the files were owned by the root user, so the postgres user didn't ever have access.