Digging into certificate expiration using OpenSSL

This popped up on my Twitter feed earlier: -

Check the expiration date of an SSL or TLS certificate

and I thought it worth book marking ...

Checking the expiration of various certificates on my network ....

openssl s_client -servername 192.168.1.1 -connect 192.168.1.1:443 < /dev/null | openssl x509 -noout -dates

depth=0 C = TW, ST = HsinChu, L = HuKou, O = DrayTek Corp., OU = DrayTek Support, CN = Vigor Router

verify error:num=18:self signed certificate

verify return:1

depth=0 C = TW, ST = HsinChu, L = HuKou, O = DrayTek Corp., OU = DrayTek Support, CN = Vigor Router

verify return:1

poll errornotBefore=Aug 30 17:21:28 2020 GMT

notAfter=Sep 29 17:21:28 2021 GMT

openssl s_client -servername 192.168.1.12 -connect 192.168.1.12:443 < /dev/null | openssl x509 -noout -dates

depth=0 CN = HP3050A, L = Vancouver, ST = Washington, C = US, O = HP, OU = HP-IPG

verify error:num=20:unable to get local issuer certificate

verify return:1

depth=0 CN = HP3050A, L = Vancouver, ST = Washington, C = US, O = HP, OU = HP-IPG

verify error:num=21:unable to verify the first certificate

verify return:1

poll errornotBefore=Mar 16 06:51:44 2012 GMT

notAfter=Mar 11 06:51:44 2032 GMT

openssl s_client -servername nessus.local.com -connect nessus.local.com:8834 < /dev/null | openssl x509 -noout -dates

depth=0 O = Nessus Users United, OU = Nessus Server, L = New York, C = US, ST = NY, CN = nessus.local.com

verify error:num=20:unable to get local issuer certificate

verify return:1

depth=0 O = Nessus Users United, OU = Nessus Server, L = New York, C = US, ST = NY, CN = nessus.local.com

verify error:num=21:unable to verify the first certificate

verify return:1

poll errornotBefore=Oct  4 12:29:02 2019 GMT

notAfter=Oct  3 12:29:02 2023 GMT

Every Day Is A School Day - Changing metadata on movie files ...

As per my earlier post: -

Sharing media libraries between Apple Macs - d'oh, I'm holding it wrong

I'm continuing to tinker with my media library, sharing between macOS, iOS, iPadOS and tvOS ...

One thing that was niggling me was that I had a series of media files, in the MP4 v2 [ISO 14496-14] format, aka .m4v, for which the metadata appeared to be wrong ...

The symptom for this was that, when I imported the .m4v files into my library, via TV.app on the macOS box that's sharing the library - an Apple Mini, the name of the media item, as shown with TV.app ( on that Mac or any other  device ) did NOT match my expectations.

As an example, I have a bunch of files: -

ls -1 Dinnerladies\ Season\ 2\ Episode\ *

Dinnerladies Season 2 Episode 1 - Catering.m4v

Dinnerladies Season 2 Episode 10 - Toast.m4v

Dinnerladies Season 2 Episode 2 - Trouble.m4v

Dinnerladies Season 2 Episode 3 - Holidays.m4v

Dinnerladies Season 2 Episode 4 - Fog.m4v

Dinnerladies Season 2 Episode 5 - Gamble.m4v

Dinnerladies Season 2 Episode 6 - Christmas.m4v

Dinnerladies Season 2 Episode 7 - Minnellium.m4v

Dinnerladies Season 2 Episode 8 - Christine.m4v

Dinnerladies Season 2 Episode 9 - Gravy.m4v

where the file name reflects the season, episode and name.

However, the episode names, as shown in TV.app, never reflected the filename ...

I spent a happy hour or two deleting/reimporting, but never managed to resolve it ...

And then I realised where I was going wrong ...

The file name is NOT the same as the episode name ...

TL;DR; this is stored in the file's metadata, which is more than just the file name itself ...

This metadata is shown via the [cmd][i] key sequence in Finder, and includes: -

Last opened

Dimensions

Codec

Colour profile

Title

Duration

Audio Channels

Yes, you've guessed it ... the Title did NOT NOT NOT match the filename

So how to change it ?

Even though [cmd][i] exposes the information, one cannot change it ...

So what do to ?

After a spot o' Googling, I found that the VLC media player CAN edit this metadata

I downloaded and installed VLC on the Mac mini, and ran through the affected .m4v files, updating the Title to reflect the filename ....

Having done this, I re-imported the files into my library and all is now good .....

VLC is my friend 👍👧👶😎

Sharing media libraries between Apple Macs - d'oh, I'm holding it wrong

 So I've got a Mac Mini on my home network, acting ( in part ) as a media server, using Media Sharing ( enabled via System Preferences -> Sharing ).

This allows me to share my DVD library across my various Apple devices, including iOS, tvOS and ... macOS.

Having confirmed that I can see / play movies etc. from my iPad and from my Apple TV, I wasn't able to see the media from another Mac on the same LAN.

In both cases, the Macs are using the native TV.app - the Mac mini to manage the library and a MacBook Pro to consume the shared media .....

As ever, I was holding it wrong ...

In TV.app, there's a Library tab, but that was only showing my purchased media: -

The thing I'd missed was the twisty / pull-down under Library on the left-hand side of the app: -

That allowed me to choose the Mac Mini ( I could've also chosen my Synology NAS as that's also got the same iTunes / Media Serving capability ).

Once I did that, c'est voila : -

Thanks to this : -

Import media using Home Sharing in the Apple TV app on Mac

for helping me out of my own mess!

Ubuntu upgrades - dealing with things that don't get upgraded

Doing an upgrade on an Ubuntu 18.04 boxen: - 

apt-get upgrade

...

Calculating upgrade... Done

The following packages were automatically installed and are no longer required:

  linux-headers-4.15.0-112 linux-headers-4.15.0-112-generic linux-image-4.15.0-112-generic linux-modules-4.15.0-112-generic linux-modules-extra-4.15.0-112-generic

Use 'apt autoremove' to remove them.

The following packages have been kept back:

  base-files libgl1-mesa-dri linux-generic linux-headers-generic linux-image-generic netplan.io ubuntu-server

0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.

...

I wondered how to handle the "kept back" warning ....

Wonder no more ....

apt-get --with-new-pkgs upgrade

...

Searching for splash image ... none found, skipping ...

Found kernel: /boot/vmlinuz-4.15.0-118-generic

Found kernel: /boot/vmlinuz-4.15.0-117-generic

Found kernel: /boot/vmlinuz-4.15.0-122-generic

Found kernel: /boot/vmlinuz-4.15.0-118-generic

Found kernel: /boot/vmlinuz-4.15.0-117-generic

Replacing config file /run/grub/menu.lst with new version

Updating /boot/grub/menu.lst ... done

/etc/kernel/postinst.d/zz-update-grub:

Sourcing file `/etc/default/grub'

Generating grub configuration file ...

Found linux image: /boot/vmlinuz-4.15.0-122-generic

Found initrd image: /boot/initrd.img-4.15.0-122-generic

Found linux image: /boot/vmlinuz-4.15.0-118-generic

Found initrd image: /boot/initrd.img-4.15.0-118-generic

Found linux image: /boot/vmlinuz-4.15.0-117-generic

Found initrd image: /boot/initrd.img-4.15.0-117-generic

done

Processing triggers for initramfs-tools (0.130ubuntu3.11) ...

update-initramfs: Generating /boot/initrd.img-4.15.0-122-generic

...

plus: -

apt autoremove

to get rid of any excess baggage.

Source: “The following packages have been kept back:” Why and how do I solve it?

Mitigating "SSL Medium Strength Cipher Suites Supported" warnings from Nessus scans

 It's been a while, but I'm back in the world of IBM Cloud Private (ICP), specifically mitigating a cipher warning message from a Nessus endpoint scan.

One of my z/Linux boxes, running Red Hat Enterprise Linux, threw up a warning: -

SSL Medium Strength Cipher Suites Supported 192.168.1.242:10259 / tcp

via a Nessus endpoint scan.

I used a simple piece of detective work to find out WHAT was listening on port 10259 : -

netstat -aonp|grep 10259

which returned: -

tcp6       0      0 :::10259                :::*                    LISTEN      45537/hyperkube      off (0.00/0/0)

and then used: -

ps auxw|grep 45537

to inspect the specific process ID ( PID ), which returned: -

root     45537  0.2  0.1 907488 61804 ?        Ssl  11:14   0:06 /hyperkube kube-scheduler --master=https://127.0.0.1:8001 --kubeconfig=/etc/cfc/conf/kube-scheduler-config.yaml --policy-config-file=/etc/cfc/conf/scheduler-policy-config.json --use-legacy-policy-config=true --v=2 --profiling=false --leader-elect=true

This confirmed that it was the kube-scheduler component of Kubernetes (K8s) - IBM Cloud Private is a distribution of K8s, as is Red Hat OpenShift Container Platform ( RHOCP ).

This K8s documentation: -

kube-scheduler

describes how the SSL/TLS ciphers for kube-scheduler can be modified using the --tls-cipher-suites switch.

This document: -

Modifying Cipher Suites used by Kubernetes in IBM Cloud Private

then describes the specifics for ICP, namely updating /etc/cfc/pods/master.json to include  --tls-cipher-suites for the scheduler Pod.

Once I backed up, and then edited, this file, the Pod was automatically restarted, and the Nessus scan ran cleanly.

macOS and the `networksetup` command - now I did not know that

Whilst providing some guidance to a colleague having fun with their Mac's DNS configuration, I "discovered" a new ( to me ) command - networksetup - which includes a HUGE array of useful commands: -

networksetup -listallnetworkservices

An asterisk (*) denotes that a network service is disabled.

USB 10/100/1000 LAN

Wi-Fi

Bluetooth PAN

Thunderbolt Bridge

networksetup -getdnsservers Wi-Fi

There aren't any DNS Servers set on Wi-Fi.

networksetup -getcomputername

Dave’s MacBook Pro

networksetup -getMTU Wi-Fi

Active MTU: 1500 (Current Setting: 1500)

networksetup -getinfo "USB 10/100/1000 LAN"

DHCP Configuration

IP address: 192.168.1.21

Subnet mask: 255.255.255.0

Router: 192.168.1.1

Client ID: 

IPv6: Automatic

IPv6 IP address: none

IPv6 Router: none

Ethernet Address: 00:a0:3c:37:02:42

etc.

The full list of sub-commands is here: -

networksetup -printcommands

networksetup -listnetworkserviceorder

networksetup -listallnetworkservices

networksetup -listallhardwareports

networksetup -detectnewhardware

networksetup -getmacaddress <hardwareport or device name>

networksetup -getcomputername

networksetup -setcomputername <name>

networksetup -getinfo <networkservice>

networksetup -setmanual <networkservice> <ip> <subnet> <router>

networksetup -setdhcp <networkservice> [clientid]

networksetup -setbootp <networkservice>

networksetup -setmanualwithdhcprouter <networkservice> <ip> 

networksetup -getadditionalroutes <networkservice>

networksetup -setadditionalroutes <networkservice> [ <dest> <mask> <gateway> ]*

networksetup -setv4off <networkservice>

networksetup -setv6off <networkservice>

networksetup -setv6automatic <networkservice>

networksetup -setv6LinkLocal <networkservice>

networksetup -setv6manual <networkservice> <networkservice> <address> <prefixlength> <router>

networksetup -getv6additionalroutes <networkservice>

networksetup -setv6additionalroutes <networkservice> [ <dest> <prefixlength> <gateway> ]*

networksetup -getdnsservers <networkservice>

networksetup -setdnsservers <networkservice> <dns1> [dns2] [...] 

networksetup -getsearchdomains <networkservice>

networksetup -setsearchdomains <networkservice> <domain1> [domain2] [...] 

networksetup -create6to4service <newnetworkservicename> 

networksetup -set6to4automatic <networkservice> 

networksetup -set6to4manual <networkservice> <relayaddress> 

networksetup -getftpproxy <networkservice>

networksetup -setftpproxy <networkservice> <domain> <port number> <authenticated> <username> <password>

networksetup -setftpproxystate <networkservice> <on off>

networksetup -getwebproxy <networkservice>

networksetup -setwebproxy <networkservice> <domain> <port number> <authenticated> <username> <password>

networksetup -setwebproxystate <networkservice> <on off>

networksetup -getsecurewebproxy <networkservice>

networksetup -setsecurewebproxy <networkservice> <domain> <port number> <authenticated> <username> <password>

networksetup -setsecurewebproxystate <networkservice> <on off>

networksetup -getstreamingproxy <networkservice>

networksetup -setstreamingproxy <networkservice> <domain> <port number> <authenticated> <username> <password>

networksetup -setstreamingproxystate <networkservice> <on off>

networksetup -getgopherproxy <networkservice>

networksetup -setgopherproxy <networkservice> <domain> <port number> <authenticated> <username> <password>

networksetup -setgopherproxystate <networkservice> <on off>

networksetup -getsocksfirewallproxy <networkservice>

networksetup -setsocksfirewallproxy <networkservice> <domain> <port number> <authenticated> <username> <password>

networksetup -setsocksfirewallproxystate <networkservice> <on off>

networksetup -getproxybypassdomains <networkservice>

networksetup -setproxybypassdomains <networkservice> <domain1> [domain2] [...] 

networksetup -getproxyautodiscovery <networkservice>

networksetup -setproxyautodiscovery <networkservice> <on off>

networksetup -getpassiveftp <networkservice>

networksetup -setpassiveftp <networkservice> <on off>

networksetup -getairportnetwork <device name>

networksetup -setairportnetwork <device name> <network> [password]

networksetup -getairportpower <device name>

networksetup -setairportpower <device name> <on off>

networksetup -listpreferredwirelessnetworks <device name>

networksetup -addpreferredwirelessnetworkatindex <device name> <network> <index> <security type> [password]

networksetup -removepreferredwirelessnetwork <device name> <network>

networksetup -removeallpreferredwirelessnetworks <device name>

networksetup -getnetworkserviceenabled <networkservice>

networksetup -setnetworkserviceenabled <networkservice> <on off>

networksetup -createnetworkservice <newnetworkservicename> <hardwareport>

networksetup -renamenetworkservice <networkservice> <newnetworkservicename>

networksetup -duplicatenetworkservice <networkservice> <newnetworkservicename>

networksetup -removenetworkservice <networkservice>

networksetup -ordernetworkservices <service1> <service2> <service3> <...>

networksetup -getMTU <hardwareport or device name>

networksetup -setMTU <hardwareport or device name> <value>

networksetup -listvalidMTUrange <hardwareport or device name>

networksetup -getmedia <hardwareport or device name>

networksetup -setmedia <hardwareport or device name> <subtype> [option1] [option2] [...]

networksetup -listvalidmedia <hardwareport or device name>

networksetup -createVLAN <VLAN name> <parent device name> <tag>

networksetup -deleteVLAN <VLAN name> <parent device name> <tag>

networksetup -listVLANs

networksetup -listdevicesthatsupportVLAN

networksetup -isBondSupported <hardwareport>

networksetup -createBond <bondname> <hardwareport1> <hardwareport2> <...>

networksetup -deleteBond <bonddevicename>

networksetup -addDeviceToBond <hardwareport> <bonddevicename>

networksetup -removeDeviceFromBond <hardwareport> <bonddevicename>

networksetup -listBonds

networksetup -showBondStatus <bonddevicename>

networksetup -listpppoeservices

networksetup -showpppoestatus <service name ie., MyPPPoEService>

networksetup -createpppoeservice <device name ie., en0> <service name> <account name> <password> [pppoe service name]

networksetup -deletepppoeservice <service name>

networksetup -setpppoeaccountname <service name> <account name>

networksetup -setpppoepassword <service name> <password>

networksetup -connectpppoeservice <service name>

networksetup -disconnectpppoeservice <service name>

networksetup -getcurrentlocation

networksetup -listlocations

networksetup -createlocation <location name> [populate]

networksetup -deletelocation <location name>

networksetup -switchtolocation <location name>

networksetup -listalluserprofiles

networksetup -listloginprofiles <service name>

networksetup -enablesystemprofile <service name> <on off>

networksetup -enableloginprofile <service name> <profile name> <on off>

networksetup -enableuserprofile <profile name> <on off>

networksetup -import8021xProfiles <service name> <file path>

networksetup -export8021xProfiles <service name> <file path> <yes no>

networksetup -export8021xUserProfiles <file path> <yes no>

networksetup -export8021xLoginProfiles <service name> <file path> <yes no>

networksetup -export8021xSystemProfile <service name> <file path> <yes no>

networksetup -settlsidentityonsystemprofile <service name> <file path> <passphrase>

>networksetup -settlsidentityonuserprofile <profile name> <file path> <passphrase>networksetup -deletesystemprofile <service name> 

networksetup -deleteloginprofile <service name> <profile name>

networksetup -deleteuserprofile <profile name>

networksetup -version

networksetup -help

networksetup -printcommands

Tinkering with Tekton - A useful read

As one of my many side-projects, I have been tinkering with Tekton Pipelines on the IBM Z ( s390x ) platform, and am currently looking at at an issue we've been seeing with Tasks and Ubuntu containers.

I found this to be a useful set of readings: -

Jenkins X Pipelines Internals

Jenkins X Pipelines Internals Part 1 — From GitHub WebHook Event to Tekton Pipeline

Jenkins X Pipelines Internals Part 2 — Meta Pipeline

Jenkins X Pipelines Internals Part 3 — Stages

Jenkins X Pipelines Internals Part 4— Steps

For reference, here's one of my earlier posts on Tekton from May: -

Build and deploy a Docker image on Kubernetes using Tekton Pipelines

Tinkering with Docker and relative vs. absolute paths for volume mounting

 A colleague asked how he might use a relative path e.g. ~/foobar or ../foobar instead of an absolute path e.g. /home/Dave or /tmp/foobar when mapping a path on the host to a path inside a container, using Docker for Mac.

Well, I had a play ....

For reference, Docker uses a concept known as bind mounting as evidenced by docker run --help 

  -v, --volume list                    Bind mount a volume

to make a file-system on the host available inside a running container.

I used BusyBox ( busybox ) : -

docker images

REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE

busybox             latest              6858809bf669        5 weeks ago         1.23MB

and tinkered with various options.

Scenario 1 - using a shortcut for /home/<user> specifically ~

echo "Hello World!" > ~/foo.txt

docker run --rm -it --volume ~:/tmp busybox:latest sh

and then validate from inside the container: -

cat /tmp/foo.txt

Hello World!

exit

so that works ....

Scenario 2 - using a shortcut / relative path such as ..

mkdir -p ~/foobar/snafu

echo "Hello World!" > ~/foobar/greeting.txt

docker run --rm -it --volume "..":/tmp busybox:latest sh

docker: Error response from daemon: create ..: ".." includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.

See 'docker run --help'.

Erk, not so good .....

Whilst I can do this: -

docker run --rm -it --volume ~/foobar:/tmp busybox:latest sh

cat /tmp/greeting.txt 

Hello World!

that kinda defeats the object ....

Enter .... realpath

which realpath

/usr/local/bin/realpath

realpath --version

realpath (GNU coreutils) 8.32

Copyright (C) 2020 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law.

Written by Padraig Brady.

realpath --help

Usage: realpath [OPTION]... FILE...

Print the resolved absolute file name;

all but the last component must exist

  -e, --canonicalize-existing  all components of the path must exist

  -m, --canonicalize-missing   no path components need exist or be a directory

  -L, --logical                resolve '..' components before symlinks

  -P, --physical               resolve symlinks as encountered (default)

  -q, --quiet                  suppress most error messages

      --relative-to=DIR        print the resolved path relative to DIR

      --relative-base=DIR      print absolute paths unless paths below DIR

  -s, --strip, --no-symlinks   don't expand symlinks

  -z, --zero                   end each output line with NUL, not newline

      --help     display this help and exit

      --version  output version information and exit

GNU coreutils online help: <https://www.gnu.org/software/coreutils/>

Full documentation <https://www.gnu.org/software/coreutils/realpath>

or available locally via: info '(coreutils) realpath invocation'

I'd heard about realpath from the colleague who'd asked the question in the first place ... so I installed it ....

brew install realpath

and so back to Docker ....

docker run --rm -it --volume `realpath ..`:/tmp busybox:latest sh

cat /tmp/greeting.txt 

Hello World

which works a treat - the trick is to use the shell-within-a-shell trick of enclosing one command inside another, via the double backpack ( ` ) symbol.

So I'm asking realpath where am I ....

realpath ..

/Users/hayd/foobar

and then using the results of that as the from for the Docker bind mount.

Scenario 3 - absolute paths that are absolutely horrible

So even though this wasn't something for which my colleague asked, I wanted to see what happened with horrible paths that include space characters .....

Personally I *HATE* spaces in paths - Microsoft did it with Windows a few decades back, with C:\Program Files\ etc. and that used to play havoc with Java and class paths.

Here's one that IBM Notes uses, for dumps etc.

cd ~/Library/Application\ Support/IBM\ Notes\ Data/IBM_TECHNICAL_SUPPORT/

Within that path, there's a subdirectory: -

SmartUpgrade

This entire path expands out to: -

/Users/hayd/Library/Application Support/IBM Notes Data/IBM_TECHNICAL_SUPPORT/SmartUpgrade

so we test it thusly: -

cd "/Users/hayd/Library/Application Support/IBM Notes Data/IBM_TECHNICAL_SUPPORT/SmartUpgrade

echo "Hello World!" > ./greeting.txt

docker run --rm -it --volume "/Users/hayd/Library/Application Support/IBM Notes Data/IBM_TECHNICAL_SUPPORT/SmartUpgrade":/tmp busybox:latest  sh

cat /tmp/greeting.txt 

Hello World!

Interestingly, the realpath "hack" didn't work: -

docker run --rm -it --volume `realpath .`:/tmp busybox:latest sh

docker: invalid reference format: repository name must be lowercase.

See 'docker run --help'.

docker run --rm -it --volume "`realpath .`":/tmp busybox:latest sh

cat /tmp/greeting.txt

Hello World!

So now we are wrapping the output of `realpath .` in double-quotes ... otherwise, the output of realpath throws back space characters to the upper shell, which breaks docker run .....

What fun!

More on SonarQube - RESTing on my laurels

 Following on from earlier posts: -

Tinkering with SonarQube for code-scanning shell scripts ...

More about SonarQube and scanning Java source code ....

I wanted to dig into the REST APIs that SonarQube provides, even with the freely available Community Edition ...

Noting that SQ happily lists out the various APIs that it supports, from the SQ box itself: -

http://192.168.1.111:9000/web_api/

I had a quick poke using curl and jq as follows: -

curl -s -X GET http://localhost:9000/api/issues/search | jq

    },

    {

      "key": "AXUceByf15xDyMB2TTL0",

      "rule": "java:S106",

      "severity": "MAJOR",

      "component": "com.mycompany.app:my-app:src/main/java/com/mycompany/app/SimpleSSLSocket.java",

      "project": "com.mycompany.app:my-app",

      "line": 102,

      "hash": "fee0b4af1274e112d02a55aa3fea0f44",

      "textRange": {

        "startLine": 102,

        "endLine": 102,

        "startOffset": 8,

        "endOffset": 18

      },

      "flows": [],

      "status": "OPEN",

      "message": "Replace this use of System.out or System.err by a logger.",

      "effort": "10min",

      "debt": "10min",

      "tags": [

        "bad-practice",

        "cert"

      ],

curl -s -X GET http://localhost:9000/api/issues/search | jq '.issues[] | select (.status=="OPEN")'

{

  "key": "AXUceByf15xDyMB2TTLw",

  "rule": "java:S1186",

  "severity": "CRITICAL",

  "component": "com.mycompany.app:my-app:src/main/java/com/mycompany/app/SimpleSSLSocket.java",

  "project": "com.mycompany.app:my-app",

  "line": 188,

  "hash": "5875de20b745c311e5d2270428e9c665",

  "textRange": {

    "startLine": 188,

    "endLine": 188,

    "startOffset": 16,

    "endOffset": 27

  },

  "flows": [],

  "status": "OPEN",

  "message": "Add a nested comment explaining why this method is empty, throw an UnsupportedOperationException or complete the implementation.",

  "effort": "5min",

  "debt": "5min",

  "tags": [

    "suspicious"

  ],

  "creationDate": "2020-10-12T11:00:16+0000",

  "updateDate": "2020-10-12T11:00:16+0000",

  "type": "CODE_SMELL",

  "organization": "default-organization",

  "scope": "MAIN"

}

curl -s -k -X GET http://localhost:9000/api/project_analyses/search?project=DaveSQJava | jq

{

  "paging": {

    "pageIndex": 1,

    "pageSize": 100,

    "total": 4

  },

  "analyses": [

    {

      "key": "AXUnCZ9lsYjbQLEaXKVY",

      "date": "2020-10-14T12:15:36+0000",

      "events": [

        {

          "key": "AXUnCaXisYjbQLEaXKYc",

          "category": "VERSION",

          "name": "not provided"

        }

      ],

      "projectVersion": "not provided",

      "manualNewCodePeriodBaseline": false

    },

    {

      "key": "AXUmhs5wsYjbQLEaXKSN",

      "date": "2020-10-14T09:52:42+0000",

      "events": [

        {

          "key": "AXUmhtX1sYjbQLEaXKVR",

          "category": "QUALITY_GATE",

          "name": "Green (was Red)",

          "description": ""

        }

      ],

      "projectVersion": "not provided",

      "manualNewCodePeriodBaseline": false

    },

    {

      "key": "AXUmgJU7sYjbQLEaXKO8",

      "date": "2020-10-14T09:45:54+0000",

      "events": [],

      "projectVersion": "not provided",

      "manualNewCodePeriodBaseline": false

    },

    {

      "key": "AXUcx98215xDyMB2TTWV",

      "date": "2020-10-12T12:27:34+0000",

      "events": [

        {

          "key": "AXUcx-Wc15xDyMB2TTZi",

          "category": "QUALITY_GATE",

          "name": "Red (was Green)",

          "description": "Reliability Rating on New Code > 1"

        },

        {

          "key": "AXUcx-Wd15xDyMB2TTZj",

          "category": "QUALITY_PROFILE",

          "name": "Use 'FindBugs Security JSP' (JSP)"

        }

      ],

      "projectVersion": "not provided",

      "manualNewCodePeriodBaseline": false

    }

  ]

}

curl -s -k -X GET http://localhost:9000/api/project_analyses/search?project=DaveSQJava | jq '.analyses[].events'

[

  {

    "key": "AXUnCaXisYjbQLEaXKYc",

    "category": "VERSION",

    "name": "not provided"

  }

]

[

  {

    "key": "AXUmhtX1sYjbQLEaXKVR",

    "category": "QUALITY_GATE",

    "name": "Green (was Red)",

    "description": ""

  }

]

[]

[

  {

    "key": "AXUcx-Wc15xDyMB2TTZi",

    "category": "QUALITY_GATE",

    "name": "Red (was Green)",

    "description": "Reliability Rating on New Code > 1"

  },

  {

    "key": "AXUcx-Wd15xDyMB2TTZj",

    "category": "QUALITY_PROFILE",

    "name": "Use 'FindBugs Security JSP' (JSP)"

  }

]

curl -s -k -X GET http://localhost:9000/api/project_analyses/search?project=DaveSQJava | jq '.analyses[].events[] | select(.name|startswith("Red"))'

{

  "key": "AXUcx-Wc15xDyMB2TTZi",

  "category": "QUALITY_GATE",

  "name": "Red (was Green)",

  "description": "Reliability Rating on New Code > 1"

}

curl -s -k -X GET http://localhost:9000/api/project_analyses/search?project=DaveSQJava | jq '.analyses[].events[] | select(.name|startswith("Green"))'

{

  "key": "AXUmhtX1sYjbQLEaXKVR",

  "category": "QUALITY_GATE",

  "name": "Green (was Red)",

  "description": ""

}

VMware Fusion - Toggling back and forth...

 A friend had asked how one can switch out of full-screen whilst running a VM within VMware Fusion on the Mac.

I had a quick dig about, and found this: -

Return to Another View from Full Screen View

which says, in part: -

To see the Mac desktop and the virtual machine desktop at the same time, or to see a guest application in a window on the Mac desktop, switch to Unity view or Single Window view.

To have your virtual machine appear in a single window on the Mac desktop, exit to Single Window view. To display guest application windows directly on your Mac desktop without seeing the virtual machine's desktop, exit to Unity view.

Procedure

    To exit Full Screen to Single Window view, in the Full Screen title bar, select View > Single Window, or use the keyboard shortcut Command key+Control+F.

    To exit Full Screen to Unity view, in the Full Screen title bar, select View > Unity, or use the keyboard shortcut Command key+Shift+U.

Whilst I'm not making use of Unity, the first tip - [command] [control] [f] - did the job nicely. For the record, I'm using VMware Fusion 12.0.0 on macOS Catalina 10.15.7, waiting for macOS Big Sur 11 to arrive .....

Thanks, VMware 

More about SonarQube and scanning Java source code ....

 As per previous posts : -

Tinkering with SonarQube for code-scanning shell scripts ...

Getting to grips with Maven - in five minutes ...

I've been tinkering further with SonarQube (SQ) to scan projects with Java files, both source .java AND compiled .class files.

I was trying to mitigate an issue where SQ or, to be more specific, the FindBugs plugin was complaining about uncompiled source ... in this instance, I've got a project that contains a single .java source file which, for various not-so-interesting reasons, has not been compiled.

This is what I did ...

Run SQ container

docker run -d --name sonarqube -e SONAR_ES_BOOTSTRAP_CHECKS_DISABLE=true -p 9000:9000 sonarqube:latest

Access SQ via browser

http://192.168.1.100:9000/about

Install FindBugs v4.0.1 plugin

http://192.168.1.100:9000/admin/marketplace?search=findbugs

Create Project

mkdir ~/DaveSQJava

Create Java source

vi ~/DaveSQJava/HelloWorld.java

public class HelloWorld

{

    public static void main(String[] args)

    {

        for (int i = 0; i < 5; i++) {

            System.out.println("Hello, World");

        }

    }

}

Scan Project

cd ~/DaveSQJava

sonar-scanner \ -Dsonar.projectKey=DaveSQJava \ -Dsonar.sources=. \ -Dsonar.host.url=http://192.168.1.100:9000 \ -Dsonar.login=2b7d7e9cd8d35baa9d9d5b8f11011bff703e4696

which fails with: -

ERROR: Error during SonarScanner execution

java.lang.IllegalStateException: Can not execute Findbugs

    at org.sonar.plugins.findbugs.FindbugsExecutor.execute(FindbugsExecutor.java:188)

    at org.sonar.plugins.findbugs.FindbugsSensor.execute(FindbugsSensor.java:114)

    at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:48)

    at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:85)

    at org.sonar.scanner.sensor.ModuleSensorsExecutor.lambda$execute$1(ModuleSensorsExecutor.java:59)

    at org.sonar.scanner.sensor.ModuleSensorsExecutor.withModuleStrategy(ModuleSensorsExecutor.java:77)

    at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:59)

    at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:82)

    at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)

    at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)

    at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:393)

    at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:389)

    at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:358)

    at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)

    at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)

    at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:144)

    at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:137)

    at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:123)

    at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:72)

    at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:66)

    at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)

    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

    at java.base/java.lang.reflect.Method.invoke(Unknown Source)

    at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)

    at com.sun.proxy.$Proxy0.execute(Unknown Source)

    at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)

    at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)

    at org.sonarsource.scanner.cli.Main.execute(Main.java:112)

    at org.sonarsource.scanner.cli.Main.execute(Main.java:75)

    at org.sonarsource.scanner.cli.Main.main(Main.java:61)

Caused by: java.lang.IllegalStateException: One (sub)project contains Java source files that are not compiled (/root/DaveSQJava).

    at org.sonar.plugins.findbugs.FindbugsConfiguration.getFindbugsProject(FindbugsConfiguration.java:123)

    at org.sonar.plugins.findbugs.FindbugsExecutor.execute(FindbugsExecutor.java:119)

    ... 31 more

ERROR:

ERROR: Re-run SonarScanner using the -X switch to enable full debug logging.

However, I found some inspiration here: -

sonar-findbugs 3.6 fails when analyzing module with non-compiled JSPs #148

which led me down a series of rabbit holes until I found a configuration option within the FindBugs plugin within the SQ web UI itself: -

Working a hunch, I decided to try sonar.findbugs.allowuncompiledcode as a command-line switch .....

What could possibly go wrong ?

Add -Dsonar.findbugs.allowuncompiledcode switch

sonar-scanner \ -Dsonar.projectKey=DaveSQJava \ -Dsonar.sources=. \ -Dsonar.host.url=http://192.168.1.100:9000 \ -Dsonar.login=2b7d7e9cd8d35baa9d9d5b8f11011bff703e4696 -Dsonar.findbugs.allowuncompiledcode

which....

WORKED !!

INFO: ------------------------------------------------------------------------

INFO: EXECUTION SUCCESS

INFO: ------------------------------------------------------------------------

INFO: Total time: 10.553s

INFO: Final Memory: 17M/60M

INFO: ------------------------------------------------------------------------

I then add a .jsp file into the mix ....

vi ~/DaveSQJava/HelloWorld.jsp

<%@ page language="java" contentType="text/html; charset=ISO-8859-1"

    pageEncoding="ISO-8859-1"%>

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<html>

<head>

<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">

<title>JSP - Hello World Tutorial - Programmer Gate</title>

</head>

<body>

<%= "Hello World!" %>

</body>

</html>

and re-ran the scan: -

sonar-scanner   -Dsonar.projectKey=DaveSQJava   -Dsonar.sources=.   -Dsonar.host.url=http://158.85.5.109:9000   -Dsonar.login=2b7d7e9cd8d35baa9d9d5b8f11011bff703e4696 -Dsonar.findbugs.allowuncompiledcode

INFO: Sensor FindBugs Sensor [findbugs]

WARN: Findbugs needs sources to be compiled. Please build project before executing sonar or check the location of compiled classes to make it possible for Findbugs to analyse your (sub)project (/root/DaveSQJava).

WARN: JSP files were found in the current (sub)project (/root/DaveSQJava) but FindBugs requires their precompiled form. For more information on how to configure JSP precompilation : https://github.com/find-sec-bugs/find-sec-bugs/wiki/JSP-precompilation

INFO: Findbugs analysis skipped for this project.

INFO: Sensor FindBugs Sensor [findbugs] (done) | time=1628ms

INFO: Sensor SurefireSensor [java]

INFO: parsing [/root/DaveSQJava/target/surefire-reports]

INFO: Sensor SurefireSensor [java] (done) | time=3ms

INFO: Sensor JavaXmlSensor [java]

INFO: Sensor JavaXmlSensor [java] (done) | time=2ms

INFO: Sensor HTML [web]

INFO: Sensor HTML [web] (done) | time=159ms

INFO: Sensor VB.NET Properties [vbnet]

INFO: Sensor VB.NET Properties [vbnet] (done) | time=1ms

INFO: ------------- Run sensors on project

INFO: Sensor Zero Coverage Sensor

INFO: Sensor Zero Coverage Sensor (done) | time=11ms

INFO: Sensor Java CPD Block Indexer

INFO: Sensor Java CPD Block Indexer (done) | time=17ms

INFO: SCM Publisher No SCM system was detected. You can use the 'sonar.scm.provider' property to explicitly specify it.

INFO: CPD Executor 1 file had no CPD blocks

INFO: CPD Executor Calculating CPD for 1 file

INFO: CPD Executor CPD calculation finished (done) | time=120ms

INFO: Analysis report generated in 96ms, dir size=84 KB

INFO: Analysis report compressed in 28ms, zip size=13 KB

INFO: Analysis report uploaded in 38ms

INFO: ANALYSIS SUCCESSFUL, you can browse http://158.85.5.109:9000/dashboard?id=DaveSQJava

INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report

INFO: More about the report processing at http://158.85.5.109:9000/api/ce/task?id=AXUcx9l_NFsnENiRRAYU

INFO: Analysis total time: 9.051 s

INFO: ------------------------------------------------------------------------

INFO: EXECUTION SUCCESS

INFO: ------------------------------------------------------------------------

INFO: Total time: 10.724s

INFO: Final Memory: 8M/34M

INFO: ------------------------------------------------------------------------

Even better, the scan actually did scan / review the .jsp ( Java Server Pages ) source, and found some bugs ...

Just to confirm, my project has NO compiled code therein ...

pwd

/root/DaveSQJava

ls -R -al

.:

total 20

drwxr-xr-x  3 root root 4096 Oct 12 12:27 .

drwx------ 15 root root 4096 Oct 12 12:27 ..

drwxr-xr-x  3 root root 4096 Oct 12 12:27 .scannerwork

-rw-r--r--  1 root root  150 Oct 12 10:33 HelloWorld.java

-rw-r--r--  1 root root  404 Oct 12 12:27 HelloWorld.jsp

./.scannerwork:

total 16

drwxr-xr-x 3 root root 4096 Oct 12 12:27 .

drwxr-xr-x 3 root root 4096 Oct 12 12:27 ..

-rw-r--r-- 1 root root    0 Oct 12 10:34 .sonar_lock

-rw-r--r-- 1 root root    0 Oct 12 12:27 class-mapping.csv

drwxr-xr-x 2 root root 4096 Oct 12 12:27 findbugs

-rw-r--r-- 1 root root  246 Oct 12 12:27 report-task.txt

./.scannerwork/findbugs:

total 8

drwxr-xr-x 2 root root 4096 Oct 12 12:27 .

drwxr-xr-x 3 root root 4096 Oct 12 12:27 ..

Job's a good 'un ....

Talking with tree - on the Mac

 Now back in the day, I remember a version of DOS or Windows that introduced the tree command that allowed one to produce a pseudo-graphical listing of a file-system.

This nice picture from Wikipedia shows the concept: -

Well, I'd missed that on macOS ...

Now I miss it no longer ....

'twas a simple installation: -

brew install tree

and now: -

which tree

/usr/local/bin/tree

tree --version

tree v1.8.0 (c) 1996 - 2018 by Steve Baker, Thomas Moore, Francesc Rocher, Florian Sesser, Kyosuke Tokoro 

tree --help

usage: tree [-acdfghilnpqrstuvxACDFJQNSUX] [-H baseHREF] [-T title ]

[-L level [-R]] [-P pattern] [-I pattern] [-o filename] [--version]

[--help] [--inodes] [--device] [--noreport] [--nolinks] [--dirsfirst]

[--charset charset] [--filelimit[=]#] [--si] [--timefmt[=]<f>]

[--sort[=]<name>] [--matchdirs] [--ignore-case] [--fromfile] [--]

[<directory list>]

  ------- Listing options -------

  -a            All files are listed.

  -d            List directories only.

  -l            Follow symbolic links like directories.

  -f            Print the full path prefix for each file.

  -x            Stay on current filesystem only.

  -L level      Descend only level directories deep.

  -R            Rerun tree when max dir level reached.

  -P pattern    List only those files that match the pattern given.

  -I pattern    Do not list files that match the given pattern.

  --ignore-case Ignore case when pattern matching.

  --matchdirs   Include directory names in -P pattern matching.

  --noreport    Turn off file/directory count at end of tree listing.

  --charset X   Use charset X for terminal/HTML and indentation line output.

  --filelimit # Do not descend dirs with more than # files in them.

  --timefmt <f> Print and format time according to the format <f>.

  -o filename   Output to file instead of stdout.

  ------- File options -------

  -q            Print non-printable characters as '?'.

  -N            Print non-printable characters as is.

  -Q            Quote filenames with double quotes.

  -p            Print the protections for each file.

  -u            Displays file owner or UID number.

  -g            Displays file group owner or GID number.

  -s            Print the size in bytes of each file.

  -h            Print the size in a more human readable way.

  --si          Like -h, but use in SI units (powers of 1000).

  -D            Print the date of last modification or (-c) status change.

  -F            Appends '/', '=', '*', '@', '|' or '>' as per ls -F.

  --inodes      Print inode number of each file.

  --device      Print device ID number to which each file belongs.

  ------- Sorting options -------

  -v            Sort files alphanumerically by version.

  -t            Sort files by last modification time.

  -c            Sort files by last status change time.

  -U            Leave files unsorted.

  -r            Reverse the order of the sort.

  --dirsfirst   List directories before files (-U disables).

  --sort X      Select sort: name,version,size,mtime,ctime.

  ------- Graphics options -------

  -i            Don't print indentation lines.

  -A            Print ANSI lines graphic indentation lines.

  -S            Print with CP437 (console) graphics indentation lines.

  -n            Turn colorization off always (-C overrides).

  -C            Turn colorization on always.

  ------- XML/HTML/JSON options -------

  -X            Prints out an XML representation of the tree.

  -J            Prints out an JSON representation of the tree.

  -H baseHREF   Prints out HTML format with baseHREF as top directory.

  -T string     Replace the default HTML title and H1 header with string.

  --nolinks     Turn off hyperlinks in HTML output.

  ------- Input options -------

  --fromfile    Reads paths from files (.=stdin)

  ------- Miscellaneous options -------

  --version     Print version and exit.

  --help        Print usage and this help message and exit.

  --            Options processing terminator.

I can even get a tree in JSON format ....

tree -J

[{"type":"directory","name": ".","contents":[

    {"type":"file","name":"pom.xml"},

    {"type":"directory","name":"src","contents":[

      {"type":"directory","name":"main","contents":[

        {"type":"directory","name":"java","contents":[

          {"type":"directory","name":"com","contents":[

            {"type":"directory","name":"dave","contents":[

              {"type":"directory","name":"app","contents":[

                {"type":"file","name":"App.java"}

              ]}

            ]}

          ]}

        ]}

      ]},

      {"type":"directory","name":"test","contents":[

        {"type":"directory","name":"java","contents":[

          {"type":"directory","name":"com","contents":[

            {"type":"directory","name":"dave","contents":[

              {"type":"directory","name":"app","contents":[

                {"type":"file","name":"AppTest.java"}

              ]}

            ]}

          ]}

        ]}

      ]}

    ]},

    {"type":"directory","name":"target","contents":[

      {"type":"directory","name":"classes","contents":[

        {"type":"directory","name":"com","contents":[

          {"type":"directory","name":"dave","contents":[

            {"type":"directory","name":"app","contents":[

              {"type":"file","name":"App.class"}

            ]}

          ]}

        ]}

      ]},

      {"type":"file","name":"daves-app-1.0-SNAPSHOT.jar"},

      {"type":"directory","name":"generated-sources","contents":[

        {"type":"directory","name":"annotations","contents":[

        ]}

      ]},

      {"type":"directory","name":"generated-test-sources","contents":[

        {"type":"directory","name":"test-annotations","contents":[

        ]}

      ]},

      {"type":"directory","name":"maven-archiver","contents":[

        {"type":"file","name":"pom.properties"}

      ]},

      {"type":"directory","name":"maven-status","contents":[

        {"type":"directory","name":"maven-compiler-plugin","contents":[

          {"type":"directory","name":"compile","contents":[

            {"type":"directory","name":"default-compile","contents":[

              {"type":"file","name":"createdFiles.lst"},

              {"type":"file","name":"inputFiles.lst"}

            ]}

          ]},

          {"type":"directory","name":"testCompile","contents":[

            {"type":"directory","name":"default-testCompile","contents":[

              {"type":"file","name":"createdFiles.lst"},

              {"type":"file","name":"inputFiles.lst"}

            ]}

          ]}

        ]}

      ]},

      {"type":"directory","name":"surefire-reports","contents":[

        {"type":"file","name":"TEST-com.dave.app.AppTest.xml"},

        {"type":"file","name":"com.dave.app.AppTest.txt"}

      ]},

      {"type":"directory","name":"test-classes","contents":[

        {"type":"directory","name":"com","contents":[

          {"type":"directory","name":"dave","contents":[

            {"type":"directory","name":"app","contents":[

              {"type":"file","name":"AppTest.class"}

            ]}

          ]}

        ]}

      ]}

    ]}

  ]},

  {"type":"report","directories":32,"files":13}

]

Do I need that ? No, but who cares !! I CAN!!

Colour me happy 😂😂😂