Wednesday, 31 July 2019

Synology NAS and SSH Ciphers

I've seen this before: -

ssh -i ~/.ssh/id_rsa admin@diskstation

Unable to negotiate with 10.0.0.43 port 22: no matching cipher found. Their offer: aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc

but had forgotten how to debug/solve it.

The NAS, a Synology DS-414, is running: -

DSM 6.2.1-23824 Update 4

and the client is my Mac, running macOS 10.14.6.

This helped: -

ssh error: unable to negotiate with IP: no matching cipher found

which advised running: -

ssh -Q cipher

3des-cbc
aes128-cbc
aes192-cbc
aes256-cbc
rijndael-cbc@lysator.liu.se
aes128-ctr
aes192-ctr
aes256-ctr
aes128-gcm@openssh.com
aes256-gcm@openssh.com
chacha20-poly1305@openssh.com

and then picking one of the ciphers that BOTH the Synology AND the Mac support.

I chose: -

aes256-cbc

as follows: -

ssh -c aes256-cbc -i ~/.ssh/id_rsa admin@diskstation

and was in like Flynn: -

admin@DiskStation:~$ uname -a

Linux DiskStation 3.2.40 #23824 SMP Fri Sep 7 12:49:31 CST 2018 armv7l GNU/Linux synology_armadaxp_ds414
admin@DiskStation:~$ 

For the record, and as I type this, I'm busy updating the Synology to the most recent firmware.

In addition, given this: -


I'm also double-checking my security, in terms of from where one can access the NAS ( hint, ONLY from the LAN, not the WAN ) and also in terms of having a nice long, strong, complex password ......


Or, to put it another way, Patchy McPatchface ( thanks, Bart )

Friday, 26 July 2019

Logging into Docker Hub, or similar container registries, with "special" passwords

Note to self, if your password contains a special character such as dollar ( $ ), ensure that you escape if IF you're putting your password into a Bash environment variable, or passing it as a parameter to a Bash script.

Obviously, NOBODY would do that but .....

So, for a password such as: -

Qp4$$w0rd

it should be escaped thusly: -

Qp4\$\$w0rd

Again, do NOT use passwords in variables or on the CLI ......

Thursday, 25 July 2019

Python - Yet another way to learn ... Edublocks

I heard about this on this week's Digital Planet podcast, and it struck a chord, in part because I've struggled with Python in the past: -

Tech4GoodAwards

Joshua LoweEduBlocks

Finalist category: BT Young Pioneer

So, what exactly is Edublocks, it’s a drag and drop version of the computer coding language Python 3, which allows students to learn the Python syntax with minimal errors, allowing younger children to access Python. This is something that proves a problem around the world, especially in the UK since the introduction of the new computing curriculum back in 2014. The aim of Edublocks is to remove the barriers faced when making the transition from block based programs like Scratch, to the text-based language Python, easier for students and teachers, as presently there is no drop-in solution that bridges this gap.

Something at which to look ......

Monday, 22 July 2019

IBM WebSphere Liberty Profile and IBM DB2 and Docker - An approach

One of my IBM colleagues asked me about an approach for "versioning" DB2 JDBC drivers within a Docker image that's using those drivers to connect to a DB2 database.

He was looking at options for ensure that the drivers can be updated to match the target DB2 runtime, without needing to rebuild the entire WebSphere Liberty Profile image each and every time.

There are a number of approaches, including storing the drivers on an external ( to the image/container ) file-system, using a Docker volume.

As an example, something like this: -

docker run -v /host/directory:/container/directory image

In the absence of volumes, we could configure the container to reach out to an external file/web-server at boot-time ( akin to a bootstrap service )

As an alternative, we considered an approach where the image hosting WebSphere Liberty Profile references the JDBC drivers from a DIFFERENT image when it's built, meaning that we CAN rebuild the Liberty image without a huge amount of impact upon the size/layers of the Liberty image itself.

This approach seemed to work ....

I started by pulling the requisite images: -

docker pull websphere-liberty

docker pull store/ibmcorp/db2_developer_c:11.1.4.4-s390x

and created an environment file to start the DB2 instance within its own container: -

vi ~/.envlist

LICENSE=accept
DB2INSTANCE=db2inst1
DB2INST1_PASSWORD=passw0rd
TO_CREATE_SAMPLEDB=true

and started the DB2 database container: -

db2=`docker run -h db2server --name db2server --restart=always --detach  --privileged=true -p 50000:50000 -p 55000:55000 --env-file ~/.envlist -v /db2data:/database store/ibmcorp/db2_developer_c:11.1.4.4-s390x`

and checked the logs until DB2 came up: -

docker logs $db2 -f

...
07/19/2019 19:10:30     0   0   SQL1063N  DB2START processing was successful.
SQL1063N  DB2START processing was successful.
(*) Starting TEXT SEARCH service ...
CIE00001 Operation completed successfully. 
...

and then logged into the running container: -

docker exec -it $db2 /bin/bash

and switched to the db2inst1 user: -

su - db2inst1

Last login: Mon Jul 22 13:44:17 UTC 2019

and listed the DB directory: -

db2 list db directory

 System Database Directory

 Number of entries in the directory = 1

Database 1 entry:

 Database alias                       = SAMPLE
 Database name                        = SAMPLE
 Local database directory             = /database/config/db2inst1
 Database release level               = 14.00
 Comment                              =
 Directory entry type                 = Indirect
 Catalog database partition number    = 0
 Alternate server hostname            =
 Alternate server port number         =

and connected to the SAMPLE DB: -

db2 connect to sample

   Database Connection Information

 Database server        = DB2/LINUXZ64 11.1.4.4
 SQL authorization ID   = DB2INST1
 Local database alias   = SAMPLE

and query the EMPLOYEE table: -

db2 'select * from employee'

EMPNO  FIRSTNME     MIDINIT LASTNAME        WORKDEPT PHONENO HIREDATE   JOB      EDLEVEL SEX BIRTHDATE  SALARY      BONUS       COMM       
------ ------------ ------- --------------- -------- ------- ---------- -------- ------- --- ---------- ----------- ----------- -----------
000010 CHRISTINE    I       HAAS            A00      3978    01/01/1995 PRES          18 F   08/24/1963   152750.00     1000.00     4220.00
000020 MICHAEL      L       THOMPSON        B01      3476    10/10/2003 MANAGER       18 M   02/02/1978    94250.00      800.00     3300.00
000030 SALLY        A       KWAN            C01      4738    04/05/2005 MANAGER       20 F   05/11/1971    98250.00      800.00     3060.00
000050 JOHN         B       GEYER           E01      6789    08/17/1979 MANAGER       16 M   09/15/1955    80175.00      800.00     3214.00
000060 IRVING       F       STERN           D11      6423    09/14/2003 MANAGER       16 M   07/07/1975    72250.00      500.00     2580.00
000070 EVA          D       PULASKI         D21      7831    09/30/2005 MANAGER       16 F   05/26/2003    96170.00      700.00     2893.00
000090 EILEEN       W       HENDERSON       E11      5498    08/15/2000 MANAGER       16 F   05/15/1971    89750.00      600.00     2380.00
000100 THEODORE     Q       SPENSER         E21      0972    06/19/2000 MANAGER       14 M   12/18/1980    86150.00      500.00     2092.00
000110 VINCENZO     G       LUCCHESSI       A00      3490    05/16/1988 SALESREP      19 M   11/05/1959    66500.00      900.00     3720.00
000120 SEAN                 O'CONNELL       A00      2167    12/05/1993 CLERK         14 M   10/18/1972    49250.00      600.00     2340.00
000130 DELORES      M       QUINTANA        C01      4578    07/28/2001 ANALYST       16 F   09/15/1955    73800.00      500.00     1904.00
000140 HEATHER      A       NICHOLLS        C01      1793    12/15/2006 ANALYST       18 F   01/19/1976    68420.00      600.00     2274.00
000150 BRUCE                ADAMSON         D11      4510    02/12/2002 DESIGNER      16 M   05/17/1977    55280.00      500.00     2022.00
000160 ELIZABETH    R       PIANKA          D11      3782    10/11/2006 DESIGNER      17 F   04/12/1980    62250.00      400.00     1780.00
000170 MASATOSHI    J       YOSHIMURA       D11      2890    09/15/1999 DESIGNER      16 M   01/05/1981    44680.00      500.00     1974.00
000180 MARILYN      S       SCOUTTEN        D11      1682    07/07/2003 DESIGNER      17 F   02/21/1979    51340.00      500.00     1707.00
000190 JAMES        H       WALKER          D11      2986    07/26/2004 DESIGNER      16 M   06/25/1982    50450.00      400.00     1636.00
000200 DAVID                BROWN           D11      4501    03/03/2002 DESIGNER      16 M   05/29/1971    57740.00      600.00     2217.00
000210 WILLIAM      T       JONES           D11      0942    04/11/1998 DESIGNER      17 M   02/23/2003    68270.00      400.00     1462.00
000220 JENNIFER     K       LUTZ            D11      0672    08/29/1998 DESIGNER      18 F   03/19/1978    49840.00      600.00     2387.00
000230 JAMES        J       JEFFERSON       D21      2094    11/21/1996 CLERK         14 M   05/30/1980    42180.00      400.00     1774.00
000240 SALVATORE    M       MARINO          D21      3780    12/05/2004 CLERK         17 M   03/31/2002    48760.00      600.00     2301.00
000250 DANIEL       S       SMITH           D21      0961    10/30/1999 CLERK         15 M   11/12/1969    49180.00      400.00     1534.00
000260 SYBIL        P       JOHNSON         D21      8953    09/11/2005 CLERK         16 F   10/05/1976    47250.00      300.00     1380.00
000270 MARIA        L       PEREZ           D21      9001    09/30/2006 CLERK         15 F   05/26/2003    37380.00      500.00     2190.00
000280 ETHEL        R       SCHNEIDER       E11      8997    03/24/1997 OPERATOR      17 F   03/28/1976    36250.00      500.00     2100.00
000290 JOHN         R       PARKER          E11      4502    05/30/2006 OPERATOR      12 M   07/09/1985    35340.00      300.00     1227.00
000300 PHILIP       X       SMITH           E11      2095    06/19/2002 OPERATOR      14 M   10/27/1976    37750.00      400.00     1420.00
000310 MAUDE        F       SETRIGHT        E11      3332    09/12/1994 OPERATOR      12 F   04/21/1961    35900.00      300.00     1272.00
000320 RAMLAL       V       MEHTA           E21      9990    07/07/1995 FIELDREP      16 M   08/11/1962    39950.00      400.00     1596.00
000330 WING                 LEE             E21      2103    02/23/2006 FIELDREP      14 M   07/18/1971    45370.00      500.00     2030.00
000340 JASON        R       GOUNOT          E21      5698    05/05/1977 FIELDREP      16 M   05/17/1956    43840.00      500.00     1907.00
200010 DIAN         J       HEMMINGER       A00      3978    01/01/1995 SALESREP      18 F   08/14/1973    46500.00     1000.00     4220.00
200120 GREG                 ORLANDO         A00      2167    05/05/2002 CLERK         14 M   10/18/1972    39250.00      600.00     2340.00
200140 KIM          N       NATZ            C01      1793    12/15/2006 ANALYST       18 F   01/19/1976    68420.00      600.00     2274.00
200170 KIYOSHI              YAMAMOTO        D11      2890    09/15/2005 DESIGNER      16 M   01/05/1981    64680.00      500.00     1974.00
200220 REBA         K       JOHN            D11      0672    08/29/2005 DESIGNER      18 F   03/19/1978    69840.00      600.00     2387.00
200240 ROBERT       M       MONTEVERDE      D21      3780    12/05/2004 CLERK         17 M   03/31/1984    37760.00      600.00     2301.00
200280 EILEEN       R       SCHWARTZ        E11      8997    03/24/1997 OPERATOR      17 F   03/28/1966    46250.00      500.00     2100.00
200310 MICHELLE     F       SPRINGER        E11      3332    09/12/1994 OPERATOR      12 F   04/21/1961    35900.00      300.00     1272.00
200330 HELENA               WONG            E21      2103    02/23/2006 FIELDREP      14 F   07/18/1971    35370.00      500.00     2030.00
200340 ROY          R       ALONZO          E21      5698    07/05/1997 FIELDREP      16 M   05/17/1956    31840.00      500.00     1907.00

  42 record(s) selected.


Having validated that the DB2 container was clean-and-green, I then proceeded to download a recent set of DB2 JDBC drivers from here: -


and extracted the relevant JARs to a newly created subdirectory: -

mkdir /db2jars
tar xzvf /tmp/v11.1.4fp4_jdbc_sqlj.tar.gz -C /tmp
unzip /tmp/jdbc_sqlj/db2_db2driver_for_jdbc_sqlj.zip -d /tmp
cp /tmp/db2jcc.jar /db2jars
cp /tmp/db2jcc4.jar /db2jars

and then created a TAR file containing those JARs: -

tar cvf dependency.tar /db2jars

I then created a Dockerfile: -

vi Dockerfile.DB2

FROM scratch
ADD dependency.tar /

and built an image: -

docker build -t db2jars:latest -f ~/Dockerfile.DB2 .

Sending build context to Docker daemon  1.459GB
Step 1/2 : FROM scratch
 ---> 
Step 2/2 : ADD dependency.tar /
 ---> d7cfd446014e
Successfully built d7cfd446014e
Successfully tagged db2jars:latest

which gave me a Docker image: -

docker images

REPOSITORY                                            TAG                 IMAGE ID            CREATED             SIZE
db2jars                                               latest              69deb549f3f0        5 days ago          8.05MB

I then created a second Dockerfile, for WebSphere Liberty Profile: -

vi ~/Dockerfile.WLP

FROM websphere-liberty:latest
ENV LICENSE accept

COPY --from=db2jars:latest /db2jars /db2jars
COPY JdbcTestDB2.class /
CMD ["java","-cp","/:/db2jars/db2jcc.jar","JdbcTestDB2","172.18.10.190","50000","sample","db2inst1 ","passw0rd"]

and built the image: -

docker build -t wlp:latest -f ~/Dockerfile.WLP .

Sending build context to Docker daemon  1.459GB
Step 1/5 : FROM websphere-liberty:latest
 ---> 5005e127f3b4
Step 2/5 : ENV LICENSE accept
 ---> Using cache
 ---> 4fb20054c1b4
Step 3/5 : COPY --from=db2jars:latest /db2jars /db2jars
 ---> Using cache
 ---> 7e7ba23d46d4
Step 4/5 : COPY JdbcTestDB2.class /
 ---> Using cache
 ---> ffe1ef5dca2c
Step 5/5 : CMD ["java","-cp","/:/db2jars/db2jcc.jar","JdbcTestDB2","172.18.10.190","50000","sample","db2inst1 ","Qp455w0rd"]
 ---> Using cache
 ---> 490fac5dabe4
Successfully built 490fac5dabe4
Successfully tagged wlp:latest

I then instantiated the container: -

wlp=`docker run -d -t -p 80:9080 -p 443:9443 wlp:latest`

and checked the logs: -

docker logs $wlp -f

000010 CHRISTINE HAAS
000020 MICHAEL THOMPSON
000030 SALLY KWAN
000050 JOHN GEYER
000060 IRVING STERN
000070 EVA PULASKI
000090 EILEEN HENDERSON
000100 THEODORE SPENSER
000110 VINCENZO LUCCHESSI
000120 SEAN O'CONNELL
000130 DELORES QUINTANA
000140 HEATHER NICHOLLS
000150 BRUCE ADAMSON
000160 ELIZABETH PIANKA
000170 MASATOSHI YOSHIMURA
000180 MARILYN SCOUTTEN
000190 JAMES WALKER
000200 DAVID BROWN
000210 WILLIAM JONES
000220 JENNIFER LUTZ
000230 JAMES JEFFERSON
000240 SALVATORE MARINO
000250 DANIEL SMITH
000260 SYBIL JOHNSON
000270 MARIA PEREZ
000280 ETHEL SCHNEIDER
000290 JOHN PARKER
000300 PHILIP SMITH
000310 MAUDE SETRIGHT
000320 RAMLAL MEHTA
000330 WING LEE
000340 JASON GOUNOT
200010 DIAN HEMMINGER
200120 GREG ORLANDO
200140 KIM NATZ
200170 KIYOSHI YAMAMOTO
200220 REBA JOHN
200240 ROBERT MONTEVERDE
200280 EILEEN SCHWARTZ
200310 MICHELLE SPRINGER
200330 HELENA WONG
200340 ROY ALONZO

Magic has occurred ....

For the record, the reason that this happens is in the last line of Dockerfile.WLP: -

...
CMD ["java","-cp","/:/db2jars/db2jcc.jar","JdbcTestDB2","172.18.10.190","50000","sample","db2inst1 ","passw0rd"]
...

which uses this Java class: -

JdbcTestDB2.java 

import java.sql.Connection ;
import java.sql.DriverManager ;
import java.sql.ResultSet ;
import java.sql.Statement ;
import java.sql.SQLException;

class JdbcTestDB2
{
public static void main (String args[])
{
try
{
Class.forName("com.ibm.db2.jcc.DB2Driver");
}
catch (ClassNotFoundException e)
{
System.err.println (e) ;
System.exit (-1) ;
  }
  String hostname      = args[0];
  String port          = args[1];
  String dbName        = args[2];
  String userName      = args[3];
  String password      = args[4];
  String sslConnection = "false";

  java.util.Properties properties = new java.util.Properties();
  properties.put("user",userName);
  properties.put("password", password);

String url = "jdbc:db2://" + hostname + ":" + port + "/" + dbName;
try
{
      Connection connection = DriverManager.getConnection(url,properties);

      String query = "select EMPNO,FIRSTNME,LASTNAME from DB2INST1.EMPLOYEE" ;

      Statement statement = connection.createStatement () ;
ResultSet rs = statement.executeQuery (query) ;

while ( rs.next () )
System.out.println (rs.getString (1) + " " + rs.getString(2) + " " + rs.getString(3)) ;
connection.close () ;
}
  catch (java.sql.SQLException e)
{
System.err.println (e) ;
System.exit (-1) ;
}
}
}

to connect to the DB2 container, using the IP address of the box hosting the container, on port 50000 ( which is mapped from container to host when we start the DB2 container, via -p 50000:50000 

So, when I'm ready to rev the DB2 drivers, I merely need to repeat the above steps to download them, TAR them up, create a new Docker image, and rebuild the WLP image, without making a huge impact upon the size / layers of the image itself.

As I said at the beginning, this is ONE way of solving the problem BUT FUN!!!!

Thursday, 18 July 2019

Jenkins and the Case of the Missing Body

I was repeatedly seeing this: -

java.lang.IllegalStateException: There is no body to invoke

with a Jenkins Pipeline that I was executing; this Pipeline executes whenever one commits new code into a GitHub Enterprise (GHE) repository, with a Pull Request.

To debug this further, I created a dummy GHE repository with a corresponding Jenkinsfile, and a new Jenkins pipeline.

This allowed me to hack iterate the code in the GHE web UI, and immediately test the Pipeline within Jenkins itself.

Without wishing to give away the plot, I'll TL;DR; and say that the problem was ME ( quelle surprise ).

Here's my initial Jenkinsfile: -

timestamps
{
    node('linuxboxen')
    checkout scm
    def givenName = "Dave"
    def familyName = "Hay"
    
    withEnv(["GIVEN_NAME=${givenName}", "FAMILY_NAME=${familyName}"])
    {
        stage('JFDI')
        {
            sh '''#!/bin/bash
            echo "Doing it"
            echo $GIVEN_NAME
            echo $FAMILY_NAME
            '''
        }
    }
}

Can you see the problem ?

It took me a while ....

The node directive is NOT followed by a set of braces, meaning that nothing actually gets done, hence the exception.

The code SHOULD look like this: -

timestamps
{
    node('linuxboxen')
    {
        checkout scm
        def givenName = "Dave"
        def familyName = "Hay"
    
        withEnv(["GIVEN_NAME=${givenName}", "FAMILY_NAME=${familyName}"])
        {
            stage('JFDI')
            {
                sh '''#!/bin/bash
                echo "Doing it"
                echo $GIVEN_NAME
                echo $FAMILY_NAME
                '''
            }
        }
    }
}

In other words, the node() directive needs something to do, hence the need for the braces, which can contain one or more stages(), plus associated directives.

Nice :-)

Tuesday, 16 July 2019

Containers: A Complete Guide

I found this whilst looking for something completely different: -

Containers: A Complete Guide

This guide looks at the importance of containers in cloud computing, highlighting the benefits and showing how containers figure into such technologies as Docker, Kubernetes, Istio, VMs, and Knative.

Quite a nice little introduction ...

Monday, 15 July 2019

Shelling out - fun with Ubuntu shells

I saw this: -

-sh: 2: [: -gt: unexpected operator
-sh: 29: [: -gt: unexpected operator

when logging into an Ubuntu boxen.

I was pretty sure that this'd worked before, but wondered whether my shell was giving me (s)hell ....

I checked what I was currently running: -

echo $SHELL

/bin/sh

which is a flavour of the Bourne Again SHell ( BASH ).

I then checked the /etc/passwd file: -

cat /etc/passwd

hayd:x:12039:12039::/home/hayd:

and realised that I didn't have an explicit shell set.

I upped my authority ( super user do ): -

sudo bash

[sudo] password for hayd: 

and then updated my account: -

usermod --shell /bin/bash hayd

Now /etc/passwd looks OK: -

hayd:x:12039:12039::/home/hayd:/bin/bash

and I'm now all good to go: -

echo $SHELL

/bin/bash


Friday, 12 July 2019

Intro Guide to Dockerfile Best Practices

Not sure how I found this ( it MAY have been Twitter ), but this is rather useful: -

Intro Guide to Dockerfile Best Practices

especially whilst I've been automating the build of Docker images via Jenkins pipelines.

Definitely a few tips to try, such as: -

Tip #4: Remove unnecessary dependencies

Remove unnecessary dependencies and do not install debugging tools. If needed debugging tools can always be installed later. Certain package managers such as apt, automatically install packages that are recommended by the user-specified package, unnecessarily increasing the footprint. Apt has the –no-install-recommends flag which ensures that dependencies that were not actually needed are not installed. If they are needed, add them explicitly.

Go read !

Now Available - IBM Cloud Hyper Protect Virtual Servers

I'm pleased to see one of the IBM Z offerings upon which my Squad are working is now available in the IBM Cloud Experimental Services section of the IBM Cloud Catalog: -




Hyper protect line of virtual servers service leveraging the unparalleled security and reliability of Secure Service Containers on IBM Z.

Features

Security

Ability to deploy a Virtual Server in a Secure Service Container ensuring confidentiality of data and code running within the VS

Z Capabilities on the cloud

Ability to deploy workload into the most secure, highly performant, Linux virtual server with extreme vertical scale

Easy to use, open, and flexible

User experience at parity with market leaders both when buying and using the VS; with the openness and flexibility of a public cloud

No Z skills required

Access Z technology without having to purchase, install, and maintain unique hardware

IBM Cloud Hyper Protect Virtual Servers

Yay us!

Friday, 5 July 2019

Book Review - Left To Our Own Devices, by Margaret E Morris

As mentioned previously, I've been writing a series of book reviews for the British Computer Society (BCS), including: -

Book Review - You'll See This Message When It Is Too Late - The Legal and Economic Aftermath of Cybersecurity Breaches

Rails, Angular, Postgres, and Bootstrap - A Book Review

Kubernetes Microservices with Docker - A Book Review

Book Review - Mastering Puppet Second Edition by Thomas Uphill

etc.

So here's the most  recent review - as before, for full disclosure, I must mention that BCS kindly provided me with a free hardcopy of the book, albeit a review version: -

Left To Our Own Devices, by Margaret E Morris

https://mitpress.mit.edu/books/left-our-own-devices

If nothing else, the title of this book intrigued me, in part because it reminded me of a Pet Shop Boys track from my youth. More seriously, the subtitle of the book: -

Outsmarting smart technology to reclaim our relationships, health and focus

resonated with a lot of recent media coverage about the impacts, both real and perceived, both positive and negative, of information technology in the modern era.

Whilst I don't claim to have strong opinions about the topic, or be particularly well-informed, apart from as a consumer, I have given thought to my family's use of mobile devices, Internet of Things gadgets, so-called smart home technology etc.

I'd especially considered limits on screen time, impact on sleep patterns, exposure to sources of news, including social media, and my tendency to live in a bubble, self-selecting news and opinions that mirror my own.

Therefore, this book came at precisely the right time, and opened my eyes to a number of use cases of technology, including smart lighting, health tracking ( including the so-called Quantified Self ), social media and messaging, technology as an art-form, self-identity, including gender and sexuality, and technology as a therapist.

Ms Morris illustrates each chapter, of which there eight, with a large number of individual user stories, taking inspiration and insight from real people, who allow her to share how they use technology, mainly for the positive, but with thought and insight.

Despite the title, and the subtitle, I found the book to be a very positive read; whilst there are definitely shortcomings to an over-use and over-reliance upon technology, the book shows how humans do manage to mostly outsmart their smart technology, and get from it what they need, whether or not that's what the original inventor intended.

I didn't come away with a list of Do's and Don'ts, but a better understanding of how, and why, people choose to use certain technologies, and, therefore, how I can evaluate my own use, and be more qualitative in my choice of technologies.

In conclusion, I strongly recommend this book, it's a relatively short read, coming in ~130 pages, and is a high enough level that one doesn't need to be a total geek to get the points raised, whether or not one is a total geek.

Out of 10, I'd give this book 10, mainly for completeness, brevity and for the all-important human touch.

Thursday, 4 July 2019

Docker Registries and Repositories - Is there a difference ? ( Hint, yes, there really is )

This came up in discussion today, and one of my colleagues pointed me here: -

Difference between Docker registry and repository

Docker registry is a service that is storing your docker images.

Docker registry could be hosted by a third party, as public or private registry, like one of the following registries:

    Docker Hub,
    Quay,
    Google Container Registry,
    AWS Container Registry

or you can host the docker registry by yourself
(see https://docs.docker.com/docker-trusted-registry/ for more details).

Docker repository is a collection of different docker images with same name, that have different tags. Tag is alphanumeric identifier of the image within a repository.

For example see https://hub.docker.com/r/library/python/tags/. There are many different tags for the official python image, these tags are all members of the official python repository on the Docker Hub. Docker Hub is a Docker Registry hosted by Docker.

To find out more read:

    https://docs.docker.com/registry/
    https://github.com/docker/distribution

IBM Cloud also helped me here, in that I have an IBM Cloud Container Registry service, aka ICCR, within which I have access to several Repositories, and the ICCR UI helpfully tells me: -

A repository is a set of related images with the same name, but different tags.



which is, as they say, nice 😂

Monday, 1 July 2019

Bash and a sufficiency of input parameters

I hit an interesting quirk in Bash earlier today; I'm passing in a list of command-line parameters to a Bash script, using the $1, $2 etc. input parameter method.

However, I noticed that the TENTH parameter failed, and I ended up with a trailing zero on the end of a string that was actually the FIRST parameter.

It appeared that Bash was stopping at 9, and then simply adding the character '0' to the end of the string provided as the FIRST parameter.

Here's an example: -

#!/bin/bash

export A=$1
export B=$2
export C=$3
export D=$4
export E=$5
export F=$6
export G=$7
export H=$8
export I=$9
export J=$10

echo $J

When I execute the script: -

~/foo.sh 1 2 3 4 5 6 7 8 9 0

I'd expect to see this: -

The tenth parameter is 0

whereas I actually saw this: -

The tenth parameter is 10

As ever, the internet came to the rescue: -


which said, in part: -

...
Use curly braces to set them off:

echo "${10}"
...

I updated my script: -

#!/bin/bash

export A=${1}
export B=${2}
export C=${3}
export D=${4}
export E=${5}
export F=${6}
export G=${7}
export H=${8}
export I=${9}
export J=${10}

echo "The tenth parameter is" $J

and now it works as expected: -

The tenth parameter is 0

To be fair, the article also said: -

...
You can also iterate over the positional parameters like this:

for arg

or

for arg in "$@"

or

while (( $# > 0 ))    # or [ $# -gt 0 ]
do
    echo "$1"
    shift
done
...

which I should definitely try .........

IBM Cloud Vulnerability Advisor - Poking the Endpoint

I've been using the Vulnerability Advisor (VA) tool to automate the testing of my built Docker images, looking for code vulnerabilities, scanning against the IBM X-Force database and known Common Vulnerabilities and Exposures (CVE) issues.

This is a nifty feature of the IBM Cloud Container Registry and provides both web UI *AND* command-line interface (CLI) options, which is super-good.

However, I'd not really looked at the REST APIs that VA provides, as documented here: -

Vulnerability Advisor for IBM Cloud Container Registry

Talking with a colleague, I realised that one can leverage API calls such as Report, which returns a JSON payload comprising ALL of the images "owned" by that IBM Cloud account, with an indication of status e.g. OK, UNSUPPORTED, FAIL, plus details of vulnerabilities, configuration issues etc.

So it's the same information that's available via the web UI and the CLI, but available for programmatic consumption ....

To consume this, one needs to pass in HTTP headers such as Account ( which IBM account is being targeted ) and Authorization ( a Bearer token ).

The first is retrieved via the command: -

bx iam accounts

which returns a list of Account GUIDs, plus the Name, Status and Owner.

The second is retrieved by the command: -

bx iam oauth-tokens

which returns a nice long string of apparent gibberish which is actually one's auth token.

Armed with the account ( which should be 32 hex characters ) and the auth token ( which should be 1074 characters ), one can hit the API endpoint.

This is the cURL command - other REST clients are available: -

curl -X GET \
  https://us.icr.io/va/api/v3/report/account \
  -H 'Accept: */*' \
  -H 'Account: db52f980f8c07a05b50cb223fae0d849' \
  -H 'Authorization: Bearer xdx60o64BxVzavTaO8ayNRZCwE4zavFdMbHh0DHh6sB2JKqqTENlQmBDVoBJdOwgQq5eYgUQiikG1KSJs6MuUGyIy2tdQD7eZ0DzIhj6YBX70V8UY0rwLjX0CEfV3UMIOHaGT1yZE7MRLy8iCLn0BOo20rI7vBqgjfJfQlJpd74o5fAkw6oppqchNRFW99GOtAJgGWxzULhx1hsG99ejHZNA7631uJ693whc1gdhp2xw46xQ0g66vGYMLKlyOMUqbm3aM9QoBJulDAp2k8TKdzy589PUafXcIKY4m3nFREAMdr6s8e6bktzkJLtFvqBPGejcaYSLXmnIuP7clOaGvMwwUd2k6KQkpeLPqqmnWP11nNnoMDAiFTnHsPSd74dXtZ4jpfb6co6fXRXmL1Nd2hqfXpVGCbn4wfHWhxbTZhVJs729UJjoZtJGZqdhRLQgrSqzO4jDre0WFfGy0JKsgTRJd37W92d2lbYyiX4Gze9rkSawPhMuRsHw9W6JwAZtqglOIweqy2AcwD3f599k2oEXhZ7LaXPUSlyJQtmTtE4VMBualv3R3zE3Q2wQrPzGLqrFHo30TdNeXDA99o5WVGxSb1QAs9oSWia3sLIr0c6IAdxBPSVv7Xb7aSnlTyB4EII4RKi0GiHWn1f4tJzeBSC9FMNCFvla7utslQB3lMPpA0SZprIa2FuQ9gdy6FYwhe8nzKlBLlCdcqGj7UG2S0m4YIdvtrxT8WDe0JiNY2kQtyZYOlF6tqDdGZNZpYjOvoWZLercfT71bKd3zpAkXcY9XK9G32CpZewBvPvFETzQj7mkdJOQirKY0ZADrUasniu1KgrrcirJt0TqnDRpkxOk23cDoLfs6utxEx3nxNeKqL0sxQdOtLjUsNCh0heranPqbUfV5DaWt4oHLBULOfTBpdcKLeOvIYl1khH9UnzW82fk7FrINp40vwUti5euXHsX2bbkfk0i0JG1qm7ynJY4ZatX4j6B4wglK4KRXeYql7wffRAq10fvbgrgKgBqrUsTHjuprElrFHHkpAS6jKvDmCquHD1ePLFA0IVOdAO407Apbe
' \

and wait for a nice long list of images and their vulnerabilities ....


Ubuntu and Docker - Handling the GUI

Long story very short, I've been building some Docker images from an existing parent image. As part of my build, I wanted to ensure th...