Saturday, 25 May 2019

Setting Authorization Headers in IBM API Connect Test and Monitor

I've recently started exploring the IBM API Connect Test and Monitor tool on IBM Cloud here: -

https://www.ibm.com/cloud/api-connect/api-test

and was having some fun n' games setting an HTTP Header, using either the basic HTTP Client or the more detailed Test Composer.

My API requires that one send an IAM token, rather than the more Basic Auth of user/password, which is nice :-)

This kept failing with HTTP403 Unauthorized.

Thankfully IBM have a nice active support community on Stack Overflow, so I posted a question here: -

Setting Authorization Headers in IBM API Connect Test and Monitor

and got some rapid feedback from the community.

In essence, I was doing it wrong ....

This is what I needed to do: -




In other words, for the Key/Value Pair (KVP) that comprises the HTTP Header field, I needed to specify Authorization as the key and Bearer XXXXXXXXXXXXXXX as the value ( with a space between the word Bearer and the actual IAM token itself.

So I was holding it wrong .... #Doofus

Apart from the basic HTTP Client, there's also a very spiffing Test Composer, that has the same requirement wrt Authorisation headers: -



For the record, there's also a GitHub project for a native client here: -

https://ibm-apiconnect.github.io/test-and-monitor/

with which I've played a wee bit .....

Wednesday, 15 May 2019

LinuxONE for Dummies


As more companies transform their infrastructures with hybrid cloud services, they require environments that protect the safety of their intellectual property, such as data and business rules.

LinuxONE is a hardware system designed to support and exploit the Linux operating system based on the value of its unique underlying architecture. In this book, you’ll learn how to

• Secure your data and spark innovation

• Leverage a large partner ecosystem

• Dynamically scale to meet your business needs

Download now to discover an enterprise-grade Linux server with a unique architecture.

LinuxONE for Dummies

Thursday, 9 May 2019

Postman on macOS - Where are my Windows ?

Having used Postman for the past few months, mainly to poke REST APIs, I hit an interesting issue last night, whereby the main Postman window refused to appear even though I'd left it open when I suspended my Mac ( closed the lid ) before leaving the office.

Now, for clarity, at work I use an external Lenovo monitor, attached via USB-C / HDMI ( using the Apple dongle ), which is positioned to the right of the MacBook Pro itself.

When I got home, I plugged in a different AOC monitor, again via USB-C / HDMI ( using a different non-Apple dongle ), again with the MBR on the left of the monitor ( "On your left" ).

Despite stopping/starting Postman multiple times, and fiddling around with the various options available on the View and Window menus, I couldn't get the main window to appear ....

Thankfully, the internet brought me to this ....

Postman opens off screen(s) when display monitors are changed #2833

which said, in part: -

...
For the moment, if you guys see this, you can reset the app's window settings manually. To do this, you'll need to delete the requester.json file from the app's data directory, this is located:

on macOS: ~/Library/Application\ Support/Postman
on Windows: C:\Users\AppData\Roaming\Postman
on Linux: ~/.config/Postman
...

and: -

...
In fact, you also need to delete the window file in the same directory.
...

Having quit Postman, I dived into Terminal ( because, Terminal ) and did this: -

rm ~/Library/Application\ Support/Postman/storage/requester.json 
rm ~/Library/Application\ Support/Postman/window 

and started Postman again .... c'est voila, here's a Postman window :-)

For the record, I'm using Postman 7.0.9, which is interesting 'cos that post is circa 2017 ....

Tuesday, 30 April 2019

Go and Glide - Problems with Update - Cannot detect VCS

I saw this today: -

glide update

which returns: -

[WARN] Unable to checkout crypto/tls
[ERROR] Error looking for crypto/tls: Cannot detect VCS

I'm using Glide 0.13.2, and Go versions 1.10.6 and 1.11.5.

Using the more detailed glide --debug update, I found a wee bit more detail: -

[DEBUG] ImportDir error on /Users/hayd/.glide/cache/src/https-crypto-tls: cannot find package "." in:
/Users/hayd/.glide/cache/src/https-crypto-tls

*BUT* this did lead me to look at my Git source folder -  $GOPATH/src - which, amongst other things had this: -

drwxr-xr-x   3 hayd  staff   96 29 Mar 19:31 crypto

which was a directory containing a sinmgle empty subdirectory: -

drwxr-xr-x  2 hayd  staff   64 29 Mar 19:33 tls

So this kinda tied up with the symptom shown up in the --debug trace i.e. cannot find package "." even though it was in a completely different place.

Once I did rm -Rf $GOPATH/src/crypto, the glide update worked a treat.

I'd previously gone down a rabbit hole with glide clear-cache and glide mirror, both of which were poisson rouge.

Hope this helps others in the same situation :-)

Friday, 26 April 2019

Input validation of REST requests using GoLang

This is an ongoing W-I-P, as I look at a set of RESTful services that I'm co-developing using Go ( aka GoLang ).

Two things that have been of immense use are: -

Validating Data Structures And Variables In Golang

Regular Expressions 101

the latter of which has been very useful in terms of creating/testing/validating Regular Expressions ( aka RegExp ).

For me, the key thing is to be able to create validation rules within the JSON data structures, and then use the Go-Validator GoLang module/plugin/add-on: -

https://github.com/go-validator/validator

Specifically, I'm using the latest version 9 ( v9 ): -

"gopkg.in/go-playground/validator.v9"

Loving my job :-)

Tuesday, 23 April 2019

The Modern Mainframe Developer Hands On Drop-In Centre - Thursday, May 2, 2019

Come along at any time, grab a machine and away you go. Refreshments and snacks will be provided. This first meetup is funded by IBM.

The following labs will be provided:

> Lab: IBM Eclipse Mainframe Development for z Systems
> Lab: REST APIs for Mainframe Applications Using z/OS Connect
> Lab: Git for Mainframe Applications
> Lab: Mainframe Code Analyzer Tools

The Modern Mainframe Developer Hands On Drop-In Centre

Sunday, 21 April 2019

MainframerZ meetup at Mediaocean, London - Thursday 20 June 2019


Come along to meet other Z professionals, and grow your network. Listen to a series of lightning talks from a range of people currently working in the Z space, and help shape the future of this group.

The lightning talks will be 10 minutes each covering a broad range of topics.

Tentative Agenda

6:15 - 6:45 Arrival and registration
7:00 - 7:10 Introductions
7:10 - 8:10 Lightning talks
8:10 - 8:45 Pizza and networking
8:45 - 9:20 Discussion
9:20 - 9:30 Wrap up

I'm honoured to be one of the speakers for the Lightning talks, so it'd be awesome to see you there, and meet IRL

MainframerZ meetup at Mediaocean

Saturday, 20 April 2019

More tales from a GoLang newbie ... expected 'IDENT', found 'break'

I'm adopting, and loving, Microsoft Visual Studio Code ( VSCode ) for all my GoLang needs, but was somewhat confused by this: -

expected 'IDENT', found 'break'


This is my code: -

package break

import (
"fmt"
)

func BreakGoTest() {
snafu := []interface{}{"First", "Second", "Third"}
fmt.Println(snafu...)
}

Can you see what I did wrong ?

Yep, I've named my package .... break .... which is a reserved word.

Once I changed my package name: -

package breaker

import (
"fmt"
)

// BreakGoTest - This function does stuff
func BreakGoTest() {
snafu := []interface{}{"First", "Second", "Third"}
fmt.Println(snafu...)
}

all is well.

For the record, this code is merely to allow me to test go fmt .....

Wednesday, 10 April 2019

GoLang - weirdness with "panic: assignment to entry in nil map"

I kept seeing this: -

--- FAIL: TestClient (0.00s)
panic: assignment to entry in nil map [recovered]
panic: assignment to entry in nil map

goroutine 5 [running]:
testing.tRunner.func1(0xc42011e0f0)
/Users/hayd/Downloads/go/src/testing/testing.go:742 +0x29d
panic(0x128d080, 0x1314b50)
/Users/hayd/Downloads/go/src/runtime/panic.go:502 +0x229
net/textproto.MIMEHeader.Add(0x0, 0x12e2d99, 0xd, 0x12e1ac2, 0x8)
/Users/hayd/Downloads/go/src/net/textproto/header.go:15 +0xec
net/http.Header.Add(0x0, 0x12e2f05, 0xd, 0x12e1ac2, 0x8)
/Users/hayd/Downloads/go/src/net/http/header.go:24 +0x53
github.com/david-hay/GoStuff/cmd/sparkles.glob..func3(0x12e1aba, 0x8, 0x12e1ea3, 0x9, 0xc42006c800, 0x0, 0x0, 0x0, 0x0)
/Users/hayd/go/src/github.com/david-hay/GoStuff/cmd/sparkles/sparkles.go:38 +0xe1
github.com/david-hay/GoStuff/cmd/sparkles.TestClient(0xc42011e0f0)
/Users/hayd/go/src/github.com/david-hay/GoStuff/cmd/sparkles/sparkles_test.go:42 +0xc6
testing.tRunner(0xc42011e0f0, 0x12f4678)
/Users/hayd/Downloads/go/src/testing/testing.go:777 +0xd0
created by testing.(*T).Run
/Users/hayd/Downloads/go/src/testing/testing.go:824 +0x2e0
FAIL github.com/david-hay/GoStuff/cmd/sparkles 0.033s

when attempting to test my very basic HTTP code, using Visual Studio Code or just through the GoLang CLI: -

go test ./...

or: -

go test -v ./...

Looking at the stack trace, I saw reference to line 38 of my code: -

req.Header.Add("cache-control", "no-cache")

I commented that out, and the test just ran :-)

Not sure why that line causes that particular exception, given that I'm not actually using a Map, but ....

C'est la vie

Friday, 5 April 2019

Seeing the tabs in Vi

I was trying to resolve some tabs vs. spaces issues in my code, and realised that I was using a combination of both.

Thankfully vi has a recipe for that.

In my ~/.vimrc file, I added this: -

set list
set listchars=tab:>-

and now I can see this: -


Nice !

Thursday, 4 April 2019

GoLang and the unexpected exit

As with most of my career to-date, I'm on a learning curve ...

This time, it's GoLang, and I'm learning at a fast pace AND loving it !

I was trying to work out why my tests: -

go test ./...

or the more verbose: -

go test -v ./...

were failing with: -

exit status 1

rather than a more useful Panic message.

This helped: -

golang test exit status -1 and shows nothing

by helping me realise that my code had: -

log.Fatal

rather than: -

log.Panic

Once I fixed my code, life got a WHOLE lot better.

#LifeIsGood

#EveryDayIsASchoolDay

Friday, 29 March 2019

IBM Cloud Hyper Protect Crypto Services is Now Available on IBM Public Cloud

A new offering from my IBM Z organisation: -

...
IBM offers now two choices for key management. IBM Cloud Key Protect supports Bring Your Own Key (BYOK) for protecting data at rest. Today, IBM Cloud is announcing the general availability of IBM Cloud Hyper Protect Crypto Services, a dedicated Key Management and Cloud HSM Service designed especially for customers looking for greater control over their data encryption keys and the hardware security modules (HSMs) that protect these keys. The service is now available in US South region, based out of Dallas, Texas.

Hyper Protect Crypto Services supports Keep Your Own Key (KYOK), which allows data encryption keys to be protected by a dedicated, customer-controlled HSM that uses FIPS 140-2 Level 4 certified hardware. Built on IBM LinuxONE technology and being part of the IBM Cloud Hyper Protect portfolio of services, this service guarantees that privileged users—including IBM Cloud administrators—have no access to customer keys. This provides an ideal base to onboard sensitive apps to the cloud. Key Protect and IBM Cloud Hyper Protect Crypto Services use a common Key Provider API to provide a consistent approach for managing keys.
...
High availability and disaster recovery: IBM Cloud Hyper Protect Crypto Services, which now supports three availability zones in a selected region, is a highly available service with automatic features that help keep your applications secure and operational. You can create IBM Cloud Hyper Protect Crypto Services resources in the supported IBM Cloud regions, which represent the geographic area where your IBM Cloud Hyper Protect Crypto Services requests are handled and processed.

Scalability: The service instance can be scaled out to a maximum of six crypto units to meet your performance requirement. Each crypto unit can crypto-process 5,000 keys. In a production environment, it is recommended to select at least two crypto units to enable high availability. By selecting three or more crypto units, these crypto units are distributed among three availability zones in the selected region.
...

 IBM Cloud Hyper Protect Crypto Services is Now Available on IBM Public Cloud

Friday, 22 March 2019

Tinkering with Docker manifests ? You need Manifest Tool and MQuery

MQuery

A simple utility and backend for querying Docker v2 API-supporting registry images and reporting on "manifest list" multi-platform image support.

This project uses IBM Cloud Functions (built on OpenWhisk) as a backend, in concert with the manifest-tool inspect capability (packaged as a Docker function) to easily report on the status of whether an image is a manifest list entry in the registry, and if so, what architecture/os pairs are supported by the image.

https://github.com/estesp/mquery

docker run --rm mplatform/mquery mplatform/mquery

Image: mplatform/mquery
 * Manifest List: Yes
 * Supported platforms:
   - linux/amd64
   - linux/arm
   - linux/arm64
   - linux/ppc64le
   - linux/s390x
   - windows/amd64:10.0.14393.1593

manifest-tool

manifest-tool is a command line utility that implements a portion of the client side of the Docker registry v2.2 API for interacting with manifest objects in a registry conforming to that specification.

This tool was mainly created for the purpose of viewing, creating, and pushing the new manifests list object type in the Docker registry. Manifest lists are defined in the v2.2 image specification and exist mainly for the purpose of supporting multi-architecture and/or multi-platform images within a Docker registry.

https://github.com/estesp/manifest-tool

docker run --rm mplatform/mquery mplatform/manifest-tool:latest

Image: mplatform/manifest-tool:latest
 * Manifest List: Yes
 * Supported platforms:
   - linux/amd64
   - linux/arm
   - linux/arm64
   - linux/ppc64le
   - linux/s390x
   - windows/amd64:10.0.14393.2312

Tainting and "untainting" nodes in a Kubernetes cluster

Having been tinkering with the taint function on an x86 node in an IBM Kubernetes Service (IKS) cluster, to force my pods to deploy onto another node in the same cluster: -

kubectl taint node node1 node1=DoNotSchedulePods:NoExecute

I was looking for an easy way to reverse the taint ( "untaint" ), and found this: -

kubectl patch node node1 -p '{"spec":{"taints":[]}}'

with thanks to this: -

Tuesday, 12 March 2019

IBM Notes 9 - How to munge the Shortcut Buttons

For too long, I've been trying to remember how I can add shortcuts to the IBM Notes client, alongside the existing two shortcuts: -

Guess what ?

It's easy !

This rather nice all-in-one IBM Notes 9 tutorial one-pager: -

https://www.quicksourcelearning.com/images/samplepdfs/978193551845.pdf

reminded me.

It's this :-)


So now I have mail, calendar AND contacts: -


Yay!

Wednesday, 27 February 2019

Kubernetes tooling - tinkering with versions

Having built a new Kubernetes cluster on the IBM Kubernetes Service (IKS), which reports as version 1.11.7_1543 within the IKS dashboard: -

https://cloud.ibm.com/containers-kubernetes/clusters/

I'd noticed that the kubectl tool was out-of-sync with the cluster itself: -

kubectl version

Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.11", GitCommit:"637c7e288581ee40ab4ca210618a89a555b6e7e9", GitTreeState:"clean", BuildDate:"2018-11-26T14:38:32Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.7+IKS", GitCommit:"498bc5434e4bdc2dafddf57b2e8496f1cbd054bc", GitTreeState:"clean", BuildDate:"2019-02-01T08:10:15Z", GoVersion:"go1.10.7", Compiler:"gc", Platform:"linux/amd64"}

Initially, I assumed (!) that it was covered by the IBM Cloud Plugins: -

Setting up the CLI and API

and checked my plugins: -

ibmcloud plugin list

Listing installed plug-ins...

Plugin Name                            Version   Status   
cloud-functions/wsk/functions/fn       1.0.29       
container-registry                     0.1.368      
container-service/kubernetes-service   0.2.53    Update Available   
dev                                    2.1.15       
sdk-gen                                0.1.12       

This appeared to confirm my suspicion so I updated the IKS plugin: -

ibmcloud plugin update kubernetes-service

Plug-in 'container-service/kubernetes-service 0.2.53' was installed.
Checking upgrades for plug-in 'container-service/kubernetes-service' from repository 'IBM Cloud'...
Update 'container-service/kubernetes-service 0.2.53' to 'container-service/kubernetes-service 0.2.61'
Attempting to download the binary file...
 23.10 MiB / 23.10 MiB [=====================================================================================================================================================] 100.00% 9s
24224568 bytes downloaded
Updating binary...
OK
The plug-in was successfully upgraded.

ibmcloud plugin list

Listing installed plug-ins...

Plugin Name                            Version   Status   
sdk-gen                                0.1.12       
cloud-functions/wsk/functions/fn       1.0.29       
container-registry                     0.1.368      
container-service/kubernetes-service   0.2.61       
dev                                    2.1.15       

BUT kubectl continued to show as back-level: -

kubectl version

Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.11", GitCommit:"637c7e288581ee40ab4ca210618a89a555b6e7e9", GitTreeState:"clean", BuildDate:"2018-11-26T14:38:32Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.7+IKS", GitCommit:"498bc5434e4bdc2dafddf57b2e8496f1cbd054bc", GitTreeState:"clean", BuildDate:"2019-02-01T08:10:15Z", GoVersion:"go1.10.7", Compiler:"gc", Platform:"linux/amd64"}

Therefore, I chose to reinstall kubectl etc. as per this: -


( specifically using Homebrew, as I'm running on macOS )

brew install kubernetes-cli

Updating Homebrew...
==> Auto-updated Homebrew!
Updated 1 tap (homebrew/core).
==> New Formulae
cafeobj                                       homeassistant-cli                             re-flex                                       riff
==> Updated Formulae
go ✔                cfengine            closure-compiler    couchdb             dartsim             dhex                fx                  node-build          pulumi
apache-arrow        cflow               cmark-gfm           cpprestsdk          davix               dialog              git-lfs             numpy               shadowsocks-libev
axel                cfr-decompiler      cointop             cproto              dcd                 diffoscope          godep               openssl@1.1         ship
azure-cli           chakra              collector-sidecar   crc32c              ddrescue            diffstat            grafana             pandoc-citeproc     siege
bzt                 check_postgres      conan               cryptominisat       deark               digdag              kube-ps1            passenger
calicoctl           checkstyle          configen            cscope              debianutils         elektra             kustomize           pgweb
cdk                 chkrootkit          consul-template     czmq                deja-gnu            fabio               libtensorflow       pre-commit
cdogs-sdl           cli53               coturn              darcs               deployer            flake8              nginx               protoc-gen-go

==> Downloading https://homebrew.bintray.com/bottles/kubernetes-cli-1.13.3.mojave.bottle.tar.gz
######################################################################## 100.0%
==> Pouring kubernetes-cli-1.13.3.mojave.bottle.tar.gz
Error: The `brew link` step did not complete successfully
The formula built, but is not symlinked into /usr/local
Could not symlink bin/kubectl
Target /usr/local/bin/kubectl
already exists. You may want to remove it:
  rm '/usr/local/bin/kubectl'

To force the link and overwrite all conflicting files:
  brew link --overwrite kubernetes-cli

To list all files that would be deleted:
  brew link --overwrite --dry-run kubernetes-cli

Possible conflicting files are:
/usr/local/bin/kubectl -> /Applications/Docker.app/Contents/Resources/bin/kubectl
==> Caveats
Bash completion has been installed to:
  /usr/local/etc/bash_completion.d

zsh completions have been installed to:
  /usr/local/share/zsh/site-functions
==> Summary
🍺  /usr/local/Cellar/kubernetes-cli/1.13.3: 207 files, 43.7MB

Notice that it did NOT replace kubectl as it was already there :-)

So I chose to remove the existing kubectl : -

rm `which kubectl`

and then re-link: -

brew link kubernetes-cli

I then checked the version: -

kubectl version

Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.3", GitCommit:"721bfa751924da8d1680787490c54b9179b1fed0", GitTreeState:"clean", BuildDate:"2019-02-04T04:48:03Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.7+IKS", GitCommit:"498bc5434e4bdc2dafddf57b2e8496f1cbd054bc", GitTreeState:"clean", BuildDate:"2019-02-01T08:10:15Z", GoVersion:"go1.10.7", Compiler:"gc", Platform:"linux/amd64"}

so now kubectl is at a later version than the cluster ....

Let's see how it goes ....

*UPDATE*

I then read this: -

If you use a kubectl CLI version that does not match at least the major.minor version of your clusters, you might experience unexpected results. Make sure to keep your Kubernetes cluster and CLI versions up-to-date.

here: -

Setting up the CLI and API

and realised that the page actually includes a download link for the right major/minor version ( 11.7 ) kubectl for macOS.

I downloaded this and replaced the existing version: -

mv ~/Downloads/kubectl  /usr/local/bin/

and then validated the versions: -

kubectl version

Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.7", GitCommit:"65ecaf0671341311ce6aea0edab46ee69f65d59e", GitTreeState:"clean", BuildDate:"2019-01-24T19:32:00Z", GoVersion:"go1.10.7", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.7+IKS", GitCommit:"498bc5434e4bdc2dafddf57b2e8496f1cbd054bc", GitTreeState:"clean", BuildDate:"2019-02-01T08:10:15Z", GoVersion:"go1.10.7", Compiler:"gc", Platform:"linux/amd64"}

which now match ( major/minor ).

Nice !

Friday, 22 February 2019

End-to-End Hyper Protection of Data on the IBM Cloud

Finally ( for now ), there's this one: -

Learn how the IBM Cloud security portfolio is helping clients achieve continuous security for cloud applications while protecting data in every form—at rest, in transit and in use. You will see hands-on demos on: 1) IBM Hyper Protect Services - Key management backed by a dedicated cloud Hardware Security Module (HSM) for customers looking for complete control over data encryption keys and HSMs; 2) IBM Hyper Protect DBaaS - Industry-leading data confidentiality that allows data owners to have complete control over their data by preventing cloud operator access, backed by unparalleled vertical scale and performance; and 3) IBM Data Shield - Data in use protection for sensitive workloads.

https://myibm.ibm.com/events/think/all-sessions/session/7994A

Security and Flexibility in the IBM Cloud: A Deep-Dive into IBM Hyper Protect Services

And there's more: -

In the last few years, there have been a lot of large-scale data leaks by major companies revealed in the news. You wouldn't want this to be you. In this session, we’ll take a deep look into how IBM Hyper Protect Services can protect sensitive personal information and prevent against the attack vectors used to compromise these systems. We will take you through Hyper Protect DBaaS, offering secure databases on-demand, as well as Hyper Protect Crypto Services, providing secure cryptographic operations. Including demos and discussion, we'll see how new cloud services acting as always-encrypted platforms can help.

https://myibm.ibm.com/events/think/all-sessions/session/7629A

Protect Your Data and Code in the Cloud with IBM Hyper Protect Services

And from one of our Senior Technical Staff Members, Angel Nunez Mencias: -

Keeping your data and code protected while deployed in the cloud is not an easy task since the hardware and system administrators are not under your control. With IBM Hyper Protect Services, code and data is protected from any access by technology, so there is no need to trust external system admins. During this season, the chief architect of Hyper Protect will present the underlying technology and how the existing set of Hyper Protect Services leverage it today. He will also show how these services are used today to protect end-to-end solutions, and what additional services are being considered for the future.

https://myibm.ibm.com/events/think/all-sessions/session/3140A

Tech Talk: Leveraging IBM Z Security in the Cloud with IBM Cloud Hyper Protect Services

My IBM colleague, Dr Chris Poole, presented upon this: -

Tech Talk: Leveraging IBM Z Security in the Cloud with IBM Cloud Hyper Protect Services 

at IBM Think last week.

His most excellent session is here: -

Thinking about how to make your cloud-based applications compliant with GDPR and other regulations? Need data-at-rest encryption but don’t want to refactor? Try IBM Hyper Protect Services! In this talk, you’ll learn about Hyper Protect Crypto Services for secure key storage, and Hyper Protect DBaaS for an encrypted MongoDB or PostgreSQL data layer.

https://myibm.ibm.com/events/think/all-sessions/session/8110A

Enjoy !

Wednesday, 20 February 2019

MainframerZ meetup at Lloyds - Tuesday 19 March 2019 - See you there


Having recently moved into the IBM Z development organisation, as mentioned before: -

New day, new job - more of the same, but in a VERY good way

I'm now working in a new area, with a new ( to me ) technology, brining what I know - Linux, Containers, Kubernetes etc.

And .... thanks to the MainframerZ community, I have the perfect opportunity to talk about what I do.

Lloyds Banking Group (LBG) have kindly invited us to bring the Community to them, hosting a Meet-up at their offices in London on Tuesday 19 March 2019.

I'm totally looking forward to meeting my new community, as I've got a HUGE amount to learn about the IBM Z and LinuxOne platforms, and the workloads that our clients are looking to host.

The details are on Meetup here: -

MainframerZ meetup at Lloyds


See you there 😀

Monday, 18 February 2019

Following my previous post: -

Security Bulletin: IBM Cloud Kubernetes Service is affected by a privilege escalation vulnerability in runc 

I also needed to update Docker on the Mac: -



to mitigate the effect of: -

CVE-2019-5736

runc through 1.0-rc6, as used in Docker before 18.09.2 and other products, allows attackers to overwrite the host runc binary (and consequently obtain host root access) by leveraging the ability to execute a command as root within one of these types of containers: (1) a new container with an attacker-controlled image, or (2) an existing container, to which the attacker previously had write access, that can be attached with docker exec. This occurs because of file-descriptor mishandling, related to /proc/self/exe. 

It's a fairly large update: -


but it's definitely worth doing.

Make it so ....

Security Bulletin: IBM Cloud Kubernetes Service is affected by a privilege escalation vulnerability in runc

Following on from this: -

CVE-2019-5736

runc through 1.0-rc6, as used in Docker before 18.09.2 and other products, allows attackers to overwrite the host runc binary (and consequently obtain host root access) by leveraging the ability to execute a command as root within one of these types of containers: (1) a new container with an attacker-controlled image, or (2) an existing container, to which the attacker previously had write access, that can be attached with docker exec. This occurs because of file-descriptor mishandling, related to /proc/self/exe. 

IBM issued this: -

Security Bulletin: IBM Cloud Kubernetes Service is affected by a privilege escalation vulnerability in runc

...
IBM Cloud Kubernetes Service is affected by a security vulnerability in runc which could allow an attacker that is authorized to run a process as root inside a container to execute arbitrary commands with root privileges on the container’s host system.
...
Updates for IBM Cloud Kubernetes Service cluster worker nodes at versions 1.10 and later will be available shortly that fix this vulnerability.  Customers must update their worker nodes to address the vulnerability.  See Updating worker nodes for details on updating worker nodes.  To verify your cluster worker nodes have been updated, use the following IBM Cloud CLI command to confirm the currently running version:
...

I've got an IKS cluster running: -

https://cloud.ibm.com/containers-kubernetes/overview

so wanted to ensure that my worker node was suitably patched.

So, having logged into IBM Cloud: -

ibmcloud login ....

I checked my cluster: -

ibmcloud ks workers --cluster dmhIKSCluster

OK
ID                                                 Public IP       Private IP      Machine Type        State    Status   Zone    Version   
kube-dal10-crbd60afb0c7ff4a98a4017fb784ee4e96-w1   192.168.153.123   10.94.221.198   u2c.2x4.encrypted   normal   Ready    dal10   1.11.6_1541*   

* To update to 1.11.7_1544 version, run 'ibmcloud ks worker-update'. Review and make any required version changes before you update: https://console.bluemix.net/docs/containers/cs_cluster_update.html#worker_node

and then updated the worker: -

ibmcloud ks worker-update --cluster dmhIKSCluster --workers kube-dal10-crbd60afb0c7ff4a98a4017fb784ee4e96-w1

Updating the worker node version can cause downtime for your apps and services. During the update, all pods might be rescheduled onto other worker nodes and data is deleted if not stored outside the pod. To avoid downtime, ensure that you have enough worker nodes to handle your workload while the selected worker nodes are updating.
You might need to change your YAML files for deployments before updating. Review the docs for details: https://console.bluemix.net/docs/containers/cs_cluster_update.html#worker_node
Are you sure you want to update your worker node [kube-dal10-crbd60afb0c7ff4a98a4017fb784ee4e96-w1] to 1.11.7_1544? [y/N]> y

Updating worker kube-dal10-crbd60afb0c7ff4a98a4017fb784ee4e96-w1...
OK

ibmcloud ks workers --cluster dmhIKSCluster

OK
ID                                                 Public IP       Private IP      Machine Type        State    Status   Zone    Version   
kube-dal10-crbd60afb0c7ff4a98a4017fb784ee4e96-w1   192.168.153.123   10.94.221.198   u2c.2x4.encrypted   normal   Ready    dal10   1.11.6_1541 --> 1.11.7_1544 (pending)   

ibmcloud ks workers --cluster dmhIKSCluster

OK
ID                                                 Public IP       Private IP      Machine Type        State    Status   Zone    Version   
kube-dal10-crbd60afb0c7ff4a98a4017fb784ee4e96-w1   192.168.153.123   10.94.221.198   u2c.2x4.encrypted   normal   Ready    dal10   1.11.6_1541 --> 1.11.7_1544 (pending)   

ibmcloud ks workers --cluster dmhIKSCluster

OK
ID                                                 Public IP       Private IP      Machine Type        State       Status                                                                Zone    Version   
kube-dal10-crbd60afb0c7ff4a98a4017fb784ee4e96-w1   192.168.153.123   10.94.221.198   u2c.2x4.encrypted   reloading   Waiting for IBM Cloud infrastructure: Setup provision configuration   dal10   1.11.6_1541 --> 1.11.7_1544 (pending)   

ibmcloud ks workers --cluster dmhIKSCluster

OK
ID                                                 Public IP       Private IP      Machine Type        State    Status   Zone    Version   
kube-dal10-crbd60afb0c7ff4a98a4017fb784ee4e96-w1   192.168.153.123   10.94.221.198   u2c.2x4.encrypted   normal   Ready    dal10   1.11.7_1544   

So, after a small amount of time, I'm all updated.

Shortly afterwards, I received an email from IBM Cloud: -

...
The operating system reload is complete for computing instance kube-dal10-crbd60afb0c7ff4a98a4017fb784ee4e96-w1.cloud.ibm [192.168.153.123].
...

and my cluster is clean n' green.

Now to finish updating Docker elsewhere ... including on the Mac

Wednesday, 13 February 2019

Following up ... defining K8S Services using YAML

As a fup to this: -

Playing with Kubernetes deployments and NodePort services

life is SO much easier if I choose to define the service using YAML ( YAML Ain't Markup Language ).

So this is with what I ended up: -

cat ~/Desktop/nginx.yaml
 
apiVersion: v1
kind: Service
metadata:
  name: my-nginx
  namespace: default
  labels:
    app: nginx
spec:
  type: NodePort
  ports:
  - name: http
    port: 80
    targetPort: 80
    protocol: TCP
  selector:
    app: nginx

remember that YAML is very positional and, apparently, tabs are abhorrent :-)

Having created - and validated using various listing plugins for Atom - the YAML, I was then able to apply it: -

kubectl apply -f ~/Desktop/nginx.yaml

service "my-nginx" created

and then validate: -

kubectl get services

NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP   172.21.0.1              443/TCP        14d
my-nginx     NodePort    172.21.24.197          80:31665/TCP   7s

and then test: -

curl http://192.168.132.131:31665

Welcome to nginx!

Welcome to nginx!

If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.

For online documentation and support please refer to
Commercial support is available at

Thank you for using nginx.

For reference, one can get YAML or JSON out of most, but not all of the K8S, commands: -

kubectl get services -o yaml

apiVersion: v1
items:
- apiVersion: v1
  kind: Service
  metadata:
    creationTimestamp: 2019-01-29T16:18:07Z
    labels:
      component: apiserver
      provider: kubernetes
    name: kubernetes
    namespace: default
    resourceVersion: "33"
    selfLink: /api/v1/namespaces/default/services/kubernetes
    uid: 76a02dea-23e1-11e9-b35e-2a02ed9d765d
  spec:
    clusterIP: 172.21.0.1
    ports:
    - name: https
      port: 443
      protocol: TCP
      targetPort: 2040
    sessionAffinity: None
    type: ClusterIP
  status:
    loadBalancer: {}
- apiVersion: v1
  kind: Service
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"nginx"},"name":"my-nginx","namespace":"default"},"spec":{"ports":[{"name":"http","port":80,"protocol":"TCP","targetPort":80}],"selector":{"app":"nginx"},"type":"NodePort"}}
    creationTimestamp: 2019-02-13T11:40:09Z
    labels:
      app: nginx
    name: my-nginx
    namespace: default
    resourceVersion: "2072491"
    selfLink: /api/v1/namespaces/default/services/my-nginx
    uid: 1da46471-2f84-11e9-9f99-1201bf98c5fb
  spec:
    clusterIP: 172.21.24.197
    externalTrafficPolicy: Cluster
    ports:
    - name: http
      nodePort: 31665
      port: 80
      protocol: TCP
      targetPort: 80
    selector:
      app: nginx
    sessionAffinity: None
    type: NodePort
  status:
    loadBalancer: {}
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

kubectl get services -o json

{
    "apiVersion": "v1",
    "items": [
        {
            "apiVersion": "v1",
            "kind": "Service",
            "metadata": {
                "creationTimestamp": "2019-01-29T16:18:07Z",
                "labels": {
                    "component": "apiserver",
                    "provider": "kubernetes"
                },
                "name": "kubernetes",
                "namespace": "default",
                "resourceVersion": "33",
                "selfLink": "/api/v1/namespaces/default/services/kubernetes",
                "uid": "76a02dea-23e1-11e9-b35e-2a02ed9d765d"
            },
            "spec": {
                "clusterIP": "172.21.0.1",
                "ports": [
                    {
                        "name": "https",
                        "port": 443,
                        "protocol": "TCP",
                        "targetPort": 2040
                    }
                ],
                "sessionAffinity": "None",
                "type": "ClusterIP"
            },
            "status": {
                "loadBalancer": {}
            }
        },
        {
            "apiVersion": "v1",
            "kind": "Service",
            "metadata": {
                "annotations": {
                    "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"v1\",\"kind\":\"Service\",\"metadata\":{\"annotations\":{},\"labels\":{\"app\":\"nginx\"},\"name\":\"my-nginx\",\"namespace\":\"default\"},\"spec\":{\"ports\":[{\"name\":\"http\",\"port\":80,\"protocol\":\"TCP\",\"targetPort\":80}],\"selector\":{\"app\":\"nginx\"},\"type\":\"NodePort\"}}\n"
                },
                "creationTimestamp": "2019-02-13T11:40:09Z",
                "labels": {
                    "app": "nginx"
                },
                "name": "my-nginx",
                "namespace": "default",
                "resourceVersion": "2072491",
                "selfLink": "/api/v1/namespaces/default/services/my-nginx",
                "uid": "1da46471-2f84-11e9-9f99-1201bf98c5fb"
            },
            "spec": {
                "clusterIP": "172.21.24.197",
                "externalTrafficPolicy": "Cluster",
                "ports": [
                    {
                        "name": "http",
                        "nodePort": 31665,
                        "port": 80,
                        "protocol": "TCP",
                        "targetPort": 80
                    }
                ],
                "selector": {
                    "app": "nginx"
                },
                "sessionAffinity": "None",
                "type": "NodePort"
            },
            "status": {
                "loadBalancer": {}
            }
        }
    ],
    "kind": "List",
    "metadata": {
        "resourceVersion": "",
        "selfLink": ""
    }
}

which is nice.

Playing with Kubernetes deployments and NodePort services

Today I'm fiddling with nginx as a workload on my IBM Kubernetes Service (IKS) cluster.

My default process was this: -

kubectl create deployment nginx --image=nginx

...
deployment.extensions "nginx" created
...

kubectl get deployments

...
NAME      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
nginx     1         1         1            1           20s
...

kubectl describe pod `kubectl get pods | grep nginx | awk '{print $1}'`

...
Events:
  Type     Reason             Age                From                     Message
  ----     ------             ----               ----                     -------
  Normal   Scheduled          39m                default-scheduler        Successfully assigned default/nginx-78f5d695bd-jxfvm to 192.168.132.123
...

kubectl create service nodeport nginx --tcp=80:80

...
service "nginx" created
...

kubectl get services

...
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP   172.21.0.1              443/TCP        14d
nginx        NodePort    172.21.113.167          80:30923/TCP   20s
...

kubectl get nodes

...
  ROLES     AGE       VERSION
...
192.168.132.123   Ready          13d       v1.11.5
...

and then combine the public IP address of the node ( 192.168.132.123 ) and the generated NodePort ( 30923 ) to allow me to access nginx: -

curl http://192.168.132.123:30923

...
Welcome to nginx!

Welcome to nginx!

If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.

For online documentation and support please refer to
Commercial support is available at

Thank you for using nginx.
...

I also realised that I could "debug" the pod hosting the nginx service: -

docker ps -a

...
CONTAINER ID        IMAGE                  COMMAND                  CREATED             STATUS              PORTS               NAMES
fda90a370446        nginx                  "nginx -g 'daemon of…"   19 seconds ago      Up 18 seconds                           k8s_nginx_nginx-78f5d695bd-8vrgl_default_d39dad7e-2f79-11e9-9f99-1201bf98c5fb_0
...

docker logs fda90a370446

...
10.23.2.59 - - [13/Feb/2019:10:30:42 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.54.0" "-"
...

However, I also "discovered" that there seems to be a correlation between the NAME of the NodePort service and the NAME of the nginx deployment.

If I create a NodePort service with a different name: -

kubectl create service nodeport foobar --tcp=80:80

and then get the newly created service: -

kubectl get services

...
NAME         TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
foobar       NodePort    172.21.0.34          80:30952/TCP   1m
kubernetes   ClusterIP   172.21.0.1            443/TCP        14d
...

I'm no longer able to hit nginx: -

curl http://192.168.132.123:30952

returns: -

...
curl: (7) Failed to connect to 192.168.132.123 port 30952: Connection refused
...

If I delete the service: -

kubectl delete service foobar

...
service "foobar" deleted
...

and recreate it with the SAME name as the deployment: -

kubectl create service nodeport nginx --tcp=80:80

...
service "nginx" created
...

I'm back in the game: -

kubectl get services

...
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP   172.21.0.1            443/TCP        14d
nginx        NodePort    172.21.31.44          80:30281/TCP   33s
...

curl http://192.168.132.123:30281

...
Welcome to nginx!

Welcome to nginx!

If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.

For online documentation and support please refer to
Commercial support is available at

Thank you for using nginx.
...

So there does APPEAR to be a correlation between the deployment name and the service name.

Obviously, K8S provides tagging for this, but I don't believe that's applicable to a NodePort service.

However, there is a different way ....

It is possible to expose an existing deployment and create a NodePort service "on the fly", as per the following: -

kubectl expose deployment nginx --type=NodePort --name=my-nginx --port 80

This creates a NodePort service: -

kubectl get services

...
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP   172.21.0.1            443/TCP        14d
my-nginx     NodePort    172.21.8.192          80:30628/TCP   39s
...

NOTE that the name of the service - my-nginx - does NOT tie up with the deployment per se and yet ....

curl http://192.168.132.123:30281

...
Welcome to nginx!

Welcome to nginx!

If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.

For online documentation and support please refer to
Commercial support is available at

Thank you for using nginx.
...

If I dig into the newly created service: -

kubectl describe service my-nginx

Name:                     my-nginx
Namespace:                default
Labels:                   app=nginx
Annotations:              
Selector:                 app=nginx
Type:                     NodePort
IP:                       172.21.8.192
Port:                      80/TCP
TargetPort:               80/TCP
NodePort:                  30628/TCP
Endpoints:                172.30.148.204:80
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                  

I can see that it's tagged against the deployment - nginx - using the Label and Selector; I suspect that it's the latter that made the difference.

If I revert back to my previous service: -

kubectl create service nodeport nginx --tcp=80:80

...
service "nginx" created
...

and dig into it: -

kubectl describe service nginx

Name:                     nginx
Namespace:                default
Labels:                   app=nginx
Annotations:              
Selector:                 app=nginx
Type:                     NodePort
IP:                       172.21.68.125
Port:                     80-80  80/TCP
TargetPort:               80/TCP
NodePort:                 80-80  31411/TCP
Endpoints:                172.30.148.204:80
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                  

So the name of the service IS used to "tag" the target deployment.

If I misname my service: -

kubectl create service nodeport foobar --tcp=80:80

service "foobar" created

kubectl describe service foobar

Name:                     foobar
Namespace:                default
Labels:                   app=foobar
Annotations:              
Selector:                 app=foobar
Type:                     NodePort
IP:                       172.21.232.250
Port:                     80-80  80/TCP
TargetPort:               80/TCP
NodePort:                 80-80  32146/TCP
Endpoints:                
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                  

which explains why this does not: -

curl http://192.168.132.123:32146

In other words, the name of the service DOES MATTER, but only where one specifically creates the service, as opposed to letting the kubectl expose deployment do it for one.


Setting Authorization Headers in IBM API Connect Test and Monitor

I've recently started exploring the IBM API Connect Test and Monitor  tool on IBM Cloud here: - https://www.ibm.com/cloud/api-connect/...