Moving past Docker on Mac, IBM Containers on Bluemix

Having tinkered with WebSphere Liberty Profile on Docker over the past few days, as per recent posts to this blog, I wanted to go one stage further, and recreate a similar WLP / application on IBM Containers, which runs on IBM Bluemix.

I'm definitely on the on-ramp for Bluemix, and the learning curve still looks rather steep …

For reference: -

Troubleshooting for accessing Bluemix

Logging in to the IBM Containers CLI plug-in (cf ic)


IBM® Bluemix® uses the Cloud Foundry command line interface, cf, to modify applications, service instances, and service bindings. You can also use Bluemix command line tool that provides extended experience to manage your Bluemix environment besides Cloud Foundry applications.

Command Line Interface reference

Bluemix CLI

Deploying your app with the command line interface

But here goes….

Download and Install Prerequisites

Bluemix_CLI_0.4.1.pkg
cf-cli-installer_6.21.1_osx.pkg
DockerToolbox-1.12.0.pkg

Sources

Deploying your app with the command line interface

Docker Toolbox

Fix up .cf subdirectory to avoid panic

panic: Config error: open /Users/davidhay/.cf/config.json: permission denied

sudo chmod -R 777 /Users/davidhay/.cf/

Connect to Bluemix

bluemix api https://api.ng.bluemix.net

Log in to Bluemix - may NOT need this, as we also login using Cloud Foundry (CF) CLI

bluemix login -u david_hay@uk.ibm.com -o david_hay@uk.ibm.com -s david_hay

-OR- use the SSO variant: -

bluemix login -sso -u david_hay@uk.ibm.com -o david_hay@uk.ibm.com -s david_hay

having got a passcode from here: -

https://login.ng.bluemix.net/UAALoginServerWAR/passcode

Install Mac Containers plugin for Cloud Foundry

cf install-plugin https://static-ice.ng.bluemix.net/ibm-containers-mac

Validate plugin

cf plugins

Listing Installed Plugins...
OK

Plugin Name      Version   Command Name   Command Help
IBM-Containers   0.8.934   ic             IBM Containers plug-in

Login to Cloud Foundry

cf login -a api.eu-gb.bluemix.net -u david_hay@uk.ibm.com -o david_hay@uk.ibm.com -s david_hay

-OR- use the SSO variant: -

cf login -sso -a api.eu-gb.bluemix.net -u david_hay@uk.ibm.com -o david_hay@uk.ibm.com -s david_hay

having got a passcode from here: -

https://login.ng.bluemix.net/UAALoginServerWAR/passcode

Set IBM Containers Namespace

cf ic namespace set david_hay

Initialise IBM Containers CLI

cf ic init

Check available images

cf ic images

REPOSITORY                                      TAG                 IMAGE ID            CREATED             SIZE
registry.eu-gb.bluemix.net/ibm-node-strong-pm   latest              37a6db86742f        8 weeks ago         240.4 MB
registry.eu-gb.bluemix.net/ibmliberty           javaee7             77b68609eecb        2 weeks ago         325.4 MB
registry.eu-gb.bluemix.net/ibmliberty           latest              77b68609eecb        2 weeks ago         325.4 MB
registry.eu-gb.bluemix.net/ibmliberty           webProfile6         cd1753bef3b8        2 weeks ago         264.8 MB
registry.eu-gb.bluemix.net/ibmliberty           webProfile7         c0aa4441c231        2 weeks ago         283.1 MB
registry.eu-gb.bluemix.net/ibmnode              latest              1575262a71a0        5 days ago          189.3 MB
registry.eu-gb.bluemix.net/ibmnode              v4                  1575262a71a0        5 days ago          189.3 MB
registry.eu-gb.bluemix.net/ibmnode              v1.1                554cb6bd436b        5 days ago          178.8 MB
registry.eu-gb.bluemix.net/ibmnode              v1.2                38c08ea9d331        6 days ago          185 MB

Start WebSphere Liberty Profile container

foobar=`cf ic run -d -t -p 80:9080 -p 443:9443 ibmliberty:latest`

See what's running

cf ic ps -a

CONTAINER ID        IMAGE                                          COMMAND             CREATED             STATUS                  PORTS               NAMES
ec2b1754-0f7        registry.eu-gb.bluemix.net/ibmliberty:latest   ""                  3 minutes ago       Running 3 minutes ago                       admiring_fermi

Deploy an application to WLP

cf ic cp ~/Downloads/ferret.war $foobar:/opt/ibm/wlp/usr/servers/defaultServer/dropins

Run a command against the container

cf ic exec -it $foobar /bin/bash

Monitor the container logs

cf ic logs $foobar -f 

Functionally test the Ferret app

http://localhost/ferret/

https://localhost/ferret

Stop the container

cf ic stop $foobar

Remove the container

cf ic rm $foobar

Log out from Cloud Foundry

cf logout

Log out from Bluemix

bluemix logout

PS Remember the tagline for my blog …. YMMV

Docker and Liberty - Scripting it

Next, I wanted to create a mechanism to allow me to automate the use of WebSphere Liberty Profile on Docker, with a view to creating a little process to: -

(1) Create a Docker container from an existing WLP image

(2) Deploy a simple web application ( I'm using Ferret, which is akin to good old Snoop from here )

(3) Monitor the WLP logs

(4) Run a basic functional test

(5) Stop the container once testing has completed

(6) Remove the container

so this is with what I came up: -

Start a Container and assign Container ID to a variable - foobar

foobar=`docker run -d -t -p 80:9080 -p 443:9443 websphere-liberty:latest`

Copy a WAR file to the Liberty dropins folder

docker cp ~/Downloads/ferret.war $foobar:/opt/ibm/wlp/usr/servers/defaultServer/dropins

Functional-test the Ferret app - via a browser

http://localhost/ferret/

Watch the Liberty logs - found in /logs/messages.log

docker logs $foobar -f

…
[AUDIT   ] CWWKF0011I: The server defaultServer is ready to run a smarter planet.
[AUDIT   ] CWWKT0016I: Web application available (default_host): http://878db1a296e5:9080/ferret/
[AUDIT   ] CWWKZ0001I: Application ferret started in 0.735 seconds.
…

Start an interactive command-line to the container

docker exec -i -t $foobar /bin/bash

Stop the Liberty runtime

docker exec -i -t $foobar bash -c "/opt/ibm/wlp/bin/server stop"

Watch the Liberty logs - found in /logs/messages.log

docker logs $foobar -f

…
[AUDIT   ] CWWKE0055I: Server shutdown requested on Tuesday, August 30, 2016 at 12:22 PM. The server defaultServer is shutting down.
[AUDIT   ] CWWKT0017I: Web application removed (default_host): http://878db1a296e5:9080/ferret/
[AUDIT   ] CWWKZ0009I: The application ferret has stopped successfully.
[AUDIT   ] CWWKI0002I: The CORBA name server is no longer available at corbaloc:iiop:localhost:2809/NameService.
[AUDIT   ] CWWKE0036I: The server defaultServer stopped after 45.191 seconds.
…

Remove the Container

docker rm $foobar

Which is nice :-)

WebSphere Liberty Profile on Docker - Where are my logs ?

As per previous posts, I'm tinkering with Docker, using WebSphere Liberty Profile as my starting point

One thing that was confusing me was the lack of Liberty logging ….

In "normal" Liberty, the logs are here: -

ls -al ~/wlp/usr/servers/defaultServer/logs/

total 344
drwxr-x---  5 davidhay  staff     170 28 Aug 05:34 .
drwxr-x---  9 davidhay  staff     306 27 Aug 10:14 ..
-rw-r-----  1 davidhay  staff   47608 28 Aug 05:34 console.log
-rw-r-----  1 davidhay  staff  123862 28 Aug 05:34 messages.log
drwxr-x---  2 davidhay  staff      68 28 Aug 05:34 state

and, yet, in a Docker container built from the latest WebSphere Liberty Profile image: -

docker images

REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
websphere-liberty   latest              f042ad3a168c        2 weeks ago         444.4 MB
websphere-liberty   webProfile7         efa42bcc5850        2 weeks ago         396 MB

docker ps -a

CONTAINER ID        IMAGE                      COMMAND                  CREATED             STATUS              PORTS                NAMES
89a3ba2841e4        websphere-liberty:latest   "/opt/ibm/docker/dock"   8 minutes ago       Up 8 minutes        9080/tcp, 9443/tcp   hungry_wozniak

I haz no logs: -

ls -al /opt/ibm/wlp/usr/servers/defaultServer/

total 28
drwxr-x--- 5 root root 4096 Aug 30 08:07 .
drwxr-xr-x 6 root root 4096 Aug 30 08:07 ..
drwxr-x--- 2 root root 4096 Aug 11 22:11 apps
drwxr-xr-x 3 root root 4096 Aug 30 08:07 configDropins
drwxr-x--- 2 root root 4096 Aug 11 22:11 dropins
-rw-r----- 1 root root   25 Aug 11 22:11 server.env
-rw-rw-r-- 1 root root 1622 Aug 11 22:13 server.xml

Thankfully, I found this: -

<snip>

Note: All ibmliberty images are configured to write Liberty log files to the directory /logs inside the container. All other files that are written by the Liberty server, are created in the directory /opt/ibm/wlp/output/defaultServer. You can access these files by using the shortcut /output.

</snip>

Getting started with ibmliberty images for Bluemix

and, quelle surprise, there they are: -

ls /output

logs  messaging  resources  work area

ls -al /output

lrwxrwxrwx 1 root root 33 Aug 11 22:11 /output -> /opt/ibm/wlp/output/defaultServer

ls -al /opt/ibm/wlp/output/defaultServer/

total 24
drwxr-x--- 6 root root 4096 Aug 30 08:07 .
drwxr-x--T 7 root root 4096 Aug 30 08:07 ..
drwxr-x--- 3 root root 4096 Aug 30 08:07 logs
drwxr-x--- 3 root root 4096 Aug 30 08:07 messaging
drwxr-x--- 3 root root 4096 Aug 30 08:07 resources
drwxr-x--- 5 root root 4096 Aug 30 08:07 workarea

ls -al /opt/ibm/wlp/output/defaultServer/logs/

total 12
drwxr-x--- 3 root root 4096 Aug 30 08:07 .
drwxr-x--- 6 root root 4096 Aug 30 08:07 ..
drwxr-x--- 2 root root 4096 Aug 30 08:07 state

ls -al /logs/

total 20
drwxr-xr-x  2 root root  4096 Aug 30 08:07 .
drwxr-xr-x 61 root root  4096 Aug 30 08:07 ..
-rw-r-----  1 root root 10681 Aug 30 08:07 messages.log

cat /logs/messages.log 

********************************************************************************
product = WebSphere Application Server 16.0.0.2 (wlp-1.0.13.cl160220160526-2258)
wlp.install.dir = /opt/ibm/wlp/
server.output.dir = /opt/ibm/wlp/output/defaultServer/
java.home = /opt/ibm/java/jre
java.version = 1.8.0
java.runtime = Java(TM) SE Runtime Environment (pxa6480sr3fp10-20160720_02 (SR3 FP10))
os = Linux (4.4.15-moby; amd64) (en_US)
process = 1@89a3ba2841e4
********************************************************************************
[8/30/16 8:07:42:009 UTC] 00000001 com.ibm.ws.kernel.launch.internal.FrameworkManager           A CWWKE0001I: The server defaultServer has been launched.
[8/30/16 8:07:42:024 UTC] 00000001 com.ibm.ws.kernel.launch.internal.FrameworkManager           A CWWKE0100I: This product is licensed for development, and limited production use. The full license terms can be viewed here: https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/wasdev/license/base_ilan/ilan/16.0.0.2/lafiles/en.html
...

[8/30/16 8:07:50:917 UTC] 0000001a com.ibm.ws.cache.ServerCache                                 I DYNA1001I: WebSphere Dynamic Cache instance named baseCache initialized successfully.

[8/30/16 8:07:50:920 UTC] 0000001a com.ibm.ws.cache.ServerCache                                 I DYNA1071I: The cache provider default is being used.
[8/30/16 8:07:50:920 UTC] 0000001a com.ibm.ws.cache.CacheServiceImpl                            I DYNA1056I: Dynamic Cache (object cache) initialized successfully.
[8/30/16 8:07:51:361 UTC] 0000001a com.ibm.ws.transport.iiop.internal.ORBWrapperInternal        A CWWKI0001I: The CORBA name server is now available at corbaloc:iiop:localhost:2809/NameService.
[8/30/16 8:07:51:711 UTC] 00000021 com.ibm.ws.kernel.feature.internal.FeatureManager            A CWWKF0012I: The server installed the following features: [servlet-3.1, beanValidation-1.1, ssl-1.0, jndi-1.0, jca-1.7, ejbPersistentTimer-3.2, appSecurity-2.0, j2eeManagement-1.1, jdbc-4.1, wasJmsServer-1.0, jaxrs-2.0, javaMail-1.5, cdi-1.2, webProfile-7.0, jcaInboundSecurity-1.0, jpa-2.1, jsp-2.3, ejbLite-3.2, managedBeans-1.0, jsf-2.2, ejbHome-3.2, jaxws-2.2, jsonp-1.0, el-3.0, jaxrsClient-2.0, concurrent-1.0, appClientSupport-1.0, ejbRemote-3.2, javaee-7.0, jaxb-2.2, mdb-3.2, jacc-1.5, batch-1.0, ejb-3.2, json-1.0, jaspic-1.1, distributedMap-1.0, websocket-1.1, wasJmsSecurity-1.0, wasJmsClient-2.0].
[8/30/16 8:07:51:711 UTC] 00000021 com.ibm.ws.kernel.feature.internal.FeatureManager            I CWWKF0008I: Feature update completed in 7.901 seconds.
[8/30/16 8:07:51:712 UTC] 00000021 com.ibm.ws.kernel.feature.internal.FeatureManager            A CWWKF0011I: The server defaultServer is ready to run a smarter planet.

WebSphere Application Server and IBM HTTP Server Security Bulletin List

This, via one of my IBM Germany friends 

Question

Is there a list that contains the security bulletins that apply to WebSphere Application Server and IBM HTTP Server?

Answer

The following table is provided to help you locate WebSphere Application Server and IBM HTTP Server security bulletins.

Note that the IBM Java runtime included with WebSphere Application Server provides an execution environment for non-IBM code. While the below table includes all IBM Java vulnerabilities related to the WebSphere Application Server product, there may be additional IBM Java vulnerabilities which impact non-IBM code running in your WebSphere Application Server environment. For a listing of all IBM Java security bulletins, please refer to IBM Java Security Alerts. To determine the Java SDK version used with WebSphere Application Server, please refer to the Verify Java SDK version shipped with WebSphere Application Server.

WebSphere Application Server and IBM HTTP Server Security Bulletin List

CWWKF1219E: The IBM WebSphere Liberty Repository cannot be reached

This one has been bugging me for a day or so: -

docker build -t websphere-liberty:artifactory .

...

Step 6 : RUN installUtility testConnection
 ---> Running in 40c4586b857e
Testing the connection to all configured repositories ...
This process might take several minutes to complete.

Configured Repositories
----------------------------------------------------------------------
Name: IBM WebSphere Liberty Repository (Default Assets Repository)
Status: Failed to connect to the configured repository.
Reason: CWWKF1219E: The IBM WebSphere Liberty Repository cannot be
    reached. Verify that your computer has network access and firewalls are
    configured correctly, then try the action again. If the connection
    still fails, the repository server might be temporarily unavailable.

The command '/bin/sh -c installUtility testConnection' returned a non-zero code: 33

…

This occurs when running a WebSphere Liberty profile container within Docker on my Mac.

Initially, I was trying to install a Liberty component - adminCenter - which was returning the same CWWKF1219E exception.

To further diagnose this, I tweaked my Dockerfile: -

FROM websphere-liberty:webProfile7
ADD server.xml /opt/ibm/wlp/usr/servers/defaultServer/
# ADD repositories.properties /opt/ibm/wlp/etc/
# ADD wlpRepo.zip /opt/ibm/wlp/usr/shared/resources/
ADD artifactory.war /opt/ibm/wlp/usr/servers/defaultServer/dropins/
ADD SuperSnoopWeb.war /opt/ibm/wlp/usr/servers/defaultServer/dropins/
ENV LICENSE accept
RUN installUtility testConnection
# RUN installUtility install --acceptLicense adminCenter-1.0

and added the line: -

RUN installUtility testConnection

commenting out the other ADD and RUN lines.

I spent many a happy hour tinkering around with local repositories, hence the lines: -

# ADD repositories.properties /opt/ibm/wlp/etc/

# ADD wlpRepo.zip /opt/ibm/wlp/usr/shared/resources/

I tested Liberty's testConnection utility: -

~/Downloads/wlp/bin/installUtility testConnection

Testing the connection to all configured repositories ...
This process might take several minutes to complete.

Configured Repositories
----------------------------------------------------------------------
Name: IBM WebSphere Liberty Repository (Default Assets Repository)
Status: Successfully connected to the configured repository.

so the actual on-line repository service was obviously A-OK.

Long story short, the problem appeared to be that Liberty within the Docker container wasn't able to reach the internet.

I dug around on Google, and found this: -

My docker container has no internet

which said, in part: -

<snip>

If you are running Docker on OSX using Docker Machine, then the following worked for me:

docker-machine restart

<...wait for it to restart, which takes up to a minute...>

docker-machine env

eval $(docker-machine env)

Then (at least in my experience), if you ping google.com from a container all will be well.
</snip>

I tried this: -

docker-machine stop default
docker-machine start default

and, quelle surprise, testConnection just worked: -

docker build -t websphere-liberty:artifactory .

Sending build context to Docker daemon 137.8 MB
Step 1 : FROM websphere-liberty:webProfile7
 ---> 2594eeac8473
Step 2 : ADD server.xml /opt/ibm/wlp/usr/servers/defaultServer/
 ---> Using cache
 ---> 147f20ffdbe6
Step 3 : ADD artifactory.war /opt/ibm/wlp/usr/servers/defaultServer/dropins/
 ---> Using cache
 ---> b53879baf155
Step 4 : ADD SuperSnoopWeb.war /opt/ibm/wlp/usr/servers/defaultServer/dropins/
 ---> Using cache
 ---> 4e968a92672d
Step 5 : ENV LICENSE accept
 ---> Using cache
 ---> 3dc40c65b753
Step 6 : RUN installUtility testConnection
 ---> Using cache
 ---> 982821eafe3cSuccessfully built 982821eafe3c

which is nice :-)

Microsoft Works 2000 to Microsoft Excel 2003 - LibreOffice has the answer

So I've given away the plot, to misquote Kenny Everett …

One of my family members had a spreadsheet, which she was struggling to open in Microsoft Excel.

Originally, the spreadsheet had been created in Microsoft Works 2000, on Windows 98.

Time has moved on ...

Now we have Excel, as part of Office 2003, which doesn't appear to have an import filter for Works.

The internet was full of solutions, including a less-than-helpful Microsoft article: -

Works: How to import and export Works spreadsheets to Excel

which assumes that one still has Works :-(

I even considered, but quickly rejected, various pieces of "Too good to be true" conversion software which just may be malware.

Oh, and a website that wanted to charge me money for a "service" ….

I should've known that there'd be a better solution …

I mailed the spreadsheet to myself, downloaded it to my Mac, double-clicked on it, and up it popped in … 

I was able to quickly save it in the right format: -

and mail it back.

Hopefully the job is a good 'un ….

PS For the record, I also considered using Google Docs - I've just checked, and that'd also have worked a treat :-)

Jenkins and Chef Compared - Someone else's PoV

I'm thinking about DevOps, after a weekend of tinkering with WebSphere Liberty Profile and Docker.

So I asked the internet about Jenkins and Chef, and this is someone's PoV: -

Jenkins is a continuous integration server, it's goal is to allow you to build and test the software you develop. The main strength of Jenkins are:

• Integration with source control (SVN, Git, ...)

• Integration with build systems (sometimes thru plugins), e.g. Maven for Java, XCode for iOS development, ...

• Worker nodes (slaves) are super easy to add (you just need to provide ssh access information)

• Huge repository of plugins for most of the use cases you can think of

The main unit of Jenkins is a Job, usually compilation and test of a piece of software. The common workflow is:

• Something is changed in source control

• Jenkins polls source control, triggers a build for the corresponding job(s)

• The job runs, reports any build or test failure, and stores build artifacts (result executable, .war file, etc ...)

Jenkins support dependency between jobs.

Chef is meant to automate deployment of application stack on servers.

The main unit of Chef is a cookbook. A cookbook contain the list of recipes needed to deploy an application. A recipe can include recipes from other cookbooks, i.e. you can have dependencies between cookbooks.

The main strength of Chef are:

• Big repository of public cookbooks (many of them maintained by Opscode) on http://community.opscode.com/

• Idempotency: cookbooks are meant to be idempotent, i.e. they always bring the system to the same state whatever the original state was, further, if no changes are needed, recipes executes pretty fast.

• erubis template engine for templating configuration files

• abstraction over most of the system resources you would need (e.g. files, services, ...), so that cookbook can be designed somehow independently of the Linux distribution used on the server.

So in short, Jenkins is used to build and test, and Chef is used to deploy. Still you can use Jenkins to deploy if your application stack is simple enough.

Source: What is the difference between Jenkins and Chef?

Error "com.sun.jersey.core.spi.scanning.ScannerException" seen whilst using Artifactory on WebSphere Liberty Profile

I'm tinkering with a new (to me) tool, JFrog Artifactory, which is a Universal Repository Manager, on IBM WebSphere Liberty Profile (WLP).

My objective is to run Artifactory on WLP within a Docker container.

However, I wanted to ensure that it worked on WLP, before adding the additional "complexity" of Docker.

This is in line with my "Make a change, test a change" approach to life, the universe, and everything.

I'm doing this on my Mac using the latest version of WLP: -

wlp-webProfile7-16.0.0.2.zip

This is my configuration: -

 ~/Downloads/wlp/usr/servers/defaultServer/server.xml

<?xml version="1.0" encoding="UTF-8"?>
<server description="new server">

    <!-- Enable features -->
    <featureManager>
        <feature>webProfile-7.0</feature>
        <feature>adminCenter-1.0</feature>
    </featureManager>

    <classloading useJarUrls="true"/>

    <quickStartSecurity userName="admin" userPassword="password"/>

    <keyStore id="defaultKeyStore" password="Liberty"/>

    <!-- To access this server from a remote client add a host attribute to the following element, e.g. host="*" -->
    <httpEndpoint id="defaultHttpEndpoint"
                  httpPort="9080"
                  httpsPort="9443" />
                  
    <!-- Automatically expand WAR files and EAR files -->
    <applicationManager autoExpand="true"/>

    <remoteFileAccess>
        <writeDir>${server.config.dir}</writeDir>
    </remoteFileAccess>

</server>

and I merely dropped the Artifactory WAR file into the dropins folder: -

 ~/Downloads/wlp/usr/servers/defaultServer/dropins/

-rw-r--r--@ 1 davidhay  staff  37669474 27 Aug 09:39 artifactory.war

However, once I started Liberty: -

~/Downloads/wlp/bin/server start

and tried to access Artifactory: -

https://localhost:9443/artifactory/

things went a bit awry: -

2016-08-27 09:43:16,097 [art-init] [INFO ] (o.a.s.ArtifactoryApplicationContext:404) - Artifactory application context is ready.
2016-08-27 09:43:16,103 [art-init] [INFO ] (o.a.w.s.ArtifactoryContextConfigListener:221) - 
###########################################################
### Artifactory successfully started (6.843 seconds)    ###
###########################################################

2016-08-27 09:43:16,122 [art-init] [ERROR] (o.a.w.s.ArtifactoryContextConfigListener:110) - Could not init org.artifactory.rest.servlet.ArtifactoryRestServlet.
javax.servlet.ServletException: java.lang.reflect.InvocationTargetException
at com.sun.jersey.spi.container.servlet.WebComponent.createResourceConfig(WebComponent.java:728) ~[jersey-servlet-1.19.jar:1.19]
at com.sun.jersey.spi.container.servlet.WebComponent.createResourceConfig(WebComponent.java:674) ~[jersey-servlet-1.19.jar:1.19]
at com.sun.jersey.spi.container.servlet.WebComponent.init(WebComponent.java:205) ~[jersey-servlet-1.19.jar:1.19]
at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:394) ~[jersey-servlet-1.19.jar:1.19]
at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:577) ~[jersey-servlet-1.19.jar:1.19]
at javax.servlet.GenericServlet.init(GenericServlet.java:244) ~[com.ibm.ws.javaee.servlet.3.1_1.0.13.jar:na]
at org.artifactory.rest.servlet.ArtifactoryRestServlet.delayedInit(ArtifactoryRestServlet.java:73) ~[artifactory-rest-4.11.2.jar:na]
at org.artifactory.webapp.servlet.ArtifactoryContextConfigListener$1.run(ArtifactoryContextConfigListener.java:108) ~[artifactory-web-application-4.11.2.jar:na]
Caused by: java.lang.reflect.InvocationTargetException: null
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_101]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_101]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_101]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_101]
at com.sun.jersey.spi.container.servlet.WebComponent.createResourceConfig(WebComponent.java:724) ~[jersey-servlet-1.19.jar:1.19]
... 7 common frames omitted
Caused by: com.sun.jersey.core.spi.scanning.ScannerException: The URI scheme wsjar of the URI wsjar:file:/Users/davidhay/Downloads/wlp/usr/servers/defaultServer/apps/expanded/artifactory.war/WEB-INF/lib/artifactory-common-4.11.2.jar!/org/artifactory/rest is not supported. Package scanning deployment is not supported for such URIs.

Apparently this is a known issue with certain OSS frameworks such as Hadoop and Jersey :-(

Thankfully, there's a solution …

PM99378: ALLOW LIBERTY PROFILE TO RETURN JAR URLS RATHER THAN WSJAR URLS FROM CLASSLOADERS.

…

Some open source frameworks throw errors when trying to process "wsjar:" URIs and WebSphere Application Server Liberty profile server does not provide a mechanism to return to "jar:" URIs.

The configuration element below can be added to the server.xml

file to enforce the JVM wide usage of "jar:" rather than

"wsjar:".

<classloading useJarUrls="true"/>

…

Once I restarted my server.xml : -

<?xml version="1.0" encoding="UTF-8"?>
<server description="new server">

    <!-- Enable features -->
    <featureManager>
        <feature>webProfile-7.0</feature>
        <feature>adminCenter-1.0</feature>
    </featureManager>

    <classloading useJarUrls="true"/>

    <quickStartSecurity userName="admin" userPassword="password"/>

    <keyStore id="defaultKeyStore" password="Liberty"/>

    <!-- To access this server from a remote client add a host attribute to the following element, e.g. host="*" -->
    <httpEndpoint id="defaultHttpEndpoint"
                  httpPort="9080"
                  httpsPort="9443" />

    <!-- Automatically expand WAR files and EAR files -->
    <applicationManager autoExpand="true"/>

    <remoteFileAccess>
        <writeDir>${server.config.dir}</writeDir>
    </remoteFileAccess>

</server>

and restarted my server, all was well: -

Internet facing deployment of IBM Business Process Manager

This Technote was published yesterday: -

Internet facing deployment of IBM Business Process Manager

…

Question

How can IBM Business Process Manager capabilities be exposed in an internet facing deployment?

Introduction

IBM Business Process Manager is a powerful process development platform - designed to allow business analysts to model process flows and user interface elements in a single package. In order to achieve this goal, there must be compromises for example in functional scope and programming model:

• While most business analysts will be happy to not having to deal with all complexities of software development, programmers will likely observe a lack of control in certain areas.

• Similarly, creating coaches by dragging and dropping reusable components from a palette on a canvas is great, but does not provide the level of control that is required for pixel perfect user interfaces.

While the simplified programming model allows you to build applications with reasonable security, it is not on the same level as a lower level application development environment which for example gives you full control over HTTP requests. Also, the simplified programming model allows you to take shortcuts to build functional, but less secure applications.

…

Definitely worth a read ...

IBM MQ - Little Gems

This from an IBM guru, Morag Hughson

SSL Client Authentication, Certificate Authorisation and IBM HTTP Server

Context

To enable IBM HTTP Server (IHS) to "authorise" incoming requests based upon the "identity" of the client. This goes above and beyond SSL Client ( aka Mutual ) Authentication, where a server will require a client to present a valid certificate which both parties trust.

This leverages the SSLClientAuthRequire directive in IHS

Detail

I tested this using IHS 8.5.5.8 on both Red Hat Linux and IBM AIX, using the Firefox browser in both cases.

In the case of Linux, I created a pair of self-signed personal certificates on my Mac, imported them into Firefox ( using the PKCS12 keystone format which includes my personal key ), and configured IHS to accept ONE of the TWO certificates using the Common Name (CN) attribute.

Client Setup

Define Self-Signed Certificates

vi davehay1.conf

[req]
default_bits           = 2048
default_keyfile        = davehay.key
distinguished_name     = macintosh
attributes             = req_attributes
prompt                 = no
output_password        = passw0rd

[macintosh]
C                      = GB
ST                     = Hampshire
L                      = Winchester
O                      = IBM
OU                     = ICCTE
CN                     = macintosh1.uk.ibm.com

[req_attributes]
challengePassword      = passw0rd

vi davehay2.conf

[req]
default_bits           = 2048
default_keyfile        = davehay.key
distinguished_name     = macintosh
attributes             = req_attributes
prompt                 = no
output_password        = passw0rd

[macintosh]
C                      = GB
ST                     = Hampshire
L                      = Winchester
O                      = IBM
OU                     = ICCTE
CN                     = macintosh2.uk.ibm.com

[req_attributes]
challengePassword      = passw0rd

Create Self-Signed Certificates

The first also generates a private key

openssl req -config davehay1.conf -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout davehay.key -out davehay1.crt

The second uses the existing private key

openssl req -config davehay2.conf -x509 -sha256 -nodes -days 365 -key davehay.key -out davehay2.crt -new

Generate PKCS12 Certificate Stores - required for import into client browser

openssl pkcs12 -export -out davehay1.p12 -inkey davehay.key -in davehay1.crt -password pass:passw0rd

openssl pkcs12 -export -out davehay2.p12 -inkey davehay.key -in davehay2.crt -password pass:passw0rd

Import PKCS12 Stores into Firefox

This is available via the Preferences -> Advanced -> Certificates -> View Certificates dialogue



Send Mac Personal Certificates to IHS

cp *.crt wasadmin@bpm856:~

wasadmin@bpm856's password: 

davehay1.crt                                                                                                                                                              100% 1237     1.2KB/s   00:00    
davehay2.crt                                                                                                                                                              100% 1237     1.2KB/s   00:00    


Server Setup

Create Key/Trust store - required for IHS

/opt/IBM/HTTPServer/bin/gskcapicmd -keydb -create -db /opt/IBM/HTTPServer/ssl/keystore.kdb -pw passw0rd -type cms -expire 3650 -stash

Generate a Self-Signed Certificate - presented by IHS to client

/opt/IBM/HTTPServer/bin/gskcapicmd -cert -create -db /opt/IBM/HTTPServer/ssl/keystore.kdb -stashed -dn "cn=bpm856.uk.ibm.com,dc=uk,dc=ibm,dc=com" -label bpm856.uk.ibm.com_ss -default_cert yes

Add Mac Personal Certificates - required to establish trust between client browser and IHS

/opt/IBM/HTTPServer/bin/gskcapicmd -cert -add -db /opt/IBM/HTTPServer/ssl/keystore.kdb -stashed -file ~/davehay1.crt 

/opt/IBM/HTTPServer/bin/gskcapicmd -cert -add -db /opt/IBM/HTTPServer/ssl/keystore.kdb -stashed -file ~/davehay2.crt 

Validate Key/Trust store

/opt/IBM/HTTPServer/bin/gskcapicmd -cert -list -db /opt/IBM/HTTPServer/ssl/keystore.kdb -stashed

Certificates found
* default, - personal, ! trusted, # secret key
! CN=macintosh1.uk.ibm.com,OU=ICCTE,O=IBM,L=Winchester,ST=Hampshire,C=GB
! CN=macintosh2.uk.ibm.com,OU=ICCTE,O=IBM,L=Winchester,ST=Hampshire,C=GB
*- bpm856.uk.ibm.com_ss

Add SSLClientAuthRequire directive to IHS httpd.conf

...
SSLClientAuthRequire (CN = "macintosh1.uk.ibm.com" )
…

This means that IHS will only accept requests when presented with the macintosh1.uk.ibm.com personal certificate.

Functional Test using Firefox on Client

Access IHS: -

https://bpm856.uk.ibm.com:8443/

When prompted, choose to present macintosh1.uk.ibm.com personal certificate

Should be able to access IHS, as IHS "trusts" this certificate via the SSLClientAuthRequire directive.

Close Firefox

Re-run functional test - choose macintosh2.uk.ibm.com personal certificate

Should receive "You don't have permission to access / on this server." and this message: -

[Wed Aug 24 19:59:49 2016] [error] [client 192.168.153.1] [7feb140008c0] [31411] SSL0279E: SSL Handshake Failed due to fatal alert from client. Client sent fatal alert [level 2 (fatal), description 48 (unknown_ca)]  [192.168.153.1:60794 -> 192.168.153.200:8443] [19:59:49.000695476] 0ms

in IHS error_log, and: -

192.168.153.1 - - [24/Aug/2016:19:59:56 +0100] "GET / HTTP/1.1" 403 273
192.168.153.1 - - [24/Aug/2016:19:59:56 +0100] "GET /favicon.ico HTTP/1.1" 403 284
192.168.153.1 - - [24/Aug/2016:19:59:56 +0100] "GET /favicon.ico HTTP/1.1" 403 284

in IHS access_log, all because macintosh2.uk.ibm.com is NOT in the SSLClientAuthRequire directive.

As we discovered, the directive isn't a catch-all - the match has to be 100%, wildcards aren't an option - the match has to be for the precise value of the certificate's CN ( in this case ).

If, for example, a certificate has a wildcard ( * ) in the CN, such as *.uk.ibm.com, that's precisely what needs to go in the directive, wrapped in double-quotes.

This is nicely documented here: -

SSLClientAuthRequire directive

Simple when you know how :-)

DSRA8040I and SQL 1060N between IBM BPM and IBM DB2

We saw this yesterday: -

Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLNonTransientException: DB2 SQL Error: SQLCODE=-1060, SQLSTATE=08004, SQLERRMC=DB2USER1, DRIVER=4.19.49 DSRA0010E: SQL State = 08004, Error Code = -1,060

Caused by: java.sql.SQLNonTransientException: DB2 SQL Error: SQLCODE=-1060, SQLSTATE=08004, SQLERRMC=DB2USER1, DRIVER=4.19.49 DSRA0010E: SQL State = 08004, Error Code = -1,060

[23/08/16 15:00:04:760 BST] 000000fc DSConfigurati W   DSRA8201W: DataSource Configuration: DSRA8040I: Failed to connect to the DataSource jdbc/WPSDB.  Encountered java.sql.SQLNonTransientException: DB2USER1 DSRA0010E: SQL State = 08004, Error Code = -1,060.

java.sql.SQLNonTransientException: DB2USER1 DSRA0010E: SQL State = 08004, Error Code = -1,060

in the SystemOut.log of our WAS 8.5.5.8 Deployment Manager, which is hosting an IBM BPM Advanced 8.5.5 Deployment Environment.

This error occurred when we hit the "Test Connection" button against the various JDBC data sources used by BPM.

Specifically, it was from the two cell-scoped data sources - jdbc/WPSDB and jdbc/mediation/messageLog.

It turned out that the target DB2 server had a so-called Trusted Context that was prohibiting access from the WAS Deployment Manager, only allowing access from the actual WAS nodes.

Trusted Context is used to provide an additional level of access control to DB2 servers, restricting access to all but a specific set of hosts.

Typically, when one hits "Test Connection" button, most data sources, which are typically node/cluster/server-scoped, require connections ONLY from the nodes.

Hence the Trusted Context rules were correct ….

… for all but the two cell-scoped data sources.

Once the DBA added the Deployment Manager into the Trusted Context, we were good to go.

For more intel on Trusted Context, please see: -

Manage access with trusted context

Apache - SSL/TLS Strong Encryption: FAQ

The wise man doesn't give the right answers, he poses the right questions.

-- Claude Levi-Strauss

SSL/TLS Strong Encryption: FAQ

A teaser from the ToC

Installation

• Why do I get permission errors related to SSLMutex when I start Apache?

• Why does mod_ssl stop with the error "Failed to generate temporary 512 bit RSA private key" when I start Apache?

Configuration

• Is it possible to provide HTTP and HTTPS from the same server?

• Which port does HTTPS use?

• How do I speak HTTPS manually for testing purposes?

• Why does the connection hang when I connect to my SSL-aware Apache server?

• Why do I get ``Connection Refused'' errors, when trying to access my newly installed Apache+mod_ssl server via HTTPS?

• Why are the SSL_XXX variables not available to my CGI & SSI scripts?

• How can I switch between HTTP and HTTPS in relative hyperlinks?

IBM on Docker - My Next Thing

I'm looking at a new piece of discovery and self-development, so I'll be digging into Chef, Puppet, UrbanCode Deploy *AND* Docker 

So this will be required reading: -

WebSphere under Docker: from Development to Production

TL;DR You can now take your Docker images for WebSphere Application Server Full and Liberty Profile into production and get support. We've provided some resources to build production Liberty images and made them available on WASdev GitHub.

Integrate Docker in your IBM Operational Decision Manager environment

IBM® Operational Decision Manager (ODM) application developers who are new to Docker might consider adding Docker to their development process. This tutorial shows how to build a Docker image for an IBM ODM Rule Execution Server and Decision Center that run on a WebSphere Liberty server. You also learn how to run the Rule Execution Server and Decision Center in a container from the Docker image. This content is part of the IBM Business Process Management Journal.

Watch this space

Gah, updating Windows 7

I keep hitting this one.

I've just installed a "new" Windows 7 VM, and was getting annoyed with the lack of Windows Updates.

This nailed it for me: -

Installing and searching for updates is slow and high CPU usage occurs in Windows 7 and Windows Server 2008 R2

which led me to this: -

Update for Windows 7 for x64-based Systems (KB3102810)

Job's a good 'un

IBM Redbooks | Deliver Modern UI for IBM BPM with the Coach Framework and Other Approaches

The IBM® Coach Framework is a key component of IBM Business Process Manager (BPM) that enables custom user interfaces (UI) to be easily embedded within business process solutions. Developer tools enable process authors to rapidly create a compelling user experience that can be delivered to desktop and mobile devices. The IBM Process Portal used by business operations to access, execute and manage their tasks is entirely coach-based and can easily be configured and styled. The corporate look and feel can be defined using a graphical theme editor and applied consistently across all process applications. The process federation capability enables business users to access and execute all their tasks using a single UI without being aware of the implementation or origin. Learn how to embed coach-based UI in other web applications, develop BPM UI using alternative UI technology, and create mobile apps for off-line working.

This IBM Redbooks® publication focuses on the capabilities that Coach Framework delivers with IBM BPM version 8.5.7, but much of what is shared in these pages continues to be of value as IBM evolves coaches in the future. This book has been produced to help you fully benefit from the power of the Coach Framework.

Deliver Modern UI for IBM BPM with the Coach Framework and Other Approaches

IBM BPM Security Hardening - use a web server

This from my IBM colleague, Jens Engelke, is definitely worth reading, bookmarking AND following :-)

IBM BPM Security Hardening - use a web server

This post is part of the IBM BPM Security Hardening series.

The first rule of secure communications here is that no end user should ever reach a BPM application server directly. While earlier versions of IBM BPM required end users (at least developers using Process Designer) to access a BPM server via multiple protocols. There was HTTP traffic, EJB traffic and JMS traffic. This is no longer the case as of IBM BPM V8.5.0.1 and later. HTTPS is the only required protocol so all traffic can pass through a web server in a demilitarized zone – matching the first rule in the WebSphere Application Server hardening guide.

Active Directory Certificate Services - "The revocation function was unable to check revocation because the revocation server was offline" - or the world's best error message

I saw this: -

Active Directory Certificate Services denied request 10 because The revocation function was unable to check revocation because the revocation server was offline. 0x80092013 (-2146885613).  The request was for CN=bpm856.uk.ibm.com, DC=uk, DC=ibm, DC=com.  Additional information: Error Constructing or Publishing Certificate

 earlier, whilst trying to issue a SSL certificate using AD Certificate Services: -

certreq -submit -attrib "CertificateTemplate:Webserver"  bpm856.uk.ibm.com_ihs.req bpm856.uk.ibm.com_ihs.cer

As ever, the solution was simple - I merely had to "turn it off and on again"

IBM Design Thinking - the Who/What/Wow factor

I was Slacking ( that's a verb now ! ) with a friend earlier about IBM Design Thinking, and referenced the Who/What/Wow

sourced from the IBM Design Thinking site, which is definitely worth a look-see.

Keyboard shortcuts in IBM Notes - revisited

Re-using one of my posts from 2008 !!

Control Key, Me Control - Anyone want a mouse ???

in case I ever forget …..

In IBM Notes on Mac OS X, press [cmd][shift][l] to bring up this: -

Note that the list is scrollable, and there's a fair few shortcuts there :-)