Sunday, 28 August 2016

Microsoft Works 2000 to Microsoft Excel 2003 - LibreOffice has the answer

So I've given away the plot, to misquote Kenny Everett …

One of my family members had a spreadsheet, which she was struggling to open in Microsoft Excel.

Originally, the spreadsheet had been created in Microsoft Works 2000, on Windows 98.

Time has moved on ...

Now we have Excel, as part of Office 2003, which doesn't appear to have an import filter for Works.

The internet was full of solutions, including a less-than-helpful Microsoft article: -


which assumes that one still has Works :-(

I even considered, but quickly rejected, various pieces of "Too good to be true" conversion software which just may be malware.

Oh, and a website that wanted to charge me money for a "service" ….

I should've known that there'd be a better solution …

I mailed the spreadsheet to myself, downloaded it to my Mac, double-clicked on it, and up it popped in … 


I was able to quickly save it in the right format: -


and mail it back.

Hopefully the job is a good 'un ….

PS For the record, I also considered using Google Docs - I've just checked, and that'd also have worked a treat :-)

Jenkins and Chef Compared - Someone else's PoV

I'm thinking about DevOps, after a weekend of tinkering with WebSphere Liberty Profile and Docker.

So I asked the internet about Jenkins and Chef, and this is someone's PoV: -

Jenkins is a continuous integration server, it's goal is to allow you to build and test the software you develop. The main strength of Jenkins are:

• Integration with source control (SVN, Git, ...)
• Integration with build systems (sometimes thru plugins), e.g. Maven for Java, XCode for iOS development, ...
• Worker nodes (slaves) are super easy to add (you just need to provide ssh access information)
• Huge repository of plugins for most of the use cases you can think of

The main unit of Jenkins is a Job, usually compilation and test of a piece of software. The common workflow is:

• Something is changed in source control
• Jenkins polls source control, triggers a build for the corresponding job(s)
• The job runs, reports any build or test failure, and stores build artifacts (result executable, .war file, etc ...)

Jenkins support dependency between jobs.

Chef is meant to automate deployment of application stack on servers.

The main unit of Chef is a cookbook. A cookbook contain the list of recipes needed to deploy an application. A recipe can include recipes from other cookbooks, i.e. you can have dependencies between cookbooks.

The main strength of Chef are:

• Big repository of public cookbooks (many of them maintained by Opscode) on http://community.opscode.com/
• Idempotency: cookbooks are meant to be idempotent, i.e. they always bring the system to the same state whatever the original state was, further, if no changes are needed, recipes executes pretty fast.
• erubis template engine for templating configuration files
• abstraction over most of the system resources you would need (e.g. files, services, ...), so that cookbook can be designed somehow independently of the Linux distribution used on the server.

So in short, Jenkins is used to build and test, and Chef is used to deploy. Still you can use Jenkins to deploy if your application stack is simple enough.

Saturday, 27 August 2016

Error "com.sun.jersey.core.spi.scanning.ScannerException" seen whilst using Artifactory on WebSphere Liberty Profile

I'm tinkering with a new (to me) tool, JFrog Artifactory, which is a Universal Repository Manager, on IBM WebSphere Liberty Profile (WLP).

My objective is to run Artifactory on WLP within a Docker container.

However, I wanted to ensure that it worked on WLP, before adding the additional "complexity" of Docker.

This is in line with my "Make a change, test a change" approach to life, the universe, and everything.

I'm doing this on my Mac using the latest version of WLP: -

wlp-webProfile7-16.0.0.2.zip

This is my configuration: -

 ~/Downloads/wlp/usr/servers/defaultServer/server.xml

<?xml version="1.0" encoding="UTF-8"?>
<server description="new server">

    <!-- Enable features -->
    <featureManager>
        <feature>webProfile-7.0</feature>
        <feature>adminCenter-1.0</feature>
    </featureManager>

    <classloading useJarUrls="true"/>

    <quickStartSecurity userName="admin" userPassword="password"/>

    <keyStore id="defaultKeyStore" password="Liberty"/>

    <!-- To access this server from a remote client add a host attribute to the following element, e.g. host="*" -->
    <httpEndpoint id="defaultHttpEndpoint"
                  httpPort="9080"
                  httpsPort="9443" />
                  
    <!-- Automatically expand WAR files and EAR files -->
    <applicationManager autoExpand="true"/>

    <remoteFileAccess>
        <writeDir>${server.config.dir}</writeDir>
    </remoteFileAccess>

</server>
and I merely dropped the Artifactory WAR file into the dropins folder: -

 ~/Downloads/wlp/usr/servers/defaultServer/dropins/

-rw-r--r--@ 1 davidhay  staff  37669474 27 Aug 09:39 artifactory.war

However, once I started Liberty: -

~/Downloads/wlp/bin/server start

and tried to access Artifactory: -


things went a bit awry: -

2016-08-27 09:43:16,097 [art-init] [INFO ] (o.a.s.ArtifactoryApplicationContext:404) - Artifactory application context is ready.
2016-08-27 09:43:16,103 [art-init] [INFO ] (o.a.w.s.ArtifactoryContextConfigListener:221) - 
###########################################################
### Artifactory successfully started (6.843 seconds)    ###
###########################################################

2016-08-27 09:43:16,122 [art-init] [ERROR] (o.a.w.s.ArtifactoryContextConfigListener:110) - Could not init org.artifactory.rest.servlet.ArtifactoryRestServlet.
javax.servlet.ServletException: java.lang.reflect.InvocationTargetException
at com.sun.jersey.spi.container.servlet.WebComponent.createResourceConfig(WebComponent.java:728) ~[jersey-servlet-1.19.jar:1.19]
at com.sun.jersey.spi.container.servlet.WebComponent.createResourceConfig(WebComponent.java:674) ~[jersey-servlet-1.19.jar:1.19]
at com.sun.jersey.spi.container.servlet.WebComponent.init(WebComponent.java:205) ~[jersey-servlet-1.19.jar:1.19]
at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:394) ~[jersey-servlet-1.19.jar:1.19]
at com.sun.jersey.spi.container.servlet.ServletContainer.init(ServletContainer.java:577) ~[jersey-servlet-1.19.jar:1.19]
at javax.servlet.GenericServlet.init(GenericServlet.java:244) ~[com.ibm.ws.javaee.servlet.3.1_1.0.13.jar:na]
at org.artifactory.rest.servlet.ArtifactoryRestServlet.delayedInit(ArtifactoryRestServlet.java:73) ~[artifactory-rest-4.11.2.jar:na]
at org.artifactory.webapp.servlet.ArtifactoryContextConfigListener$1.run(ArtifactoryContextConfigListener.java:108) ~[artifactory-web-application-4.11.2.jar:na]
Caused by: java.lang.reflect.InvocationTargetException: null
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_101]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_101]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_101]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_101]
at com.sun.jersey.spi.container.servlet.WebComponent.createResourceConfig(WebComponent.java:724) ~[jersey-servlet-1.19.jar:1.19]
... 7 common frames omitted
Caused by: com.sun.jersey.core.spi.scanning.ScannerException: The URI scheme wsjar of the URI wsjar:file:/Users/davidhay/Downloads/wlp/usr/servers/defaultServer/apps/expanded/artifactory.war/WEB-INF/lib/artifactory-common-4.11.2.jar!/org/artifactory/rest is not supported. Package scanning deployment is not supported for such URIs.


Apparently this is a known issue with certain OSS frameworks such as Hadoop and Jersey :-(

Thankfully, there's a solution …


Some open source frameworks throw errors when trying to process "wsjar:" URIs and WebSphere Application Server Liberty profile server does not provide a mechanism to return to "jar:" URIs.

The configuration element below can be added to the server.xml
file to enforce the JVM wide usage of "jar:" rather than
"wsjar:".

<classloading useJarUrls="true"/>

Once I restarted my server.xml : -

<?xml version="1.0" encoding="UTF-8"?>
<server description="new server">

    <!-- Enable features -->
    <featureManager>
        <feature>webProfile-7.0</feature>
        <feature>adminCenter-1.0</feature>
    </featureManager>

    <classloading useJarUrls="true"/>

    <quickStartSecurity userName="admin" userPassword="password"/>

    <keyStore id="defaultKeyStore" password="Liberty"/>

    <!-- To access this server from a remote client add a host attribute to the following element, e.g. host="*" -->
    <httpEndpoint id="defaultHttpEndpoint"
                  httpPort="9080"
                  httpsPort="9443" />

    <!-- Automatically expand WAR files and EAR files -->
    <applicationManager autoExpand="true"/>


    <remoteFileAccess>
        <writeDir>${server.config.dir}</writeDir>
    </remoteFileAccess>

</server>

and restarted my server, all was well: -



Internet facing deployment of IBM Business Process Manager

This Technote was published yesterday: -

Question

How can IBM Business Process Manager capabilities be exposed in an internet facing deployment?

Introduction

IBM Business Process Manager is a powerful process development platform - designed to allow business analysts to model process flows and user interface elements in a single package. In order to achieve this goal, there must be compromises for example in functional scope and programming model:

• While most business analysts will be happy to not having to deal with all complexities of software development, programmers will likely observe a lack of control in certain areas.
• Similarly, creating coaches by dragging and dropping reusable components from a palette on a canvas is great, but does not provide the level of control that is required for pixel perfect user interfaces.

While the simplified programming model allows you to build applications with reasonable security, it is not on the same level as a lower level application development environment which for example gives you full control over HTTP requests. Also, the simplified programming model allows you to take shortcuts to build functional, but less secure applications.

Definitely worth a read ...

Wednesday, 24 August 2016

SSL Client Authentication, Certificate Authorisation and IBM HTTP Server

Context

To enable IBM HTTP Server (IHS) to "authorise" incoming requests based upon the "identity" of the client. This goes above and beyond SSL Client ( aka Mutual ) Authentication, where a server will require a client to present a valid certificate which both parties trust.

This leverages the SSLClientAuthRequire directive in IHS

Detail

I tested this using IHS 8.5.5.8 on both Red Hat Linux and IBM AIX, using the Firefox browser in both cases.

In the case of Linux, I created a pair of self-signed personal certificates on my Mac, imported them into Firefox ( using the PKCS12 keystone format which includes my personal key ), and configured IHS to accept ONE of the TWO certificates using the Common Name (CN) attribute.

Client Setup

Define Self-Signed Certificates

vi davehay1.conf

[req]
default_bits           = 2048
default_keyfile        = davehay.key
distinguished_name     = macintosh
attributes             = req_attributes
prompt                 = no
output_password        = passw0rd

[macintosh]
C                      = GB
ST                     = Hampshire
L                      = Winchester
O                      = IBM
OU                     = ICCTE
CN                     = macintosh1.uk.ibm.com

[req_attributes]
challengePassword      = passw0rd


vi davehay2.conf

[req]
default_bits           = 2048
default_keyfile        = davehay.key
distinguished_name     = macintosh
attributes             = req_attributes
prompt                 = no
output_password        = passw0rd

[macintosh]
C                      = GB
ST                     = Hampshire
L                      = Winchester
O                      = IBM
OU                     = ICCTE
CN                     = macintosh2.uk.ibm.com

[req_attributes]
challengePassword      = passw0rd

Create Self-Signed Certificates

The first also generates a private key

openssl req -config davehay1.conf -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout davehay.key -out davehay1.crt

The second uses the existing private key

openssl req -config davehay2.conf -x509 -sha256 -nodes -days 365 -key davehay.key -out davehay2.crt -new

Generate PKCS12 Certificate Stores - required for import into client browser

openssl pkcs12 -export -out davehay1.p12 -inkey davehay.key -in davehay1.crt -password pass:passw0rd

openssl pkcs12 -export -out davehay2.p12 -inkey davehay.key -in davehay2.crt -password pass:passw0rd


Import PKCS12 Stores into Firefox

This is available via the Preferences -> Advanced -> Certificates -> View Certificates dialogue



Send Mac Personal Certificates to IHS

cp *.crt wasadmin@bpm856:~

wasadmin@bpm856's password: 
davehay1.crt                                                                                                                                                              100% 1237     1.2KB/s   00:00    
davehay2.crt                                                                                                                                                              100% 1237     1.2KB/s   00:00    



Server Setup

Create Key/Trust store - required for IHS

/opt/IBM/HTTPServer/bin/gskcapicmd -keydb -create -db /opt/IBM/HTTPServer/ssl/keystore.kdb -pw passw0rd -type cms -expire 3650 -stash

Generate a Self-Signed Certificate - presented by IHS to client

/opt/IBM/HTTPServer/bin/gskcapicmd -cert -create -db /opt/IBM/HTTPServer/ssl/keystore.kdb -stashed -dn "cn=bpm856.uk.ibm.com,dc=uk,dc=ibm,dc=com" -label bpm856.uk.ibm.com_ss -default_cert yes

Add Mac Personal Certificates - required to establish trust between client browser and IHS

/opt/IBM/HTTPServer/bin/gskcapicmd -cert -add -db /opt/IBM/HTTPServer/ssl/keystore.kdb -stashed -file ~/davehay1.crt 

/opt/IBM/HTTPServer/bin/gskcapicmd -cert -add -db /opt/IBM/HTTPServer/ssl/keystore.kdb -stashed -file ~/davehay2.crt 


Validate Key/Trust store

/opt/IBM/HTTPServer/bin/gskcapicmd -cert -list -db /opt/IBM/HTTPServer/ssl/keystore.kdb -stashed

Certificates found
* default, - personal, ! trusted, # secret key
! CN=macintosh1.uk.ibm.com,OU=ICCTE,O=IBM,L=Winchester,ST=Hampshire,C=GB
! CN=macintosh2.uk.ibm.com,OU=ICCTE,O=IBM,L=Winchester,ST=Hampshire,C=GB
*- bpm856.uk.ibm.com_ss


Add SSLClientAuthRequire directive to IHS httpd.conf

...
SSLClientAuthRequire (CN = "macintosh1.uk.ibm.com" )


This means that IHS will only accept requests when presented with the macintosh1.uk.ibm.com personal certificate.

Functional Test using Firefox on Client

Access IHS: -

https://bpm856.uk.ibm.com:8443/

When prompted, choose to present macintosh1.uk.ibm.com personal certificate

Should be able to access IHS, as IHS "trusts" this certificate via the SSLClientAuthRequire directive.

Close Firefox

Re-run functional test - choose macintosh2.uk.ibm.com personal certificate

Should receive "You don't have permission to access / on this server." and this message: -

[Wed Aug 24 19:59:49 2016] [error] [client 192.168.153.1] [7feb140008c0] [31411] SSL0279E: SSL Handshake Failed due to fatal alert from client. Client sent fatal alert [level 2 (fatal), description 48 (unknown_ca)]  [192.168.153.1:60794 -> 192.168.153.200:8443] [19:59:49.000695476] 0ms

in IHS error_log, and: -

192.168.153.1 - - [24/Aug/2016:19:59:56 +0100] "GET / HTTP/1.1" 403 273
192.168.153.1 - - [24/Aug/2016:19:59:56 +0100] "GET /favicon.ico HTTP/1.1" 403 284
192.168.153.1 - - [24/Aug/2016:19:59:56 +0100] "GET /favicon.ico HTTP/1.1" 403 284


in IHS access_log, all because macintosh2.uk.ibm.com is NOT in the SSLClientAuthRequire directive.

As we discovered, the directive isn't a catch-all - the match has to be 100%, wildcards aren't an option - the match has to be for the precise value of the certificate's CN ( in this case ).

If, for example, a certificate has a wildcard ( * ) in the CN, such as *.uk.ibm.com, that's precisely what needs to go in the directive, wrapped in double-quotes.

This is nicely documented here: -

Simple when you know how :-)

DSRA8040I and SQL 1060N between IBM BPM and IBM DB2

We saw this yesterday: -

Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLNonTransientException: DB2 SQL Error: SQLCODE=-1060, SQLSTATE=08004, SQLERRMC=DB2USER1, DRIVER=4.19.49 DSRA0010E: SQL State = 08004, Error Code = -1,060

Caused by: java.sql.SQLNonTransientException: DB2 SQL Error: SQLCODE=-1060, SQLSTATE=08004, SQLERRMC=DB2USER1, DRIVER=4.19.49 DSRA0010E: SQL State = 08004, Error Code = -1,060

[23/08/16 15:00:04:760 BST] 000000fc DSConfigurati W   DSRA8201W: DataSource Configuration: DSRA8040I: Failed to connect to the DataSource jdbc/WPSDB.  Encountered java.sql.SQLNonTransientException: DB2USER1 DSRA0010E: SQL State = 08004, Error Code = -1,060.

java.sql.SQLNonTransientException: DB2USER1 DSRA0010E: SQL State = 08004, Error Code = -1,060

in the SystemOut.log of our WAS 8.5.5.8 Deployment Manager, which is hosting an IBM BPM Advanced 8.5.5 Deployment Environment.

This error occurred when we hit the "Test Connection" button against the various JDBC data sources used by BPM.

Specifically, it was from the two cell-scoped data sources - jdbc/WPSDB and jdbc/mediation/messageLog.

It turned out that the target DB2 server had a so-called Trusted Context that was prohibiting access from the WAS Deployment Manager, only allowing access from the actual WAS nodes.

Trusted Context is used to provide an additional level of access control to DB2 servers, restricting access to all but a specific set of hosts.

Typically, when one hits "Test Connection" button, most data sources, which are typically node/cluster/server-scoped, require connections ONLY from the nodes.

Hence the Trusted Context rules were correct ….

… for all but the two cell-scoped data sources.

Once the DBA added the Deployment Manager into the Trusted Context, we were good to go.

For more intel on Trusted Context, please see: -