Wednesday, 30 March 2016

Oops, my WAS upgrade broke my TLS

I saw this earlier today: -

[30/03/16 11:35:53:371 BST] 00000001 ORBRas        E com.ibm.ws.orbimpl.transport.WSTransport createServerSocket P=152627:O=0:CT ORBX0390E: Cannot create listener thread. Exception=[ org.omg.CORBA.INTERNAL: CAUGHT_EXCEPTION_WHILE_CONFIGURING_SSL_SERVER_SOCKET, Exception=java.lang.IllegalArgumentException: Cannot support SSL_ECDHE_RSA_WITH_AES_256_GCM_SHA384 with currently installed providers vmcid: 0x49421000  minor code: 77  completed: No - received while attempting to open server socket on port 9403 ].

[30/03/16 11:35:53:397 BST] 00000001 FfdcProvider  W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/ibm/WebSphereProfiles/ODMCell1Dmgr01/logs/ffdc/dmgr_498f4d6b_16.03.30_11.35.53.3748582431257869454898.txt com.ibm.ws.orbimpl.transport.WSTransport.startListening 805

[30/03/16 11:35:53:419 BST] 00000001 FfdcProvider  W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/ibm/WebSphereProfiles/ODMCell1Dmgr01/logs/ffdc/dmgr_498f4d6b_16.03.30_11.35.53.3975558083781694628181.txt com.ibm.ws.orbimpl.transport.WSTransport.createListener 724

[30/03/16 11:35:53:420 BST] 00000001 WsServerImpl  E   WSVR0009E: Error occurred during startup com.ibm.ws.exception.RuntimeError: org.omg.CORBA.INTERNAL: CREATE_LISTENER_FAILED_4  vmcid: 0x49421000  minor code: 56  completed: No

whilst starting a WAS ND Deployment Manager.

Coincidentally (!) this happened RIGHT after I'd upgraded from WAS 8.5.5.4 to 8.5.5.8.

A quick Google search led me here: -


which said, in part: -

The problem can be related to unrestricted policy file due to SDK upgrade. Please download/install unrestricted policy files, the steps can be found at: http://www14.software.ibm.com/webapp/wsbroker/redirect?version=phil&product=was-nd-dist&topic=tsecegs (Under "Complete the following steps to download and install the new policy files:").

at which point I metaphorically slapped my forehead and said "Doh!".

I fired up my trusty CipherTest Java class: -

java -cp /mnt/Utilities/ CipherTest

which reported: -

FAILED: Max AES key length too small! (128).

I ran my script to update the Java policies to use the unrestricted world-wide ciphers: -

#!/bin/bash
cd /opt/ibm/WebSphere/AppServer/java/jre/lib/security
mv local_policy.jar local_policy.raj
mv US_export_policy.jar US_export_policy.raj
/opt/ibm/WebSphere/AppServer/java/bin/jar xvf /mnt/Java/unrestrictedpolicyfiles.zip
chmod a+rwx *.jar

and re-ran the CipherTest: -

java -cp /mnt/Utilities/ CipherTest

PASSED: Max AES key length OK! - >= 256 (2147483647).

Sorted :-) And "Doh!" And "SLAP!"

Monday, 28 March 2016

Thycotic - Good insight into SSL/TLS

I found this: -


whilst doing some research into the SSL/TLS ciphers supported by various versions of Windows.

Whilst I was specifically reading Part 2, the complete set of articles is definitely something to which I'm going to return and return.

Nice job, Thycotic.

Saturday, 26 March 2016

Using OpenSSL to generate personal certificates on Mac OS X and Windows

Following on from a previous post: -


and, from my ongoing voyage of discovery this weekend: -

Mac

Generate Private Key

openssl genrsa -out ~/macintosh.uk.ibm.com.key 2048

Create Certificate Service Request

openssl req -new -sha256 -key ~/macintosh.uk.ibm.com.key -out ~/macintosh.uk.ibm.com.csr


Generate Personal Certificate using Certificate Service Request ( on Windows Server 2008 R2 )

certreq.exe -submit -attrib "CertificateTemplate:User" macintosh.uk.ibm.com.csr macintosh.uk.ibm.com.cer

Combine Private Key, Personal Certificate and Certificate Service Request into PKCS12 Export File ( .pfx )

openssl pkcs12 -export -out macintosh.uk.ibm.com.pfx -inkey macintosh.uk.ibm.com.key -in macintosh.uk.ibm.com.cer -certfile macintosh.uk.ibm.com.csr 

The .pfx file can then be imported into a browser such as Firefox, which expects an import to (a) include the private key (b) include the personal certificate and (c) have a password set on the import file

Windows

Setup Path to OpenSSL Configuration

set OPENSSL_CONF=c:\OpenSSL-Win32\bin\openssl.cfg

Generate Private Key

c:\OpenSSL-Win32\bin\openssl.exe genrsa -out c:\Users\Administrator\Desktop\w2k8.uk.ibm.com.key 2048

Create Certificate Service Request

c:\OpenSSL-Win32\bin\openssl.exe req -new -sha256 -key c:\Users\Administrator\Desktop\w2k8.uk.ibm.com.key -out c:\Users\Administrator\Desktop\w2k8.uk.ibm.com.csr

Generate Personal Certificate using Certificate Service Request ( on Windows Server 2008 R2 )

certreq.exe -submit -attrib "CertificateTemplate:User" c:\Users\Administrator\Desktop\w2k8.uk.ibm.com.csr c:\Users\Administrator\Desktop\w2k8.uk.ibm.com.cer

Combine Private Key, Personal Certificate and Certificate Service Request into PKCS12 Export File ( .pfx )

c:\OpenSSL-Win32\bin\openssl.exe pkcs12 -export -out c:\Users\Administrator\Desktop\w2k8.uk.ibm.com.pfx -inkey c:\Users\Administrator\Desktop\w2k8.uk.ibm.com.key -in c:\Users\Administrator\Desktop\w2k8.uk.ibm.com.cer -certfile c:\Users\Administrator\Desktop\w2k8.uk.ibm.com.csr 

- Note; this fails with unable to load certificates - it appears that, with Windows, one does NOT need to provide the CSR, merely the private key and personal certificate

c:\OpenSSL-Win32\bin\openssl.exe pkcs12 -export -out c:\Users\Administrator\Desktop\w2k8.uk.ibm.com.pfx -in c:\Users\Administrator\Desktop\w2k8.uk.ibm.com.cer -inkey c:\Users\Administrator\Desktop\w2k8.uk.ibm.com.key

For the record, on the Mac, I have this version of OpenSSL: -

openssl version

OpenSSL 0.9.8zh 14 Jan 2016

c:\OpenSSL-Win32\bin\openssl.exe version

OpenSSL 1.0.2g  1 Mar 2016

"javax.net.ssl.SSLHandshakeException: no cipher suites in common" seen in WebSphere Application Server Node Agent logs

Following hot on the heels from my last post: -


I started seeing: -

[26/03/16 15:38:45:206 GMT] 00000048 SSLHandshakeE E   SSLC0008E: Unable to initialize SSL connection.  Unauthorized access was denied or security settings have expired.  Exception is javax.net.ssl.SSLHandshakeException: no cipher suites in common
Caused by: javax.net.ssl.SSLHandshakeException: no cipher suites in common
[26/03/16 15:41:13:771 GMT] 00000075 ORBRas        E com.ibm.ws.security.orbssl.WSSSLClientSocketFactoryImpl createSSLSocket ProcessDiscovery : 0 JSSL0080E: javax.net.ssl.SSLHandshakeException - The client and server could not negotiate the desired level of security.  Reason: Received fatal alert: handshake_failure javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

in the SystemOut.log of the Node Agent.

The solution was, thankfully, the same; I updated the node-level ssl.client.props : -

vi /opt/IBM/WebSphere/AppServer/profiles/AppSrv01/properties/ssl.client.props 

and adding: -

com.ibm.ssl.enabledCipherSuites=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384


Error handshake_failure seen when connecting to WebSphere Application Server using SOAP over HTTPS

I saw a bunch of SSL-related errors when attempting to use / access the WebSphere Application Server SOAP-based administration service: -

/opt/IBM/WebSphere/AppServer/profiles/Dmgr01/bin/wsadmin.sh -lang jython -user wasadmin -password passw0rd -host `hostname` -port 8879

including: -

...
WASX7023E: Error creating "SOAP" connection to host "bpm856.uk.ibm.com"; exception information: com.ibm.websphere.management.exception.ConnectorNotAvailableException: [SOAPException: faultCode=SOAP-ENV:Client; msg=Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: handshake_failure; targetException=java.lang.IllegalArgumentException: Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: handshake_failure]
com.ibm.websphere.management.exception.ConnectorNotAvailableException: [SOAPException: faultCode=SOAP-ENV:Client; msg=Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: handshake_failure; targetException=java.lang.IllegalArgumentException: Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: handshake_failure]
...
Caused by: [SOAPException: faultCode=SOAP-ENV:Client; msg=Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: handshake_failure; targetException=java.lang.IllegalArgumentException: Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: handshake_failure]

This started happening immediately after I'd locked down WAS using TLS 1.2 and Mutual Authentication.

However, all had been working UNTIL I enforced WAS to use a pair of TLS 1.2 ciphers: -


Specifically, I'm using these: -

TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384


across the "estate" including IBM HTTP Server, WAS and DB2.

This proved to be the root cause ….

I needed to update the Deployment Manager profile's SSL configuration: -

/opt/IBM/WebSphere/AppServer/profiles/Dmgr01/properties/ssl.client.props

from: -

#com.ibm.ssl.enabledCipherSuites=

to: -

com.ibm.ssl.enabledCipherSuites=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384

*NOTE* The list is separated with a space character NOT a comma; see: -

ssl.client.props client configuration file

for details.

For the record, I'd previously done this when I enforced TLS 1.2, by changing: -

com.ibm.ssl.protocol=SSL_TLS

to: -

com.ibm.ssl.protocol=TLSv1.2

Friday, 25 March 2016

IBM HTTP Server - Tinkering with SSL Client Authentication

I'm tinkering with so-called Mutual Authentication (MA) between various middleware components, from browser through to back-end WebSphere Application Server (WAS).

Here's what I'm reading: -




I'm also referencing this: -


as I'm using openSSL to create personal device certificates etc. to allow a client browser to MA to IHS: -

Generate a Private Key on my Mac

openssl genrsa -out ~/macintosh.uk.ibm.com.key 2048

Generate a Certificate Service Request

openssl req -new -sha256 -key ~/macintosh.uk.ibm.com.key -out ~/macintosh.uk.ibm.com.csr

Generate a Personal Certificate from the Certificate Service Request

certreq -submit -attrib "CertificateTemplate:Webserver" macintosh.uk.ibm.com.csr macintosh.uk.ibm.com.cer

- This on a Windows Server 2008 R2 box

Convert the Personal Certificate and Private Key and Certificate Service Request into a PKCS12 file

openssl pkcs12 -export -out macintosh.uk.ibm.com.pfx -inkey macintosh.uk.ibm.com.key -in macintosh.uk.ibm.com.cer -certfile macintosh.uk.ibm.com.csr 

- This is required to allow me to import the private key and personal certificate into Firefox

For the record, this is relevant part of my IHS configuration - httpf.conf


LoadModule ibm_ssl_module modules/mod_ibm_ssl.so
Listen 8443
<VirtualHost *:8443>
SSLProtocolDisable SSLv2 SSLv3 TLSv10 TLSv11
SSLProtocolEnable TLSv12
SSLCipherSpec ALL NONE
SSLCipherSpec TLSv12 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
SSLCipherSpec TLSv12 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
SSLClientAuth Required
SSLEnable
</VirtualHost>
KeyFile /opt/IBM/HTTPServer/ssl/keystore.kdb
SSLDisable


...

and this is the validation of same: -

/opt/IBM/HTTPServer/bin/apachectl -DDUMP_SSL_CONFIG

SSL configuration:
Default server
Server name: bpm856.uk.ibm.com:8080
SSL enabled: NO

SSL server defined at: /opt/IBM/HTTPServer/conf/httpd.conf:852
Server name: bpm856.uk.ibm.com:8443
SSL enabled: YES
FIPS enabled: 0
Keyfile: /opt/IBM/HTTPServer/ssl/keystore.kdb
Protocols enabled: TLSv12
Ciphers for SSLV2: (protocol disabled)
Ciphers for SSLV3: (protocol disabled)
Ciphers for TLSv10: (protocol disabled)
Ciphers for TLSv11: (protocol disabled)
Ciphers for TLSv12: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256(C02F),TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384(C030)

Syntax OK


and this is how I validate IHS from a client connection, using openSSL: -

openssl s_client -connect bpm856.uk.ibm.com:8443 < /dev/null

New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256


Thursday, 24 March 2016

Java EE, the next inception: A primer to WebSphere Liberty for Java EE developers

I saw this on Twitter and wanted to reshare: -


Learn the basics of how IBM® WebSphere® Application Server Liberty profile works, and how its architecture differs significantly from that of older Java™ EE application servers. This background will be helpful for understanding how to install a local Java EE application development environment using Eclipse and Liberty, as well as how to deploy Liberty applications.

Something else with which to play ….

Monday, 21 March 2016

Virtual Proof of Technology: Highlighting new Features of WebSphere Liberty for 1Q 2016 - 30 March 2016

A new edition of our Virtual Proof Of Technology (VPOT) Series for IBM® WebSphere® Liberty will again be offered in 1st Quarter. We will briefly cover the primary benefits of WebSphere Liberty, and show how it provides your path to Hybrid Cloud for Java applications. 

We will show why Liberty is an ideal runtime for new microservices applications, and then drill down into the new enhancements in Liberty V 8.5.5.8 available since December, 2015, and Liberty V8.5.5.9 planned to be available on March 18th, 2016.  Liberty's continuous delivery model provides new optionally installable features that can be added to an existing WebSphere Liberty V8.5.5 installation with no requirement for a version upgrade or migration. IBM WebSphere Liberty is included in IBM WebSphere Application Server V8.5.5 Base and Network Deployment editions.

Thursday, 10 March 2016

IBM BPM Advanced - Process Inspector, no go-go

I saw this: -


The attempt to establish a secure connection to the server has failed. See <a href="../../../BPMHelp/topic/com.ibm.wbpm.admin.doc/topics/troubleshooting_hostname_failure.html" target="_blank">SSL fails when host name configuration fails</a> for details.

when attempting to access Process Inspector, via Process Admin, in IBM BPM Advanced 8.5.5.

I also saw this: -

[09/03/16 09:17:07:483 GMT] 00000137 BPMInspectorR W   Exception during request processing:
javax.servlet.ServletException: com.ibm.processinspector.rest.ProcessAdminRestException: The attempt to establish a secure connection to the server has failed. See <a href="../../../BPMHelp/topic/com.ibm.wbpm.admin.doc/topics/troubleshooting_hostname_failure.html" target="_blank">SSL fails when host name configuration fails</a> for details.
        at com.ibm.processinspector.server.handlers.ProcessInstanceSearchHandler.handleRequest(ProcessInstanceSearchHandler.java:309)


in the AppCluster SystemOut.log.

The solution was simple: -

Update the BPM Virtual Host configuration

Start WSAdmin Client

/opt/ibm/WebSphereProfiles/PSCell1Dmgr01/bin/wsadmin.sh -lang jython

List Existing Virtual Host(s)

print AdminConfig.list('BPMVirtualHostInfo')

(cells/PSCell1/clusters/AppCluster|cluster-bpm.xml#BPMVirtualHostInfo_1424775281789)

Remove Existing Virtual Host(s)

for i in AdminConfig.list('BPMVirtualHostInfo').split():
 AdminConfig.remove(i)

Create new Virtual Host

cellID=AdminControl.getCell()
dePath='/Cell:'+ cellID+'/BPMCellConfigExtension:/BPMDeploymentEnvironment:/'
de=AdminConfig.getid(dePath)
webserver_vh = AdminConfig.create('BPMVirtualHostInfo',de,[['name','webserver_vh'],['transportProtocol', 'https'], ['hostname', 'ihs.uk.ibm.com'], ['port','8443']],'virtualHosts')
AdminConfig.modify(de,[['defaultVH',webserver_vh]])

 
Validate Virtual Host

print AdminConfig.show(webserver_vh)

[hostname ihs.uk.ibm.com]
[name webserver_vh]
[port 8443]
[transportProtocol https]


Retrieve IHS Signer Certificate into WAS Cell-Default Trust Store

AdminTask.retrieveSignerFromPort('[-keyStoreName CellDefaultTrustStore -keyStoreScope (cell):'+cellID+' -host ihs.uk.ibm.com -port 8443 -certificateAlias IHS -sslConfigScopeName (cell):'+cellID+' ]')

Save and sync

AdminConfig.save()
AdminNodeManagement.syncActiveNodes()

Quit

quit

Restart AppCluster

clusterName="AppCluster"
AdminControl.invoke('WebSphere:name='+clusterName+',process=dmgr,platform=common,node=dmgr,version=8.5.5.4,type=Cluster,mbeanIdentifier='+clusterName+',cell='+cellID+',spec=1.0', 'stop')
AdminControl.invoke('WebSphere:name='+clusterName+',process=dmgr,platform=common,node=dmgr,version=8.5.5.4,type=Cluster,mbeanIdentifier='+clusterName+',cell='+cellID+',spec=1.0', 'start')

The job, she's a good 'un

openSSL on Windows - random lack of state

I saw this exception: -

unable to write 'random state'

whilst trying to use openSSL on Windows.

I was trying to convert a CA-signed personal certificate ( and the corresponding private key ) into a PKCS12 file, in order to import the same into Firefox.

Thankfully Google drew me to this: -


which led me to a solution.

I was running the openSSL tool via a Windows command-prompt ( terminal ) session but was NOT running it as an administrative user.

Once I clicked [Start] [Run], entered cmd.exe and chose "Run as Administrator", it just worked.

I need to write up the end-to-end process for a future blog post …

Advanced database support for IBM Business Process Manager (BPM) Standard, IBM Business Process Manager Advanced, and IBM Business Monitor on distributed platforms

I had occasion to reference this earlier: -

Question

Which advanced configurations of IBM DB2, Oracle, and Microsoft SQL Server databases are supported for the IBM Business Process Manager and IBM Business Monitor products?

Answer

This document outlines which configurations of IBM DB2, Oracle, and Microsoft SQL Server databases are supported for IBM Business Process Manager and IBM Business Monitor. In this document, the following information is available:

• Database independent limitations
• IBM DB2 database support for IBM Business Process Manager and IBM Business Monitor
• IBM DB2 limitations
• Oracle database support for IBM Business Process Manager and IBM Business Monitor
• Oracle limitations
• Microsoft SQL Server database support for IBM Business Process Manager and IBM Business Monitor
• Microsoft SQL Server limitations

Tuesday, 8 March 2016

Playing with Veertu on the Mac

I heard about Veertu on a recent British Tech Network Mac podcast and have been tinkering with it on and off since.

From their website: -

• Run Windows and Linux on your Mac.
• Fast, light and secure.
• The only virtualization solution approved by Apple App Store.

The free-to-use version only allows one to create VMs by downloading the "raw" ISOs from the internet: -



Therefore, I chose to download/install CentOS Linux.

One thing that caught me out was that, having chosen a minimal installation, I found that when I logged in, I was unable to get access to the most basic commands, specifically ifconfig.

As ever, a quick Google search took me here: -


which had me use: -

yum install net-tools

to install the net-tools package, which includes ifconfig, from the public CentOS mirror.

Of course,  had my VM not had network connectivity ……

:-)

I'll continue to tinker and post back in due course ….

IBM BPM 8.5.6 - Scrabbling with Snapshots

As part of a BPM Advanced 8.5.6 build, I wanted to be able to test the creation/deployment of a Process Application snapshot from a Process Center to a Process Server.

I used this: -


as an aide memoire to remind me how to create a Process Application in Process Designer ( it's been a while ).

Having finished testing, I archived and deleted the snapshot from Process Center.

However, I also wanted to do the same for Process Server, rather than leaving my stuff lying about :-)

This is what I did: -

Connect WSAdmin client to the SOAP port of one of the Process Server AppCluster members

/opt/IBM/WebSphere/AppServer/profiles/PSDmgr01/bin/wsadmin.sh -lang jython -host bpm1.uk.ibm.com -port 46015 -user PSdeAdmin -password passw0rd

List the Process Applications

print AdminTask.BPMListProcessApplications()

Show the details of MY Process Application

print AdminTask.BPMShowProcessApplication('-containerAcronym DMH101')

Name: DaveHay
Acronym: DMH101
Description:
Toolkit: false
Tracks:

        List of Snapshots:
                Name: D22
                Acronym: D22
                Created On: 2016-03-07 21:10:33.327
                Created By: User.9
                Is Default: true
                State: State[Active]
                Capability: Capability[Standard]
                No of running instances: 0

                Name: D2
                Acronym: D2
                Created On: 2016-03-07 20:59:26.935
                Created By: User.9
                Is Default: false
                State: State[Inactive]
                Capability: Capability[Standard]
                No of running instances: 0



Delete the Inactive non-default snapshot

print AdminTask.BPMDeleteSnapshot('-containerAcronym DMH101 -containerSnapshotAcronyms D2')

BPMDeleteSnapshot passed.

Deactivate the active default snapshot

print AdminTask.BPMDeactivate('-containerAcronym DMH101  -containerSnapshotAcronym D22 -force')

Delete the now-deactivated default snapshot - using force as it's default

print AdminTask.BPMDeleteSnapshot('-containerAcronym DMH101 -containerSnapshotAcronyms D22 -force')

BPMDeleteSnapshot passed.

With thanks to this: -


Wednesday, 2 March 2016

Trumpet Blowing - The Andy and Dave Show at InterConnect 2016

My frolleague, Andy Garratt, and I co-presented at InterConnect in Vegas last week, and our materials have just been posted: -

Smarter Process Discovery and Design focuses on the people, the systems, and how to make the process Smarter; that is, the functional requirements. But this is only half the story. Neglecting non-functional Requirements (NFRs) upfront can lead to big problems later on. This session covers the key BPM NFRs, such as: Performance—why response-time does not equal throughput; compatibility and interoperability with other systems; operability—can you run it once it's live?; maintainability—can you patch and upgrade it?; migration and versioning of processes; accessibility and usability; and availability—HA and DR. Come and find out which of these NFRs need to be added to your agile backlog, so you don't have to fix them in production!

BPM talks about "project to program"—but can your BPM architecture and implementation keep up with your business needs, especially in a large enterprise? What makes it big? What influences the size of your BPM implementation? How many environments should you have... and how many do you need? How many BPM instances do you need, and what should they look like? How should you distribute your processes around your infrastructure? How can you scale BPM and what are the scaling options? What kinds of processes are there, and why does it matter? Business Process decisions can have big impacts on the technical implementation. Come and find out what these impacts are, and how to make sure your big BPM avoids big problems!


More about BPM and Scalability from InterConnect 2016 - Or, someone has a deck for that :-)





PS These links are all PDFs, and were sourced here: -