Saturday, 30 May 2015

WebSphere Application Server 8.5.5.5, TLS 1.2 and DB2

Some more blogging over at the WebSphere User Group ( aka Global WebSphere Community ), following my continuing voyage of discovery in the world of Transport Layer Security (TLS): -


...
I've been working through the configuration of Transport Layer Security (TLS) 1.2 between DB2 and WebSphere Application Server (WAS).

I've learned a heck of a lot about this in the past 48 hours, but the key aspect is that it's necessary to configure BOTH DB2 *AND* WAS to support TLS 1.2
....

Things that make you go "Hmmm" - #432 - WebSphere Application Server Transaction and Partner Logs

Over the past few weeks, I've written about my experiences configuring IBM Business Process Manager and IBM Business Monitor to connect via a TLS-encrypted tunnel to IBM DB2: -


and am just about to create a post covering the experiences learned whilst configuring WebSphere Application Server to support the current latest Transport Layer Security (TLS) 1.2.

However, I hit a small glitch....

Whilst validating my current setup ( IBM Business Monitor 8.5.6 on WAS ND 8.5.5.5 connecting via TLS 1.0 to DB2 10.5.0.5 ), I noted the following exception in one of my cluster member logs ( specifically the AppTarget ): -

[30/05/15 06:54:23:906 BST] 00000065 RecoveryManag I   WTRN0135I: Transaction service recovering no transactions.
[30/05/15 06:54:23:917 BST] 00000065 RecoveryManag A   WTRN0134I: Recovering 1 XA resource manager(s) from the transaction partner logs
[30/05/15 06:54:23:954 BST] 00000065 XARecoveryDat A   WTRN0151I: Preparing to call xa recover on XAResource: Monitor_Database
[30/05/15 06:54:24:891 BST] 00000065 DMAdapter     I com.ibm.ws.ffdc.impl.DMAdapter getAnalysisEngine FFDC1009I: Analysis Engine using data base: /opt/IBM/WebSphere/AppServer/properties/logbr/ffdc/adv/ffdcdb.xml
[30/05/15 06:54:24:897 BST] 00000065 FfdcProvider  W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/IBM/WebSphere/AppServer/profiles/BAMCell1AppSrv01/logs/ffdc/AppClusterMember1_c432d1b_15.05.30_06.54.24.8794642297944511548081.txt com.ibm.ws.rsadapter.spi.InternalGenericDataStoreHelper.getPooledCon 1298
[30/05/15 06:54:24:939 BST] 00000065 FfdcProvider  W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/IBM/WebSphere/AppServer/profiles/BAMCell1AppSrv01/logs/ffdc/AppClusterMember1_c432d1b_15.05.30_06.54.24.9204315923791292855617.txt com.ibm.ejs.j2c.J2CXAResourceFactory.getXAResource 310
[30/05/15 06:54:24:942 BST] 00000065 J2CXAResource W   J2CA0061W: Error creating XA Connection and Resource com.ibm.ws.exception.WsException: DSRA8100E: Unable to get a XAConnection from the DataSource jdbc/wbm/MonitorDatabase. with SQL State : 08001 SQL Code : -4499


Caused by: com.ibm.websphere.ce.cm.StaleConnectionException: [jcc][t4][2043][11550][4.18.60] Exception java.net.ConnectException: Error opening socket to server bam856.uk.ibm.com/127.0.0.1 on port 60,006 with message: Connection refused. ERRORCODE=-4499, SQLSTATE=08001 DSRA0010E: SQL State = 08001, Error Code = -4,499


java.sql.SQLNonTransientException: [jcc][t4][2043][11550][4.18.60] Exception java.net.ConnectException: Error opening socket to server bam856.uk.ibm.com/127.0.0.1 on port 60,006 with message: Connection refused. ERRORCODE=-4499, SQLSTATE=08001 DSRA0010E: SQL State = 08001, Error Code = -4,499

Caused by: java.net.ConnectException: Connection refused

Having gone through the configuration with a fine tooth comb ( whatever one of those is ), I could NOT find ANY reference to port  60006 anywhere.

For the record, port 60006 is the non-TLS port that I'd previously used, before switching to port 60007 for a TLS-encrypted connection.

After much trial and quite a lot of error, I re-read the log, specifically these two lines: -

[30/05/15 06:54:23:917 BST] 00000065 RecoveryManag A   WTRN0134I: Recovering 1 XA resource manager(s) from the transaction partner logs
[30/05/15 06:54:23:954 BST] 00000065 XARecoveryDat A   WTRN0151I: Preparing to call xa recover on XAResource: Monitor_Database


which started me thinking about the Transaction Manager.

What, I wondered, was the possibility that the OLD pre-TLS configuration was still persisted in a transaction that had previously NOT completed before I switched the configuration across ?

I did some further digging ( using the command fgrep -R 60006 * ) inside the directory that hosts the Transaction, Recovery and Partner logs for the AppCluster: -

cd /opt/IBM/WebSphere/AppServer/profiles/BAMCell1AppSrv01/tranlog/BAMCell1/AppSrv01Node/AppClusterMember1/transaction

and found two binary files: -

log1
log2

here: -

/opt/IBM/WebSphere/AppServer/profiles/BAMCell1AppSrv01/tranlog/BAMCell1/AppSrv01Node/AppClusterMember1/transaction/partner

both of which contained references to the string 60006.

That confirmed my suspicion.

Now there's a third file in this directory, sensibly named: -

DO NOT DELETE LOG FILES

That's there for a VERY good reason - one should NEVER delete the Transaction or Partner Log files.

*** WARNING - CAVEAT EMPTOR ***

Having said NEVER, this is MY own test environment with NO important or critical data - if I break things, I simply rebuild the WAS cell, which takes ~30 minutes.

So, ignoring my own ( and IBM's ) advice, I delete the Partner Logs: -

cd /opt/IBM/WebSphere/AppServer/profiles/BAMCell1AppSrv01/tranlog/BAMCell1/AppSrv01Node/AppClusterMember1/transaction/partner
rm -Rf *

having shut down the AppCluster.

Quelle surprise, when I restarted the cluster, there were no failed transactions to recover, and WAS came up clean and green with NO JDBC exceptions.

*** WARNING - CAVEAT EMPTOR ***

Thinking about it after the event, I probably could have achieved the same thing by re-opening port 60006 on DB2, which I'd previously disabled using the db2set command as follows: -

db2set DB2COMM=SSL

thus overriding the previous configuration: -

db2set DB2COMM=SSL,TCPIP

For the record, the SSL value means that DB2 observes the SSL service configuration within the Database Manager: -

SSL service name                         (SSL_SVCENAME) = db2c_ssl

whereas TCPIP means that it observes the TCPIP service configuration: -

TCP/IP Service name                          (SVCENAME) = db2c_db2inst1

In each case, the Service Name is inferred from /etc/services which ensures that the instance is listening on the appropriate ports: -

DB2_db2inst1 60000/tcp
DB2_db2inst1_1 60001/tcp
DB2_db2inst1_2 60002/tcp
DB2_db2inst1_3 60003/tcp
DB2_db2inst1_4 60004/tcp
DB2_db2inst1_END 60005/tcp
db2c_db2inst1 60006/tcp
db2c_ssl 60007/tcp


So, had I enabled BOTH services, WAS would've been able to connect via a non-TLS connection to port 60006 and the transaction would have been recovered / completed.

Life is, as ever, a learning curve :-)

For future reference, there's plenty of good material covering the WAS Java Transaction Service, including this: -


which does cover the costs and benefits of deleting the Transaction and Partner Logs, especially in the context of WebSphere Process Server ( now IBM BPM ).

So, again, do NOT NOT NOT delete Tran/Partner Logs unless you really really really know what you're doing.

Friday, 29 May 2015

IBM HTTP Server, Transport Layer Security and Google Chrome

In another of my occasional posts for the Global WebSphere Community (GWC), aka the WebSphere User Group, I've just submitted an article covering my experiences with IBM HTTP Server, SSL/TLS and Chrome.

It's on the GWC site here: -


I hope it's of some use :-)

Friday, 15 May 2015

Business Process Management Design Guide: Using IBM Business Process Manager

IBM® Business Process Manager (IBM BPM) is a comprehensive business process management (BPM) suite that provides visibility and management of your business processes. IBM BPM supports the whole BPM lifecycle approach: 

• Discover and document
• Plan
• Implement
• Deploy
• Manage
• Optimize
Process owners and business owners can use this solution to engage directly in the improvement of their business processes.

IBM BPM excels in integrating role-based process design, and provides a social BPM experience. It enables asset sharing and creating versions through its Process Center. The Process Center acts as a unified repository, making it possible to manage changes to the business processes with confidence.

IBM BPM supports a wide range of standards for process modeling and exchange. Built-in analytics and search capabilities help to further improve and optimize the business processes.

This IBM Redbooks® publication provides valuable information for project teams and business people that are involved in projects using IBM BPM. It describes the important design decisions that you face as a team. These decisions invariably have an effect on the success of your project.

These decisions range from the more business-centric decisions, such as which should be your first process, to the more technical decisions, such as solution analysis and architectural considerations.

Table of contents

Chapter 1. Introduction to successful business process management
Chapter 2. Approaches and process discovery
Chapter 3. Solution analysis and architecture considerations
Chapter 4. Security architecture considerations
Chapter 5. Design considerations and patterns
Chapter 6. Business-centric visibility
Chapter 7. Performance and IT-centric visibility

Thursday, 14 May 2015

Continuing to learn - IBM BPM and IBM Business Monitor to DB2 via SSL/TLS

I've written a couple of posts on the WebSphere User Group blog here: -



My next trick will be to force WebSphere Application Server (WAS) to use a specific encryption standard, namely TLS version 1.2.

In DB2, this can be enforced as follows: -

db2 update dbm config using SSL_VERSIONS TLSV12

for version 1.2 or: -

db2 update dbm config using SSL_VERSIONS TLSV1

for version 1.0, or: -

db2 update dbm config using SSL_VERSIONS NULL

to revert back to SSL.

...
If you set the parameter to null or TLSv1, the parameter enables support for TLS version 1.0 (RFC2246) and TLS version 1.1 (RFC4346).

Note: During SSL handshake, the client and the server negotiate and find the most secure version to use either TLS version 1.0 or TLS version 1.1. If there is no compatible version between the client and the server, the connection fails. If the client supports TLS version 1.0 and TLS version 1.1, but the server support TLS version 1.0 only, then TLS version 1.0 is used.
If you set the parameter to TLSv12 (RFC5246), the parameter enables support for TLS version 1.2. This setting is required to comply with NIST SP 800-131A.

If you set the parameter to TLSv12 and TLSv1, the parameter enables support for TLS version 1.2 with the option to fall back on TLS version 1.0 and 1.1.
...

All of the SSL-related settings can be queried thusly: -

db2 get dbm config | grep SSL

 SSL server keydb file                   (SSL_SVR_KEYDB) = /home/db2inst1/keystore.kdb
 SSL server stash file                   (SSL_SVR_STASH) = /home/db2inst1/keystore.sth
 SSL server certificate label            (SSL_SVR_LABEL) = bpm856.uk.ibm.com
 SSL service name                         (SSL_SVCENAME) = db2c_ssl
 SSL cipher specs                      (SSL_CIPHERSPECS) = 
 SSL versions                             (SSL_VERSIONS) = 
 SSL client keydb file                  (SSL_CLNT_KEYDB) = 
 SSL client stash file                  (SSL_CLNT_STASH) = 

Note that we also have SSL_CIPHERSPECS to specify the cipher specifications that one wishes to use, as per this: -


and: -

Wednesday, 13 May 2015

IBM Cognos - Working with SSL/TLS Keystore

This is an ongoing voyage of discovery, as I seek to replicate my success: -


( this is a post that I authored for the WebSphere User Group on their Global WebSphere Community site )

with IBM Business Monitor.

Whilst the DB2 and WAS aspects ( configuring the DB2 instance and listener for SSL, updating the WAS JDBC data sources, adding the DB2 signer certificate into he WAS trust store etc. ) are the same, the Cognos BI engine is quite different.

I don't yet have it cracked, but I did discover a few more things about Cognos BI today, specifically in terms of where it keeps its own SSL/TLS key store.

It's here: -

-rw-r--r-- 1 wasadmin wasadmins 19728 May 13 16:07 /opt/IBM/WebSphere/AppServer/profiles/BAMCell1AppSrv01/cognos/SupClusterMember1/configuration/certs/CAMKeystore

Why do I know this ?

Because I wanted to test a hypothesis by adding the DB2 server's signer certificate to it.

This is how I first retrieved the signer certificate: -

openssl s_client -showcerts -connect localhost:60007 </dev/null > ~/db2.cer

and I happily verified the certificate: -

openssl x509 -fingerprint -noout -text -in ~/db2.cer

SHA1 Fingerprint=FC:BB:C1:24:4E:6E:B8:55:5B:33:87:69:C7:E2:10:E4:E6:0F:7A:CC
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1681898445175821098 (0x17574db98cfc932a)
    Signature Algorithm: sha1WithRSAEncryption
        Issuer: DC=com, DC=ibm, DC=uk, CN=bpm856.uk.ibm.com
        Validity
            Not Before: May 11 10:01:51 2015 GMT
            Not After : May 11 10:01:51 2016 GMT
        Subject: DC=com, DC=ibm, DC=uk, CN=bpm856.uk.ibm.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (1024 bit)
                Modulus:
                    00:cc:de:34:78:ca:b8:48:c1:24:43:3b:39:ca:79:
                    6e:7d:bd:2f:fd:a5:86:cc:fa:d1:0f:9f:6b:d2:04:
                    ac:5f:3e:4f:42:81:89:03:88:fb:95:86:ed:fd:f4:
                    c5:a1:c0:8e:b4:70:b7:2d:36:c8:2e:1a:5c:d7:b5:
                    83:e0:f4:36:f8:0a:8f:32:54:47:1a:b7:a4:b6:42:
                    d8:4c:60:ee:e5:2c:de:a2:77:ee:10:b0:fc:c3:a2:
                    7a:e2:3b:45:c4:2f:8a:11:43:bc:fb:a2:e1:cd:69:
                    0f:aa:bb:e2:7c:de:2b:8b:3c:76:cd:56:a8:5d:3e:
                    5c:e7:fb:ef:b1:15:f9:14:41
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints: critical
                CA:TRUE
            X509v3 Subject Key Identifier: 
                56:25:C5:62:51:0C:60:55:4D:61:9D:71:EF:D4:A4:E9:AA:07:24:85
            X509v3 Authority Key Identifier: 
                keyid:56:25:C5:62:51:0C:60:55:4D:61:9D:71:EF:D4:A4:E9:AA:07:24:85

    Signature Algorithm: sha1WithRSAEncryption
         99:4c:9c:6e:65:a9:d1:c8:b1:d7:44:30:cd:9a:bc:d5:77:a0:
         9f:69:8b:97:2e:e7:13:95:97:b2:b4:57:d0:74:14:e3:e3:ea:
         ae:22:ef:01:2c:2e:b7:37:1a:85:e7:00:48:41:71:9b:25:a4:
         25:79:76:04:6d:3c:a5:a3:ce:9c:e2:ea:26:33:56:6d:2e:40:
         1f:0e:bf:e8:b7:de:06:1b:d1:8c:65:c4:19:8c:c8:39:92:d8:
         f5:ad:18:56:c3:ef:d6:25:a1:4c:a9:64:40:df:df:75:a0:5e:
         ec:7e:ea:cc:8e:dc:2c:1e:71:4a:8d:74:7f:d6:84:8a:20:05:
         fb:64

However, when I tried to add it to the Cognos key store: -

/opt/IBM/WebSphere/AppServer/java/jre/bin/keytool -import -file ~/db2.cer -alias DB2 -keystore CAMKeystore -storepass MONITOR -storetype PKCS12

I saw this: -

keytool error: java.lang.Exception: Input not an X.509 certificate

Happily a quick Google search later, and I found this: -


which says, in part: -

<snip>
While I agree with Ari's answer (and upvoted it :), I needed to do an extra step to get it to work with Java on Windows (where it needed to be deployed):

openssl s_client -showcerts -connect www.example.com:443 < /dev/null | openssl x509 -outform DER > derp.der

Before adding the openssl x509 -outform DER conversion, I was getting an error from keytool on Windows complaining about the certificate's format. Importing the .der file worked fine.
</snip>

I re-retrieved the certificate from DB2: -

openssl s_client -showcerts -connect localhost:60007 </dev/null | openssl x509 -outform DER > ~/db2.cer

( adding in the relevant Hogwarts magic to get the resulting file in x509 DER ) and was then able to import it: -

/opt/IBM/WebSphere/AppServer/java/jre/bin/keytool -import -file ~/db2.cer -alias DB2 -keystore CAMKeystore -storepass MONITOR -storetype PKCS12

Owner: CN=bpm856.uk.ibm.com, DC=uk, DC=ibm, DC=com
Issuer: CN=bpm856.uk.ibm.com, DC=uk, DC=ibm, DC=com
Serial number: 17574db98cfc932a
Valid from: 11/05/15 11:01 until: 11/05/16 11:01
Certificate fingerprints:
 MD5:  81:B0:E7:81:A3:1B:79:64:07:1B:41:9E:7E:0A:F3:08
 SHA1: FC:BB:C1:24:4E:6E:B8:55:5B:33:87:69:C7:E2:10:E4:E6:0F:7A:CC
Trust this certificate? [no]:  y
Certificate was added to keystore

which is nice.

Did that fix my problem ? Alas, no, but it's another step on the journey to ......... ?

Tuesday, 12 May 2015

IBM Integration Designer - 101

I'm learning to get to grips with IBM Integration Designer (IID) at present, hence my post on the WUG here: -


Part of my self-enablement has come from a pair of tutorials, cannily called Hello World, that I found a few days ago: -


I was using older ( version 7.5.1 ) editions of these two tutorials, although they worked perfectly ( user errors notwithstanding !! ).

However, here's the more up-to-date source for the most recent versions of IBM BPM: -



Interestingly, Hello World seems to have disappeared with the latest version of the product.

In addition, BPM 8.5.6 also has some tutorials in the Knowledge Centre here: -


which is nice.