Wednesday, 3 June 2015

Securing the Database - IBM DB2 10.5 and Transport Layer Security 1.2

This builds upon a series of earlier posts, including: -

This week, my colleague, JohnR, and I have been endeavouring to understand more about the way that one can use Transport Layer Security (TLS) 1.2 in the context of IBM DB2, with specific regard to Java client connectivity.

To that end, I've set up my DB2 server to only accept incoming connections on a specific port ( 60007 ) via TLS 1.2 using a very specific cipher specification.

This is what I have: -

db2 get dbm config | grep SSL

 SSL server keydb file                   (SSL_SVR_KEYDB) = /home/db2inst1/keystore.kdb
 SSL server stash file                   (SSL_SVR_STASH) = /home/db2inst1/keystore.sth
 SSL server certificate label            (SSL_SVR_LABEL) =
 SSL service name                         (SSL_SVCENAME) = db2c_ssl
 SSL cipher specs                      (SSL_CIPHERSPECS) = TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
 SSL versions                             (SSL_VERSIONS) = TLSV12
 SSL client keydb file                  (SSL_CLNT_KEYDB) = 
 SSL client stash file                  (SSL_CLNT_STASH) = 

cat /etc/services

DB2_db2inst1 60000/tcp
DB2_db2inst1_1 60001/tcp
DB2_db2inst1_2 60002/tcp
DB2_db2inst1_3 60003/tcp
DB2_db2inst1_4 60004/tcp
DB2_db2inst1_END 60005/tcp
db2c_db2inst1 60006/tcp
db2c_ssl 60007/tcp

( I've highlighted the most specific aspects above )

This essentially means that DB2 will: -

(a) Listen on port 60007
(b) Only accept incoming connections that use TLS 1.2
(c) Only accept incoming connections that support the TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 cipher specification
(d) Present a signer certificate with the alias
(e) Use a local keystore - /home/db2inst1/keystore.kdb  - to store the signer certificate
(f) Use a local stashed password file - /home/db2inst1/keystore.sth

Having set all of this up, we were somewhat surprised to find that a standard Java class wouldn't connect, instead returning handshake_failure exceptions such: - [jcc][t4][2030][11211][3.69.24] A communication error occurred during operations on the connection's underlying socket, socket input stream, 
or socket output stream.  Error location: Reply.fill() - (-1).  Message: Received fatal alert: handshake_failure. ERRORCODE=-4499, SQLSTATE=08001

This proved to be for a number of reasons.

These are three of them: -

(1) It's important to use the "right" Java Runtime Environment, as TLS 1.2 support was added relatively recently - we experimented with various versions of Java 7 and Java 8, with varying degrees of success
(2) The AES 256 ciphers require the JRE to be augmented with Unrestricted SDK JCE policy files - this is definitely true for the IBM JRE, and may also be true for Oracle
(3) Not all ciphers work with all JREs - John and I had varying results

In addition, DB2 has a quirk in that the Cipher Specification(s) that are configured in the instance Database Manager Configuration are prefixed with tls_ whereas Java appears to want to prefix them with ssl_ 

As an example, here's an excerpt from the IBM SDK 7.1: -

Default enabled cipher suites in order of preference:


whereas DB2 wants the suite specified as: -


as per this: -

When I tried to persuade DB2 to use the same consistent naming convention as Java, this is what I saw: -

db2 update dbm config using SSL_CIPHERSPECS SSL_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384

SQL6112N  The configuration parameter was not updated because the resulting 
configuration parameter settings would not be valid.  Reason code "8".

So, in the context of working JREs, I did note that the Java 7 JRE that's included with DB2 -

/opt/ibm/db2/V10.5/java/jdk64/jre/bin/java -version

java version "1.7.0"
Java(TM) SE Runtime Environment (build pxa6470sr6-20131015_01(SR6))
IBM J9 VM (build 2.6, JRE 1.7.0 Linux amd64-64 Compressed References 20131013_170512 (JIT enabled, AOT enabled)
J9VM - R26_Java726_SR6_20131013_1510_B170512
JIT  - r11.b05_20131003_47443
GC   - R26_Java726_SR6_20131013_1510_B170512_CMPRSS
J9CL - 20131013_170512)
JCL - 20131011_01 based on Oracle 7u45-b18

returns the handshake_exception whereas that shipped with WebSphere Application Server -

/opt/IBM/WebSphere/AppServer/java/jre/bin/java -version

java version "1.6.0"
Java(TM) SE Runtime Environment (build pxa6460_26sr8fp3-20141218_02(SR8 FP3))
IBM J9 VM (build 2.6, JRE 1.6.0 Linux amd64-64 Compressed References 20141211_226933 (JIT enabled, AOT enabled)
J9VM - R26_Java626_SR8_20141211_2359_B226933
JIT  - r11.b07_20141003_74578.05
GC   - R26_Java626_SR8_20141211_2359_B226933_CMPRSS
J9CL - 20141212_226933)
JCL  - 20141216_01

works perfectly.

Therefore, it's important to ensure that one uses the correct JRE and that one fully tests each required Cipher Suite.

Finally, I mentioned the need to augment the JRE with Unrestricted SDK JCE policy files - this is most clearly documented here: -

** Cipher suites that use AES_256 require installation of the   JCE Unlimited Strength Jurisdiction Policy Files.


As far as I'm aware, the use of the Unrestricted SDK JCE policy files is definitely something that needs to be considered on a case by case basis, as there are license agreement considerations, as described in this example: -

which then links to this: -


In my own case, having downloaded the policy files, this gave me a ZIP file: -

-rw-r--r-- 1 db2inst1 db2iadm1 4.0K Jun  2 21:06

This contains two files: -

-r--r--r--  1 db2inst1 db2iadm1 2253 Oct 12  2012 local_policy.jar
-r--r--r--  1 db2inst1 db2iadm1 2240 Oct 12  2012 US_export_policy.jar

which I placed here: -


having backed up and moved the original versions: -

-rwxr-xr-x.  1 wasadmin wasadmins  3890 Feb 19 17:29 blacklist
-rwxr-xr-x.  1 wasadmin wasadmins 77924 Jun  3 14:49 cacerts
-rwxr-xr-x.  1 wasadmin wasadmins  2532 Apr 16  2012 java.policy
-rwxr-xr-x.  1 wasadmin wasadmins 10560 Apr 16  2012
-rwxr-xr-x.  1 wasadmin wasadmins    98 Feb 19 17:29 javaws.policy
-r--r--r--   1 root     root       2253 Jun  3 15:18 local_policy.jar
-rwxr-xr-x.  1 wasadmin wasadmins  2640 Feb 19 17:29 local_policy.RAJ
-rwxr-xr-x.  1 wasadmin wasadmins     0 Feb 19 17:29 trusted.libraries
-r--r--r--   1 root     root       2240 Jun  3 15:18 US_export_policy.jar
-rwxr-xr-x.  1 wasadmin wasadmins  2175 Feb 19 17:29 US_export_policy.RAJ

( I have highlighted the original files which I moved to a .RAJ extension and also the new files )

With all of the above in place: -

(a) The right version of Java 7 ( in my case ) to support TLS 1.2
(b) The right TLS cipher specification(s)
(c) The unrestricted policy files - if using AES256 ciphers

This is, of course, over and above the configuration required both server-side ( DB2 ) and client-side ( Java ), in terms of hosting key stores, signer certificates etc.

This is how I extracted the signer certificate from DB2 and stored in a Java KeyStore (JKS) file for use by my Java code: -

openssl s_client -showcerts -connect localhost:60007 </dev/null | openssl x509 -outform DER > ~/db2.cer

depth=0 DC = com, DC = ibm, DC = uk, CN =
verify error:num=18:self signed certificate
verify return:1
depth=0 DC = com, DC = ibm, DC = uk, CN =
verify return:1

/opt/IBM/WebSphere/AppServer/java/jre/bin/keytool -import -file ~/db2.cer -keystore /tmp/davehay.jks -alias DB22 -storepass davehay

Owner:, DC=uk, DC=ibm, DC=com
Issuer:, DC=uk, DC=ibm, DC=com
Serial number: 686dcce6267d5fb4
Valid from: 28/05/15 13:54 until: 28/05/16 13:54
Certificate fingerprints:
 MD5:  55:22:9D:A3:F8:60:EA:E6:2C:4F:C9:74:59:16:7B:22
 SHA1: B9:07:FB:AC:0C:77:18:4D:B9:52:CD:71:5E:00:DB:93:F4:A9:FA:6A
Trust this certificate? [no]:  y
Certificate was added to keystore

as validated below: -

keytool -list -keystore /tmp/davehay.jks -storepass davehay

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 1 entry

db2, 03-Jun-2015, trustedCertEntry, 
Certificate fingerprint (SHA1): B9:07:FB:AC:0C:77:18:4D:B9:52:CD:71:5E:00:DB:93:F4:A9:FA:6A

For the record, here's the Java class: -

import java.sql.Connection ;
import java.sql.DriverManager ;
import java.sql.ResultSet ;
import java.sql.Statement ;
import java.sql.SQLException;

import org.omg.CORBA.VersionSpecHelper;

class JdbcTestDB2
public static void main (String args[])
catch (ClassNotFoundException e)
System.err.println (e) ;
System.exit (-1) ;
String hostname = "";
int port = 60007;
String dbName = "SAMPLE";
String userName = "db2inst1";
String password = "passw0rd";
String sslConnection = "true";

java.util.Properties properties = new java.util.Properties();
properties.put("password", password);
properties.put("sslConnection", sslConnection);

String url = "jdbc:db2://" + hostname + ":" + port + "/" + dbName;
Connection connection = DriverManager.getConnection(url,properties);

String query = "select EMPNO,FIRSTNME,LASTNAME from DB2INST1.EMPLOYEE" ;

Statement statement = connection.createStatement () ;
ResultSet rs = statement.executeQuery (query) ;

while ( () )
System.out.println (rs.getString (1) + " " + rs.getString(2) + " " + rs.getString(3)) ;
connection.close () ;
catch (java.sql.SQLException e)
System.err.println (e) ;
System.exit (-1) ;

Note that I've highlighted the variables that I'm then setting for the JDBC connection using a java.util.Properties object.

Finally, during the problem determination phase, I used the JVM command to get more information about the handshake_failure : -

java  JdbcTestDB2

*** ClientHello, TLSv1
RandomCookie:  GMT: 1433356707 bytes = { 139, 106, 65, 168, 154, 169, 66, 60, 183, 34, 1, 227, 142, 239, 237, 139, 58, 162, 251, 68, 84, 56, 204, 216, 182, 194, 237, 101 }
Session ID:  {}
Compression Methods:  { 0 }
main, WRITE: TLSv1 Handshake, length = 67
main, READ: TLSv1.2 Alert, length = 2
main, RECV TLSv1 ALERT:  fatal, handshake_failure
main, called closeSocket()
main, handling exception: Received fatal alert: handshake_failure
main, called close()
main, called closeInternal(true)
main, called close()
main, called closeInternal(true)
main, called close()
main, called closeInternal(true) [jcc][t4][2030][11211][3.69.24] A communication error occurred during operations on the connection's underlying socket, socket input stream, 
or socket output stream.  Error location: Reply.fill() - (-1).  Message: Received fatal alert: handshake_failure. ERRORCODE=-4499, SQLSTATE=08001


Note that, whilst the ClientHello phase starts, we never see the resulting ServerHello response.

This is with the wrong JRE: -

ls -al `which java`

lrwxrwxrwx. 1 root root 22 Apr  5 21:52 /usr/bin/java -> /etc/alternatives/java

java -version

java version "1.7.0_65"
OpenJDK Runtime Environment (rhel- u65-b17)
OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode)

whereas this is what I see with the right JRE: -

*** ClientHello, TLSv1.2
RandomCookie:  GMT: 1433356928 bytes = { 209, 47, 22, 227, 221, 42, 210, 36, 159, 234, 33, 130, 46, 110, 132, 83, 32, 121, 46, 38, 107, 8, 238, 212, 19, 125, 148, 178 }
Session ID:  {}
Compression Methods:  { 0 }
Extension elliptic_curves, curve names: {secp256r1, secp192r1, secp224r1, secp384r1, secp521r1, secp160k1, secp160r1, secp160r2, secp192k1, secp224k1, secp256k1}
Extension ec_point_formats, formats: [uncompressed]
Extension signature_algorithms, signature_algorithms: SHA512withECDSA, SHA512withRSA, SHA384withECDSA, SHA384withRSA, SHA256withECDSA, SHA256withRSA, SHA224withECDSA, SHA224withRSA, SHA1withECDSA, SHA1withRSA, SHA256withDSA, SHA1withDSA, MD5withRSA
main, WRITE: TLSv1.2 Handshake, length = 195
main, READ: TLSv1.2 Handshake, length = 1375
*** ServerHello, TLSv1.2
RandomCookie:  GMT: -128 bytes = { 238, 184, 253, 53, 112, 242, 137, 166, 205, 83, 9, 182, 17, 177, 233, 43, 206, 14, 0, 217, 246, 26, 214, 153, 47, 150, 202, 51 }
Session ID:  {105, 109, 0, 0, 212, 197, 167, 14, 199, 117, 87, 153, 13, 215, 101, 219, 250, 202, 212, 98, 88, 88, 88, 88, 128, 75, 111, 85, 0, 0, 33, 144}
Compression Method: 0
Extension ec_point_formats, formats: [uncompressed]
Extension renegotiation_info, ri_length: 0, ri_connection_data: { null }
JsseJCE:  Using MessageDigest SHA-384 from provider IBMJCE version 1.7
%% Initialized:  [Session-1, SSL_ECDHE_RSA_WITH_AES_256_CBC_SHA384]


Again, the ClientHello phase lists a huge number of ciphers being presented from Java to DB2, and the ServerHello shows the single cipher that DB2 is presenting back.

Right, that's it for now :-) 


Prashant More said...

HI Dave,
I followed same steps given by u to configure TLS 1.2 on both DB2 and WAS still i m facing following handshake error,
reverifed all the checklist mentioned in blog
i would appriciate if u could help me in this,
for db2 tls 1.2 configuration i refered to
for was 8.5 TLS 1.2 config i refered to
under ssl debug logs, ****i hv shortened log due to size limitation. Please provide ur mail ID i ll send full logs.
adding as trusted cert:
[8/18/17 12:14:00:114 IST] 00000001 SystemOut O Subject: CN=db.test.lfin.internal, OU=XYX, O=XYX, L=Norcross, ST=Georgia, C=US
[8/18/17 12:14:00:114 IST] 00000001 SystemOut O Issuer: CN=db.test.lfin.internal, OU=XYX, O=XYX, L=Norcross, ST=Georgia, C=US
Is initial handshake: true
[8/18/17 12:14:00:990 IST] 00000001 SystemOut O %% No cached client session
[8/18/17 12:14:00:992 IST] 00000001 SystemOut O *** ClientHello, TLSv1.2
[8/18/17 12:14:00:992 IST] 00000001 SystemOut O RandomCookie: GMT: 1486195632 bytes = { 124, 13, 239, 142, 65, 163, 203, 240, 31, 107, 71, 73, 220, 166, 43, 183, 226, 95, 220, 148, 96, 232, 217, 21, 104, 5, 34, 117 }
[8/18/17 12:14:00:993 IST] 00000001 SystemOut O Session ID: {}
[8/18/17 12:14:00:993 IST] 00000001 SystemOut O Cipher Suites: [List for all ciphers ]
[8/18/17 12:14:00:993 IST] 00000001 SystemOut O Compression Methods: { 0 }
[8/18/17 12:14:00:999 IST] 00000001 SystemOut O Extension elliptic_curves, curve names: {secp256r1, secp192r1, secp224r1, secp384r1, secp521r1, secp160k1, secp160r1, secp160r2, secp192k1, secp224k1, secp256k1}
[8/18/17 12:14:00:999 IST] 00000001 SystemOut O Extension renegotiation_info, ri_length: 0, ri_connection_data: { null }
[8/18/17 12:14:00:999 IST] 00000001 SystemOut O ***
[8/18/17 12:14:00:999 IST] 00000001 SystemOut O [write] MD5 and SHA1 hashes: len = 206
encrypted msg
[8/18/17 12:14:00:999 IST] 00000001 SystemOut O P=639808:O=0:CT, WRITE: TLSv1.2 Handshake, length = 206
[8/18/17 12:14:01:001 IST] 00000001 SystemOut O P=639808:O=0:CT, READ: TLSv1.2 Alert, length = 2
[8/18/17 12:14:01:001 IST] 00000001 SystemOut O P=639808:O=0:CT, RECV TLSv1 ALERT: fatal, handshake_failure
P=639808:O=0:CT, called closeSocket()
[8/18/17 12:14:01:002 IST] 00000001 SystemOut O P=639808:O=0:CT, handling exception: Received fatal alert: handshake_failure
Caused by: [jcc][t4][2030][11211][3.52.95] A communication error occurred during operations on the connection's underlying socket, socket input stream,
or socket output stream. Error location: T4Agent.sendRequest(). Message: Received fatal alert: handshake_failure. ERRORCODE=-4499, SQLSTATE=08001
at java.sql.DriverManager.getConnection(
at java.sql.DriverManager.getConnection(
at security.realm.ConnectionFactory.newInstance(
at security.realm.pool.Pool.getInstance(
at cecurity.realm.ConnectionPool.getConnection(
at security.realm.Query.retryableExecute(
at security.realm.Query.execute(
at security.realm.SecurityRealmDelegate.getUserByName(
at security.realm.SecurityRealmDelegate.authenticate(
at security.realm.SecurityRealmImpl.authenticate(
... 75 more

Dave Hay said...

Hi Prashant

One thing that often caught us out was forgetting to restart the DB2 instance, having enabled TLS 1.2.

Once the instance has been restarted, use a client tool such as openSSL to test the endpoint e.g.

openssl s_client -connect

and/or use this useful Java class - TestSSLServer -

Once you're completely confident that DB2 is presenting the right protocol/cipher, then move onto WAS.

One other common thing is to forget to use the appropriate Java security policies, especially if you've just switched to Java 7 or upgraded your JRE.

You can check whether the appropriate strong ciphers are supported, using the CipherTest class that I mentioned in my blog here -

If needed, please raise a PMR with IBM Support, as that's the best channel for formal support.

Cheers, Dave

Prashant More said...

Hi Dave,

Thanks for quick reply,

i hv tested with WAS data-source test connection which is successful after configuring all TLS configuration on both WAS and DB2 level.

Also CipherTest is passed .

output of open ssl command
openssl s_client -connect localhost:60006

Loading 'screen' into random state - done
13796:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:./ssl/s23_clnt.c:585:.

Still having same issue.

Dave Hay said...

Hi Prashant

Thanks for the feedback.

I'm somewhat confused, because you say that the JDBC data source connection test works OK.

Therefore, what, specifically, is the problem ?

Cheers, Dave

Prashant More said...

Hi Dave,

When I click on Test JDBC Connection through WAS admin on TLS 1.2 port(60006) it shows successful connection.

in our project we configure data source in WAS admin console,This datasource is used at various places and our project is deployed over WAS , but there are some startup classes which uses DriverManager.getConnection(URL,propes) method to get connection to DB2.

So when i start application through eclipse ->WAS server it throws above handshake error in those startup classes where DriverManager.getConnection is used.

Following is connection factory code ,

Connection conn = null;
try {


Properties props = new Properties();
props.setProperty("user", user);
props.setProperty("password", password);
System.err.println("user :"+user +"\t pwd :"+password);
System.err.println("security.realm.ConnectionFactory.newInstance 10awwww.........");

if ((isSSLEnabled != null) && ("true".equalsIgnoreCase(isSSLEnabled))) {
props.setProperty("sslConnection", "true");
//C:\WAS855DEV\installationDir\java_1.7_64\jre\lib\security\cacerts;changeit; C:/db2SSL/keystore;passw0rd

//System.setProperty("jdk.tls.client.protocols", "TLSv1.2");
/*Security.setProperty("ssl.SocketFactory.provider", "");
Security.setProperty("ssl.ServerSocketFactory.provider", "");
System.setProperty("", "JKS");
System.setProperty("", "C:/db2SSL/keystore");

/*SSLContext sslContext = SSLContext.getInstance("TLSv1.2");
sslContext.init(null, null, null);

System.err.println("security.realm.ConnectionFactory.........SSL ON");
} else {
System.err.println("security.realm.ConnectionFactory.........SSL OFF:"+ isSSLEnabled);

conn = DriverManager.getConnection(url, props);

On Local machine we add project in eclipse WAS Server1 and Start it for local testing, where this is failing.

I have tested same code from above class on standalone main () class it works fine , but same class with exact same connection parameter doesn't work when our project from WAS get started.

Dave Hay said...

Hi Prashant

Hmmm, OK, so you're running some Java code that doesn't leverage the WAS infrastructure e.g. JDBC datasources, SSL configuration etc.

I've just revisited this configuration, using some J2SE code, which I've compiled/run using the Java 7 JRE that's part of my WAS environment, and immediately hit similar handshake_failure exceptions etc.

I was able to resolve this - it's all down to the JDBC driver that I was using.

I've written this up as a new blog post here -

Bottom line, check out the version of the JDBC driver that you are using for your standalone code; it may well be relevant.

Cheers, Dave

Yay, VMware Fusion and macOS Big Sur - no longer "NAT good friends" - forgive the double negative and the terrible pun ...

After macOS 11 Big Sur was released in 2020, VMware updated their Fusion product to v12 and, sadly, managed to break Network Address Trans...