Saturday, 26 March 2016

Error handshake_failure seen when connecting to WebSphere Application Server using SOAP over HTTPS

I saw a bunch of SSL-related errors when attempting to use / access the WebSphere Application Server SOAP-based administration service: -

/opt/IBM/WebSphere/AppServer/profiles/Dmgr01/bin/wsadmin.sh -lang jython -user wasadmin -password passw0rd -host `hostname` -port 8879

including: -

...
WASX7023E: Error creating "SOAP" connection to host "bpm856.uk.ibm.com"; exception information: com.ibm.websphere.management.exception.ConnectorNotAvailableException: [SOAPException: faultCode=SOAP-ENV:Client; msg=Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: handshake_failure; targetException=java.lang.IllegalArgumentException: Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: handshake_failure]
com.ibm.websphere.management.exception.ConnectorNotAvailableException: [SOAPException: faultCode=SOAP-ENV:Client; msg=Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: handshake_failure; targetException=java.lang.IllegalArgumentException: Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: handshake_failure]
...
Caused by: [SOAPException: faultCode=SOAP-ENV:Client; msg=Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: handshake_failure; targetException=java.lang.IllegalArgumentException: Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: handshake_failure]

This started happening immediately after I'd locked down WAS using TLS 1.2 and Mutual Authentication.

However, all had been working UNTIL I enforced WAS to use a pair of TLS 1.2 ciphers: -


Specifically, I'm using these: -

TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384


across the "estate" including IBM HTTP Server, WAS and DB2.

This proved to be the root cause ….

I needed to update the Deployment Manager profile's SSL configuration: -

/opt/IBM/WebSphere/AppServer/profiles/Dmgr01/properties/ssl.client.props

from: -

#com.ibm.ssl.enabledCipherSuites=

to: -

com.ibm.ssl.enabledCipherSuites=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384

*NOTE* The list is separated with a space character NOT a comma; see: -

ssl.client.props client configuration file

for details.

For the record, I'd previously done this when I enforced TLS 1.2, by changing: -

com.ibm.ssl.protocol=SSL_TLS

to: -

com.ibm.ssl.protocol=TLSv1.2

No comments: