Saturday, 26 March 2016

Error handshake_failure seen when connecting to WebSphere Application Server using SOAP over HTTPS

I saw a bunch of SSL-related errors when attempting to use / access the WebSphere Application Server SOAP-based administration service: -

/opt/IBM/WebSphere/AppServer/profiles/Dmgr01/bin/wsadmin.sh -lang jython -user wasadmin -password passw0rd -host `hostname` -port 8879

including: -

...
WASX7023E: Error creating "SOAP" connection to host "bpm856.uk.ibm.com"; exception information: com.ibm.websphere.management.exception.ConnectorNotAvailableException: [SOAPException: faultCode=SOAP-ENV:Client; msg=Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: handshake_failure; targetException=java.lang.IllegalArgumentException: Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: handshake_failure]
com.ibm.websphere.management.exception.ConnectorNotAvailableException: [SOAPException: faultCode=SOAP-ENV:Client; msg=Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: handshake_failure; targetException=java.lang.IllegalArgumentException: Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: handshake_failure]
...
Caused by: [SOAPException: faultCode=SOAP-ENV:Client; msg=Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: handshake_failure; targetException=java.lang.IllegalArgumentException: Error opening socket: java.io.IOException: Exception during sslSocket.startHandshake: Received fatal alert: handshake_failure]

This started happening immediately after I'd locked down WAS using TLS 1.2 and Mutual Authentication.

However, all had been working UNTIL I enforced WAS to use a pair of TLS 1.2 ciphers: -


Specifically, I'm using these: -

TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384


across the "estate" including IBM HTTP Server, WAS and DB2.

This proved to be the root cause ….

I needed to update the Deployment Manager profile's SSL configuration: -

/opt/IBM/WebSphere/AppServer/profiles/Dmgr01/properties/ssl.client.props

from: -

#com.ibm.ssl.enabledCipherSuites=

to: -

com.ibm.ssl.enabledCipherSuites=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384

*NOTE* The list is separated with a space character NOT a comma; see: -

ssl.client.props client configuration file

for details.

For the record, I'd previously done this when I enforced TLS 1.2, by changing: -

com.ibm.ssl.protocol=SSL_TLS

to: -

com.ibm.ssl.protocol=TLSv1.2

4 comments:

Srikanth said...

Thank you Dave, this helped a lot

Dave Hay said...

Srikanth, great news, thanks for letting me know :-)

Saurav Verma said...

Hi Dave,

Thanks for this post, I have similar kind of issue.
I am trying to make a SOAP connection with TLSv1.2 from client to WAS and the connection type is same on both sides. I have already tried with the ciphers suite as you already suggested. What are the conditions or configurations are required if we are making soap connection between client app to WAS..

Can you please help?

Regards,
Saurav

Dave Hay said...

Hi Saurav

Thanks for the comment.

It really depends upon from where you're making the SOAP client. It's important to configure the relevant client tool to support TLS 1.2, plus the required ciphers. It's also important to bear in mind that, if the SOAP client is written using Java, you may need to install the unrestricted Java policy files on the JRE that underlies the client itself.

Bottom line, it does depend from client to client, but you'll need to apply similar configuration changes in any case e.g. support for TLS 1.2, JRE unrestricted policies etc.

This may also help: -

https://portal2portal.blogspot.co.uk/2016/11/websphere-application-server-using-java_15.html?m=1

Cheers, Dave