WAS and Kerberos and SPNEGO - it's been a while - but it's all right now, in fact it it's a gas

It's been a while since first I worked with WAS, Kerberos and SPNEGO - 2011 was SUCH a long time ago.

We're configuring desktop SSO between WAS ND 8.5.5.0 ( specifically for BPM Advanced 8.5.5.0 but that's not important right now ) and MS Active Directory 2008.

I'd previously documented the required steps to create the Service Principal Name (SPN) and Kerberos key tab in AD, and I was confident that there were no issues there.

However, when we started up the WAS JVM ( we were specifically looking at the Deployment Manager ), we saw the following in SystemOut.log: -

[25/06/15 14:05:19:001 BST] 0000015a ServerCredent E com.ibm.ws.security.spnego.ServerCredential initialize CWSPN0014E: An exception occurred during Kerberos initialization. Failure: org.ietf.jgss.GSSException, major code: 13, minor code: 0 major string: Invalid credentials minor string: Cannot get credential from JAAS Subject for principal: HTTP/bpm101@2008ADMINLDAPS.

[25/06/15 14:05:19:008 BST] 0000015a ServerCredent E com.ibm.ws.security.spnego.ServerCredentialsFactory initializeServer CWSPN0015E: Unable to create a GSSCredential for: HTTP/bpm101@2008ADMINLDAPS

[25/06/15 14:05:19:008 BST] 0000015a ServerCredent E com.ibm.ws.security.spnego.ServerCredentialsFactory initializeServerCreds CWSPN0017E: Unable to create GSSCredentials for any of the hosts specified in the configuration properties.

[25/06/15 14:05:19:048 BST] 0000015a TrustAssociat E com.ibm.ws.security.spnego.TrustAssociationInterceptorImpl initialize CWSPN0009E: SPNEGO Trust Association Interceptor configuration is not valid. Failure condition: initializeServerCreds: no credentials successfully created.. If you are not using the SPNEGO TAI, you can ignore this message.

and, in the FFDC logs: -

org.ietf.jgss.GSSException, major code: 13, minor code: 0 major string: Invalid credentials
minor string: Cannot get credential from JAAS Subject for principal: HTTP/bpm101@2008ADMINLDAPS
 at com.ibm.security.jgss.i18n.I18NException.throwGSSException(I18NException.java:35)

[25/06/15 14:05:19:005 BST]     FFDC Exception:org.ietf.jgss.GSSException SourceId:com.ibm.ws.security.spnego.ServerCredentialsFactory.initializeServer ProbeId:68 org.ietf.jgss.GSSException, major code: 13, minor code: 0 major string: Invalid credentials
minor string: Cannot get credential from JAAS Subject for principal: HTTP/bpm101@2008ADMINLDAPS

[25/06/15 14:05:19:023 BST]     FFDC Exception:com.ibm.ws.security.spnego.TAIConfigurationException SourceId:com.ibm.ws.security.spnego.TrustAssociationInterceptorImpl.initialize ProbeId:332 Reporter:com.ibm.ws.security.spnego.TrustAssociationInterceptorImpl@7238f7dd

com.ibm.ws.security.spnego.TAIConfigurationException: initializeServerCreds: no credentials successfully created.

We started by checking the WAS configuration and then moved on to the Kerberos key tab: -

/opt/ibm/WebSphere/AppServer/java/jre/bin/java com.ibm.security.krb5.internal.tools.Klist -e -k /home/wasadmin/keytabs/bpm.keytab

which returns: -

Key table: /home/wasadmin/keytabs/bpm.keytab
Number of entries: 1

1] principal: HTTP/bpm101.uk.ibm.com@2008ADMINLDAPS
        KVNO: 4

       Encryption type: RC4 with HMAC

We also checked the WAS Kerberos configuration file, krb5.conf.

[libdefaults]
        default_realm = uk.ibm.com
        default_keytab_name = FILE:/home/wasadmin/keytabs/bpm.keytab
        default_tkt_enctypes = des3-cbc-sha1
        default_tgs_enctypes = des3-cbc-sha1
        forwardable  = true
        renewable  = true
        noaddresses = true
        clockskew  = 300

[realms]
        uk.ibm.com = {
                kdc = 2008ADMINLDAPS.uk.ibm.com:88
                default_domain = uk.ibm.com
        }

[domain_realm]
        .uk.ibm.com = uk.ibm.com

We then started to look at the hostname of the server, as compared with what DNS shows.

Looking back at an older blog post ( from June 2014 ): -

More on Java and Canonical Hostnames

I compiled and tested a Java class: -

import java.net.InetAddress;
import java.net.UnknownHostException;

public class hostStuff
{
public static void main(String[] args)
{

try
{
InetAddress address = InetAddress.getLocalHost();
System.out.println("My IP address ( via InetAddress.getLocalHost() ) is " + address.toString());
System.out.println("My hostname ( via InetAddress.getHostName() ) is " + address.getHostName());
System.out.println("My hostname ( via InetAddress.getCanonicalHostname() ) is  " + address.getCanonicalHostName());
}
catch (UnknownHostException e)
{
       System.out.println("I'm sorry. I don't know my own name.");
}
}
}

which showed that, whilst we were seeing the correct hostname, we were ONLY seeing the short name, rather than the fully-qualified service name.

In other words, we were seeing bpm101 rather than bpm101.uk.ibm.com.

When we checked further, that's what /etc/hosts was configured to use.

We asked the AIX sysadmin to change this - he commented out the entry in the hosts file - and then restarted WAS.

Immediately, SPNEGO burst into life.

Bottom line, it doesn't matter how good the SPNEGO and WAS configuration is, it is CRUCIAL to ensure that the hostname that WAS can see, via the various Java methods, matches what's been configured for the Service Principal Name (SPN).

Life is, as they say, a learning curve :-)