IBM Integration Bus - The Fun Continues .... MQ reason code 2035 while trying to connect

So I now have IBM Integration Bus (IIB) 9.0.0.2 and the Toolkit running on my Red Hat Enterprise Linux VM, as per these most recent posts: -

Installing IBM Integration Bus 9.0.0.2 on Red Hat Enterprise Linux 6.3

IBM Integration Bus 9.0.0.2 - Tooling Up

I have the toolkit installed as user wasadmin because that's the user with which I installed a whole slew of other WebSphere products, using IBM Installation Manager.

Therefore, in order to use the Toolkit to create/administer an Integration Node ( fka Broker ), I need to ensure that the wasadmin user is setup to access IIB and WebSphere MQ (WMQ).

I did this by: -

(a) Adding the user into the appropriate Linux groups - this is what I now have: -

id wasadmin

uid=500(wasadmin) gid=505(mqm) groups=505(mqm),506(mqbrkrs)

id mqm

uid=505(mqm) gid=505(mqm) groups=505(mqm)

id wmbadmin

uid=506(wmbadmin) gid=506(mqbrkrs) groups=506(mqbrkrs),505(mqm)

In essence, I added the wasadmin user to the mqbrkrs group as follows: -

usermod -G mqm,mqbrkrs wasadmin

( as root )

(b) configuring the Bash environment for user wasadmin to use BOTH WMQ and IIB: -

cat /home/wasadmin/.bashrc

# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi

# User specific aliases and functions
umask 022
set -o vi
alias hist='history | cut -c 8-'

. /opt/ibm/mqm/bin/setmqenv -s -k

source /opt/ibm/mqsi/9.0.0.2/bin/mqsiprofile

I have highlighted in bold the two lines added to achieve this.

Now I can log in as wasadmin and create an Integration Node ( aka Broker ), as follows: -

mqsicreatebroker DAVEHAY -q DAVEHAY

WebSphere MQ queue manager created.
Directory '/var/mqm/qmgrs/DAVEHAY' created.
The queue manager is associated with installation 'Installation2'.
Creating or replacing default objects for queue manager 'DAVEHAY'.
Default objects statistics : 79 created. 0 replaced. 0 failed.
Completing setup.
Setup completed.
WebSphere MQ queue manager 'DAVEHAY' starting.
The queue manager is associated with installation 'Installation2'.
5 log records accessed on queue manager 'DAVEHAY' during the log replay phase.
Log replay for queue manager 'DAVEHAY' complete.
Transaction manager state recovered for queue manager 'DAVEHAY'.
WebSphere MQ queue manager 'DAVEHAY' started using V8.0.0.0.
BIP8071I: Successful command completion. 

mqsistart DAVEHAY

BIP8096I: Successful command initiation, check the system log to ensure that the component started without problem and that it continues to run without problem. 

mqsilist

BIP1285I: Broker 'DAVEHAY' on queue manager 'DAVEHAY' is stopped. 
BIP8071I: Successful command completion.

dspmq

QMNAME(QM_MDB)                                            STATUS(Running)
QMNAME(DAVEHAY)                                           STATUS(Running)

I can then start the Toolkit: -

~/IBM/IntegrationToolkit90/launcher

One small glitch - I see this within the Toolkit when I attempt to connect to the Integration Node: -

The user 'wasadmin' is not authorized to connect to queue manager 'DAVEHAY' (MQ reason code 2035 while trying to connect) 

Happily the solution is relatively simple :-)

I looked at the logs: -

cd /var/mqm/qmgrs/DAVEHAY/errors

cat AMQERR01.LOG

wherein I could see: -

AMQ5534: User ID 'wasadmin' authentication failed

EXPLANATION:

The user ID and password supplied by 'javaw' could not be authenticated.

ACTION:

Ensure that the correct user ID and password are provided by the application. Ensure that the authentication repository is correctly configured. Look at previous error messages for any additional information.

AMQ5542: The failed authentication check was caused by the queue manager CONNAUTH CHCKLOCL(OPTIONAL) configuration.

EXPLANATION:

The user ID 'wasadmin' and its password were checked because the queue manager connection authority (CONNAUTH) configuration refers to an authentication information (AUTHINFO) object named 'SYSTEM.DEFAULT.AUTHINFO.IDPWOS' with CHCKLOCL(OPTIONAL).

This message accompanies a previous error to clarify the reason for the user ID and password check.

ACTION:
Refer to the previous error for more information.

Ensure that a password is specified by the client application and that the password is correct for the user ID. The authentication configuration of the queue manager connection determines the user ID repository. For example, the local operating system user database or an LDAP server.

If the CHCKCLNT setting is OPTIONAL, the authentication check can be avoided by not passing a user ID across the channel. For example, by omitting the MQCSP structure from the client MQCONNX API call.

To avoid the authentication check, you can amend the authentication configuration of the queue manager connection, but you should generally not allow unauthenticated remote access.

After some digging, I think that the problem is that the Toolkit is attempting to connect to the DAVEHAY Queue Manager, and is not sending the correct authentication string e.g. user ID *AND* password.

Looking at the Queue Manager in more detail: -

runmqsc DAVEHAY

5724-H72 (C) Copyright IBM Corp. 1994, 2014.
Starting MQSC for queue manager DAVEHAY.

display qmgr connauth

AMQ8408: Display Queue Manager details.
   QMNAME(DAVEHAY)                      
   CONNAUTH(SYSTEM.DEFAULT.AUTHINFO.IDPWOS)

display AUTHINFO(SYSTEM.DEFAULT.AUTHINFO.IDPWOS)

AMQ8566: Display authentication information details.
   AUTHINFO(SYSTEM.DEFAULT.AUTHINFO.IDPWOS)
   AUTHTYPE(IDPWOS)                        ADOPTCTX(NO)
   DESCR( )                                CHCKCLNT(REQDADM)
   CHCKLOCL(OPTIONAL)                      FAILDLAY(1)
   ALTDATE(2014-09-19)                     ALTTIME(11.21.38)

My reading of this is: -

(a) the Toolkit is sending a user ID ( wasadmin ) but NOT sending a password

(b) the Queue Manager expects BOTH a user ID and a password

Interestingly, the Toolkit, leastways on Linux, doesn't give one the opportunity to specific connection credentials :-)

Therefore, the Toolkit, whilst running as wasadmin, is trying to connect to the Queue Manager as that user wit NO password :-)

Ironically, I'm making a LOCAL connection as both the Toolkit AND the Queue Manager are running on the same OS.

After much faffing about, I found this excellent deck from my IBM Hursley colleague, Morag Hughson: -

Session 1873: IBM WebSphere MQ Security: Latest Features Deep Dive

which led me to this solution: -

ALTER AUTHINFO(SYSTEM.DEFAULT.AUTHINFO.IDPWOS) AUTHTYPE(IDPWOS) CHCKCLNT(REQDADM) CHCKLOCL(NONE)    
REFRESH SECURITY TYPE(CONNAUTH)

which, as far as I can establish, means that MQ will require a valid set of credentials for client connections, via CHCKCLNT(REQDADM), but not for local connections, via CHCKLOCL(NONE).

To verify this hypothesis, I switched back to the old state: -

ALTER AUTHINFO(SYSTEM.DEFAULT.AUTHINFO.IDPWOS) AUTHTYPE(IDPWOS) CHCKCLNT(REQDADM) CHCKLOCL(OPTIONAL)    

REFRESH SECURITY TYPE(CONNAUTH)

and my old friend: -

The user 'wasadmin' is not authorized to connect to queue manager 'DAVEHAY' (MQ reason code 2035 while trying to connect) 

popped up.

I reverted back to CHCKLOCL(NONE) and I'm now in like Flynn: -

On a related note, I had a similar, but different problem with WMQ Explorer ( running as another user - wmbadmin ): -

/opt/ibm/WebSphere_MQ_Explorer/MQExplorer

With this, I was attempting to make a remote connection to the same Queue Manager: -

Host name or IP address bam8012.uk.ibm.com

Port number 1414

Server-connection channel SYSTEM.DEF.SVRCONN

Unlike the IIB Toolkit, this DOES allow me to enter credentials.

However, Explorer failed to connect with: -

Could not establish a connection to the queue manager - reason 2538. (AMQ4059)
  Could not establish a connection to the queue manager - reason 2538. (AMQ4059)
  Severity: 10 (Warning)
  Explanation: The attempt to connect to the queue manager failed. This could be because the queue manager is incorrectly configured to allow a connection from this system, or the connection has been broken.
  Response: Try the operation again. If the error persists, examine the problem determination information to see if any information has been recorded.

I took the default port of 1414 so then checked to see whether it was running: -

netstat -aon | grep LISTENING | grep 1414

unix  2      [ ACC ]     STREAM     LISTENING     11460  /var/run/mcelog-client

In other words, nothing was listening on port 1414.

I did have a Channel defined: -

display CHANNEL(SYSTEM.DEF.SVRCONN)

AMQ8414: Display Channel details.
   CHANNEL(SYSTEM.DEF.SVRCONN)             CHLTYPE(SVRCONN)
   ALTDATE(2014-09-19)                     ALTTIME(11.21.38)
   CERTLABL( )                             COMPHDR(NONE)
   COMPMSG(NONE)                           DESCR( )
   DISCINT(0)                              HBINT(300)
   KAINT(AUTO)                             MAXINST(999999999)
   MAXINSTC(999999999)                     MAXMSGL(4194304)
   MCAUSER( )                              MONCHL(QMGR)
   RCVDATA( )                              RCVEXIT( )
   SCYDATA( )                              SCYEXIT( )
   SENDDATA( )                             SENDEXIT( )
   SHARECNV(10)                            SSLCAUTH(REQUIRED)
   SSLCIPH( )                              SSLPEER( )
   TRPTYPE(TCP)                         

which was running: -

display chstatus(*)

AMQ8417: Display Channel Status details.
   CHANNEL(SYSTEM.DEF.SVRCONN)             CHLTYPE(SVRCONN)
   CONNAME(192.168.8.100)                  CURRENT
   STATUS(RUNNING)                         SUBSTATE(RECEIVE)

and a default Listener: -

display    LISTENER(SYSTEM.DEFAULT.LISTENER.TCP)

AMQ8630: Display listener information details.
   LISTENER(SYSTEM.DEFAULT.LISTENER.TCP)   CONTROL(MANUAL)
   TRPTYPE(TCP)                            PORT(0)
   IPADDR( )                               BACKLOG(0)
   DESCR( )                                ALTDATE(2014-09-19)
   ALTTIME(11.21.38)                    

*BUT* the Listener was listening on .... port 0, which ain't ever gonna work :-)

Therefore, I needed to create a new Listener: -

DEFINE LISTENER(DAVEHAY) TRPTYPE(TCP) CONTROL(QMGR) PORT(1414)

AMQ8626: WebSphere MQ listener created.

and then start it: -

START LISTENER(DAVEHAY)

AMQ8021: Request to start WebSphere MQ listener accepted.

Again, I'm in like Flynn: -

which is nice.

Ain't it sweet ?