Wednesday, 15 March 2017

JVMCFRE003 bad major version; class=org/apache/derby/jdbc/EmbeddedConnectionPoolDataSource40, offset=6

I saw this today: -

[ERROR   ] An error has occurred in trying to access data source 'jdbc/ilogDataSource': Could not lookup datasource named 'jdbc/ilogDataSource'. Check that the data source exists on the application server or contact your administrator.
An error has occurred in trying to access data source 'jdbc/ilogDataSource': Could not lookup datasource named 'jdbc/ilogDataSource'. Check that the data source exists on the application server or contact your administrator.
[ERROR   ] CWWKE0701E: FrameworkEvent ERROR Bundle:com.ibm.ws.jdbc(id=110) org.osgi.framework.ServiceException: Exception in com.ibm.ws.resource.internal.ResourceFactoryTrackerData$1.getService()
at org.eclipse.osgi.internal.serviceregistry.ServiceFactoryUse.factoryGetService(ServiceFactoryUse.java:222)
at [internal classes]
at javax.naming.InitialContext.lookup(InitialContext.java:423)
at ilog.rules.teamserver.ejb.service.dao.IlrElementDAOFactory.getInstance(IlrElementDAOFactory.java:70)
at ilog.rules.teamserver.ejb.service.IlrSessionFacadeImpl.loginInitializationWithTxnPart1(IlrSessionFacadeImpl.java:414)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at ilog.rules.teamserver.transaction.IlrTransactionalInvocationHandler.invoke(IlrTransactionalInvocationHandler.java:39)
at com.sun.proxy.$Proxy36.loginInitializationWithTxnPart1(Unknown Source)
at ilog.rules.teamserver.ejb.service.IlrLocalSessionImpl.loginInitializationWithTxnPart1(IlrLocalSessionImpl.java:64)
at ilog.rules.teamserver.ejb.service.IlrSessionFacadeImpl.login(IlrSessionFacadeImpl.java:393)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at ilog.rules.teamserver.transaction.IlrTransactionalInvocationHandler.invoke(IlrTransactionalInvocationHandler.java:39)
at com.sun.proxy.$Proxy36.login(Unknown Source)
at ilog.rules.teamserver.ejb.service.IlrLocalSessionImpl.login(IlrLocalSessionImpl.java:58)
at ilog.rules.teamserver.ejb.service.IlrLocalSessionFactory.makeSession(IlrLocalSessionFactory.java:68)
at ilog.rules.teamserver.ejb.service.IlrLocalSessionFactory.connect(IlrLocalSessionFactory.java:49)
at ilog.rules.teamserver.ejb.service.IlrLocalSessionFactory.connect(IlrLocalSessionFactory.java:59)
at com.ibm.rules.decisioncenter.web.core.UserContext.connect(UserContext.java:48)
at com.ibm.rules.decisioncenter.web.core.ApplicationInterceptor.preHandle(ApplicationInterceptor.java:117)
at org.springframework.web.servlet.HandlerExecutionChain.applyPreHandle(HandlerExecutionChain.java:134)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:913)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:851)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:953)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:844)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:575)
at [internal classes]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:829)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at [internal classes]
at com.ibm.rules.decisioncenter.web.core.filters.SecurityCheckPointFilter.doFilter(SecurityCheckPointFilter.java:95)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:207)
at [internal classes]
at com.ibm.rules.decisioncenter.web.core.filters.RemoteSessionFilter.doFilterInternal(RemoteSessionFilter.java:79)
at com.ibm.rules.decisioncenter.web.core.filters.RemoteSessionFilter.doFilter(RemoteSessionFilter.java:59)
at com.ibm.rules.decisioncenter.web.core.filters.SessionFilter.access$001(SessionFilter.java:32)
at com.ibm.rules.decisioncenter.web.core.filters.SessionFilter$1.doFilter(SessionFilter.java:73)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:106)
at com.ibm.rules.decisioncenter.web.core.filters.SessionFilter.doFilter(SessionFilter.java:70)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:207)
at [internal classes]
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:106)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:207)
at [internal classes]
at com.ibm.rules.decisioncenter.web.core.filters.HttpPUTRequestFilter.doFilterInternal(HttpPUTRequestFilter.java:65)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:106)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:207)
at [internal classes]
Caused by: java.lang.UnsupportedClassVersionError: JVMCFRE003 bad major version; class=org/apache/derby/jdbc/EmbeddedConnectionPoolDataSource40, offset=6
at java.lang.ClassLoader.defineClassImpl(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:331)
at com.ibm.ws.classloading.internal.AppClassLoader.definePackageAndClass(AppClassLoader.java:327)
... 54 more

Referencing back to a previous post: -


reminded me to do this: -

/opt/ibm/WebSphere/Liberty/usr/shared/resources/derby/derby.jar!/org/apache/derby/jdbc/EmbeddedConnectionPoolDataSource40.class

which resulted in: -

  Last modified 15-Oct-2016; size 334 bytes
  MD5 checksum 9e105007b71e5d0460499ca4564efd1b
public class org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource40 extends org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource implements javax.sql.ConnectionPoolDataSource
  Deprecated: true
  minor version: 0
  major version: 52
  flags: ACC_PUBLIC, ACC_SUPER
Constant pool:
   #1 = Methodref          #3.#14         //  org/apache/derby/jdbc/EmbeddedConnectionPoolDataSource."<init>":()V
   #2 = Class              #15            //  org/apache/derby/jdbc/EmbeddedConnectionPoolDataSource40
   #3 = Class              #16            //  org/apache/derby/jdbc/EmbeddedConnectionPoolDataSource
   #4 = Class              #17            //  javax/sql/ConnectionPoolDataSource
   #5 = Utf8               serialVersionUID
   #6 = Utf8               J
   #7 = Utf8               ConstantValue
   #8 = Long               -4368824293743156916l
  #10 = Utf8               <init>
  #11 = Utf8               ()V
  #12 = Utf8               Code
  #13 = Utf8               Deprecated
  #14 = NameAndType        #10:#11        //  "<init>":()V
  #15 = Utf8               org/apache/derby/jdbc/EmbeddedConnectionPoolDataSource40
  #16 = Utf8               org/apache/derby/jdbc/EmbeddedConnectionPoolDataSource
  #17 = Utf8               javax/sql/ConnectionPoolDataSource
{
  public org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource40();
    flags: ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0       
         1: invokespecial #1                  // Method org/apache/derby/jdbc/EmbeddedConnectionPoolDataSource."<init>":()V
         4: return        
}

The previous post took me back to this: -



I checked my JRE version: -

/opt/ibm/WebSphere/Liberty/java/java_1.7.1_64/bin/java -version

java version "1.7.0"
Java(TM) SE Runtime Environment (build pxa6470_27sr3fp40ifix-20160711_01(SR3 FP40+IX90174))
IBM J9 VM (build 2.7, JRE 1.7.0 Linux amd64-64 Compressed References 20160406_298393 (JIT enabled, AOT enabled)
J9VM - R27_Java727_SR3_20160406_0942_B298393
JIT  - tr.r13.java_20160328_114186
GC   - R27_Java727_SR3_20160406_0942_B298393_CMPRSS
J9CL - 20160406_298393)
JCL - 20160421_01 based on Oracle jdk7u101-b14


which confirmed my feelings :-)

I was using a version of Derby DB that was compiled using Java 8 whereas I'm using Java 7.

This was what I had: -


<snip>
The 10.13 release family supports the following Java and JDBC versions:

• Java SE 8 with JDBC 4.2
</snip>

so I switched to: -


<snip>
Support for Java SE 6 and Java SE 7 is being sunsetted. The 10.13 release family will not support those platforms. The 10.12 release family supports the following Java and JDBC versions:

• Java SE 6 and higher with JDBC 4.0, 4.1, and 4.2.
• Java SE 8 compact profile 2.
</snip>

Having downloaded the .ZIP, and expanded it to /tmp I grabbed the .JAR file: -

cd /opt/ibm/WebSphere/Liberty/usr/shared/resources/derby
cp /tmp/db-derby-10.12.1.1-bin/lib/derby.jar .

Job done :-)



Monday, 13 March 2017

WebSphere Application Server - DNS, you can't fool it - or can you ?

I saw this: -

[3/12/17 19:55:21:158 UTC] 00000001 LogAdapter    E   DCSV9403E: Received an illegal configuration argument. Parameter MulticastInterface, value: 9.20.65.171. Exception is java.lang.Exception: Network I
nterface 9.20.65.171 was not found in local machine network interface list. Make sure that the NetworkInterface property is properly configured!
at com.ibm.rmm.mtl.transmitter.Config.<init>(Config.java:238)


while attempting to start a WebSphere Application Serve (WAS) Deployment Manager ( as part of an IBM BPM Advanced build ).

I wanted to drill into the issue, and see what hostname the offending IP address ( 9.20.65.171 ) was aliased to.

I tried/failed to use dig as it wasn't installed on my RHEL 7.3 VM.

So I installed bind-utils: -

yum install -y bind-utils

and then used dig as follows: -

dig 9.20.65.171

; <<>> DiG 9.9.4-RedHat-9.9.4-38.el7_3.2 <<>> 9.20.65.171
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 39457
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;9.20.65.171. IN A

;; AUTHORITY SECTION:
. 10800 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2017031201 1800 900 604800 86400

;; Query time: 20 msec
;; SERVER: 9.20.136.11#53(9.20.136.11)
;; WHEN: Sun Mar 12 21:07:38 UTC 2017
;; MSG SIZE  rcvd: 115


and host : -

host 9.20.65.171

171.65.20.9.in-addr.arpa domain name pointer hyc1-vm2417.hursley.ibm.com.

I also checked /etc/hosts which contained: -

# Your system has configured 'manage_etc_hosts' as True.
# As a result, if you wish for changes to this file to persist
# then you will need to either
# a.) make changes to the master file in /etc/cloud/templates/hosts.redhat.tmpl
# b.) change or remove the value of 'manage_etc_hosts' in
#     /etc/cloud/cloud.cfg or cloud-config from user-data

# The following lines are desirable for IPv4 capable hosts
127.0.0.1 bpm857.novalocal bpm857 bpm857.uk.ibm.com
127.0.0.1 localhost.localdomain localhost
127.0.0.1 localhost4.localdomain4 localhost4

# The following lines are desirable for IPv6 capable hosts
::1 bpm857.novalocal bpm857
::1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6

9.20.65.75 oracle.uk.ibm.com oracle
9.20.65.140 windows2012.uk.ibm.com windows2012
9.20.65.171 bpm857.uk.ibm.com bpm857

I revisited an old post of mine: -


and resurrected an old Java class: -

hostStuff.java

import java.net.InetAddress;
import java.net.UnknownHostException;

public class hostStuff
{
public static void main(String[] args)
{

try
{
InetAddress address = InetAddress.getLocalHost();
System.out.println("My IP address ( via InetAddress.getLocalHost() ) is " + address.toString());
System.out.println("My hostname ( via InetAddress.getHostName() ) is " + address.getHostName());
System.out.println("My hostname ( via InetAddress.getCanonicalHostname() ) is  " + address.getCanonicalHostName());
}
catch (UnknownHostException e)
{
       System.out.println("I'm sorry. I don't know my own name.");
}
}
}


which, when run, returned: -

My IP address ( via InetAddress.getLocalHost() ) is bpm857.uk.ibm.com/127.0.0.1
My hostname ( via InetAddress.getHostName() ) is bpm857.uk.ibm.com
My hostname ( via InetAddress.getCanonicalHostname() ) is  bpm857.novalocal

I suspect that WAS is executing InetAddress.getCanonicalHostname() under the covers, which is the real hostname of the box - bpm857.novalocal - as dictated by OpenStack, whereas I'm trying to spoof WAS to use a different name.

So I "hacked" /etc/hosts : -

# Your system has configured 'manage_etc_hosts' as True.
# As a result, if you wish for changes to this file to persist
# then you will need to either
# a.) make changes to the master file in /etc/cloud/templates/hosts.redhat.tmpl
# b.) change or remove the value of 'manage_etc_hosts' in
#     /etc/cloud/cloud.cfg or cloud-config from user-data

# The following lines are desirable for IPv4 capable hosts
127.0.0.1 bpm857.uk.ibm.com bpm857
127.0.0.1 localhost.localdomain localhost
127.0.0.1 localhost4.localdomain4 localhost4

# The following lines are desirable for IPv6 capable hosts
::1 bpm857.novalocal bpm857
::1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6

9.20.65.75 oracle.uk.ibm.com oracle
9.20.65.140 windows2012.uk.ibm.com windows2012
9.20.65.171 bpm857.uk.ibm.com bpm857


effectively removing reference to bpm857.novalocal which my class confirmed: -

My IP address ( via InetAddress.getLocalHost() ) is bpm857.uk.ibm.com/127.0.0.1
My hostname ( via InetAddress.getHostName() ) is bpm857.uk.ibm.com
My hostname ( via InetAddress.getCanonicalHostname() ) is  bpm857.uk.ibm.com

I tried to start the Deployment Manager, but that again failed with the same exception as before.

I further "hacked" /etc/hosts : -

# Your system has configured 'manage_etc_hosts' as True.
# As a result, if you wish for changes to this file to persist
# then you will need to either
# a.) make changes to the master file in /etc/cloud/templates/hosts.redhat.tmpl
# b.) change or remove the value of 'manage_etc_hosts' in
#     /etc/cloud/cloud.cfg or cloud-config from user-data

# The following lines are desirable for IPv4 capable hosts
127.0.0.1 localhost.localdomain localhost
127.0.0.1 localhost4.localdomain4 localhost4

# The following lines are desirable for IPv6 capable hosts
::1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6

9.20.65.75 oracle.uk.ibm.com oracle
9.20.65.140 windows2012.uk.ibm.com windows2012
9.20.65.171 bpm857.uk.ibm.com bpm857


but to no avail.

In the end, I bowed to the inevitable and stuck with the host / domain that OpenStack was allocating, especially given that my VM is going to be connecting to other VMs on the same OpenStack box, as well as to the outside world.

Therefore, I reverted my WAS configuration to use the bpm857.novalocal host/domain name internally, and continued to reference it externally as bpm857.uk.ibm.com.

Thankfully, when I rebooted the VM, the /etc/hosts file was reverted back to its original state, 

Now, when I run my hostStuff class, I get: -

My IP address ( via InetAddress.getLocalHost() ) is bpm857.novalocal/127.0.0.1
My hostname ( via InetAddress.getHostName() ) is bpm857.novalocal
My hostname ( via InetAddress.getCanonicalHostname() ) is  bpm857.novalocal


and I've reverted my response file accordingly: -

cat /tmp/ResponseFiles/Advanced-PS-ThreeClusters-Oracle.properties |grep -i hostname

bpm.de.psProcessCenterHostname=
bpm.dmgr.hostname=bpm857.novalocal
bpm.de.node.1.hostname=bpm857.novalocal
bpm.de.db.1.hostname=oracle.novalocal
bpm.de.db.2.hostname=oracle.novalocal
bpm.de.db.3.hostname=oracle.novalocal
bpm.de.db.4.hostname=oracle.novalocal


So we're back in the game ….

The moral of the story ? Ensure that you use host/domain names that are valid for your environment, networking with your network folks, as required.

Also, never use localhost for WAS hostnames :-)

Sunday, 12 March 2017

Weirdness - The database is not consistent with the BPM Product version

So we saw this issue last week: -

[3/9/17 7:06:35:804 UTC] 00000001 WsServerImpl  E   WSVR0009E: Error occurred during startup
com.ibm.ws.exception.RuntimeError: The database [jndi = jdbc/PerformanceDB] version [8.5.7.201612] is not consistent with the BPM Product version [8.5.7.0], please ensure the database is updated successfully before starting server.


during the build of a new IBM Business Process Manager (BPM) Advanced 8.5.7 Deployment Environment with an Oracle 12c database.

The problem appeared to be an inconsistency between database tables within Oracle and the expectations of the BPM product, specifically the LSW_SYSTEM_SCHEMA tables, of which there are two: -

PSUSER.LSW_SYSTEM_SCHEMA
PDWUSER.LSW_SYSTEM_SCHEMA

So, here's the thing …

These tables are defined when one first builds the BPM Deployment Environment using BPMConfig.sh: -

/opt/ibm/WebSphere/AppServer/bin/BPMConfig.sh -create -de /mnt/ResponseFiles/ Advanced-PS-ThreeClusters-Oracle.properties

specifically, here: -

/opt/ibm/WebSphereProfiles/Dmgr01/dbscripts/PSCell1.De1/Oracle/orcl/psuser/createSchema_Advanced.sql: psuser.LSW_SYSTEM_SCHEMA WHERE PROPNAME = 'DatabaseSchemaVersion' ; 
/opt/ibm/WebSphereProfiles/Dmgr01/dbscripts/PSCell1.De1/Oracle/orcl/psuser/createSchema_Advanced.sql: VALUES ('DatabaseSchemaVersion',

and: -

/opt/ibm/WebSphereProfiles/Dmgr01/dbscripts/PSCell1.De1/Oracle/orcl/pdwuser/createSchema_Advanced.sql: pdwuser.LSW_SYSTEM_SCHEMA WHERE PROPNAME = 'DatabaseSchemaVersion' ; 
/opt/ibm/WebSphereProfiles/Dmgr01/dbscripts/PSCell1.De1/Oracle/orcl/pdwuser/createSchema_Advanced.sql: VALUES ('DatabaseSchemaVersion',

thanks to fgrep : -

fgrep -R DatabaseSchemaVersion /opt/ibm/WebSphereProfiles/Dmgr01/dbscripts/

Note that the exception related to the Performance Data Warehouse (PDW) database/schema, because we were starting the SupCluster, within which the PDW applications run.

When we saw this issue last week, the problem was that the tables in Oracle contained the value: -

8.5.7

( namely three columns / digits ), as evidenced: -

select * from PSUSER.LSW_SYSTEM_SCHEMA;

PROPNAME PROPVALUE
DatabaseSchemaVersion 8.5.7

select * from PDWUSER.LSW_SYSTEM_SCHEMA;

PROPNAME PROPVALUE
DatabaseSchemaVersion 8.5.7


whereas BPM appeared to be expected four columns / digits.

To prove / circumvent the issue, our friendly Oracle DBA deleted the current row: -

delete from PDWUSER.LSW_SYSTEM_SCHEMA where PROPNAME='DatabaseSchemaVersion';

and inserted a new row: -

insert into PDWUSER.LSW_SYSTEM_SCHEMA(PROPNAME,PROPVALUE) VALUES('DatabaseSchemaVersion','8.5.7.0');

and validated the change: -

select PROPNAME,PROPVALUE from PDWUSER.LSW_SYSTEM_SCHEMA where PROPNAME='DatabaseSchemaVersion';

PROPNAME PROPVALUE
DatabaseSchemaVersion 8.5.7.0


Once we did this, the SupCluster started without exception.

The strange thing ?

I can't recreate the same problem :-(

I'm using the same software: -

/opt/ibm/InstallationManager/eclipse/tools/imcl listInstalledPackages

com.ibm.cic.agent_1.8.6000.20161118_1611
com.ibm.bpm.ADV.v85_8.5.7000.20160301_1551
com.ibm.websphere.ND.v85_8.5.5011.20161206_1434
com.ibm.websphere.IHS.v85_8.5.5011.20161206_1434
com.ibm.websphere.PLG.v85_8.5.5011.20161206_1434

and followed the same installation process; specifically installing BPM Advanced 8.5.0.0 and patching to 8.5.7.0 at the same time ( plus also adding the Q1 2016 cumulative fix pack - CF2016.03

Specifically, this is what I see in the tables created by BPConfig.sh : -

/opt/ibm/WebSphereProfiles/Dmgr01/dbscripts/PSCell1.De1/Oracle/orcl/psuser/createSchema_Advanced.sql

DECLARE
v_table_count NUMBER;
  BEGIN
   SELECT COUNT(*) INTO v_table_count FROM
    psuser.LSW_SYSTEM_SCHEMA WHERE PROPNAME = 'DatabaseSchemaVersion' ;
     IF (v_table_count = 0) THEN
      INSERT INTO
    psuser.LSW_SYSTEM_SCHEMA("PROPNAME",
 "PROPVALUE")
VALUES ('DatabaseSchemaVersion',
 '8.5.7') ;
END IF ;
END ;


/opt/ibm/WebSphereProfiles/Dmgr01/dbscripts/PSCell1.De1/Oracle/orcl/pdwuser/createSchema_Advanced.sql

DECLARE
v_table_count NUMBER;
  BEGIN
   SELECT COUNT(*) INTO v_table_count FROM
    pdwuser.LSW_SYSTEM_SCHEMA WHERE PROPNAME = 'DatabaseSchemaVersion' ;
     IF (v_table_count = 0) THEN
      INSERT INTO
    pdwuser.LSW_SYSTEM_SCHEMA("PROPNAME",
 "PROPVALUE")
VALUES ('DatabaseSchemaVersion',
 '8.5.7') ;
END IF ;
END ;


So, as far as I can establish, we shouldn't have needed to apply the modification in Oracle :-(

Whilst I'm not 100% sure what went wrong, I do, at least, have a mitigation if it occurs again.

For the client in question, we're going for a clean installation of BPM Advanced 8.5.7.0 plus the Q4 2016 cumulative fix pack - CF2016.12.

For the record, I had seen a similar problem before: -


That time around, I'd NOT run the appropriate createSchema_Advanced script.

Also, the IBM Knowledge Centre reports: -


• Make sure that there is one row in the LSW_SYSTEM_SCHEMA table of the Process Server or Performance Data Warehouse database with the name DatabaseSchemaVersion. Make sure that the value is the same as the value of the IBM BPM version of your current profile.
• If the row does not exist, insert a new row with the correct value. If the row does not exist, insert a new row with the correct value. The value must be a 3-digit version number, for example, 8.5.6.
• If the row exists but the value is not the same as your current profile, you might not have finished migration. Make sure to run the database upgrade step to upgrade your database.
• To disable version validation, set a WebSphere variable in the cell scope. Log in to the administrative console and go to Environment > WebSphere variables. Under cell scope, create a variable named SKIPSTDDBVERSIONCHECK and set its value to true. Restart the servers. The version validation is skipped during server start.

which is pretty spot-on.

Saturday, 11 March 2017

Scripting in Python and Jython with added OS commands

I was writing a generic ( use anywhere ) script to add a BPM URL to my IBM BPM Advanced 8.5.7 environment.

For me, all of the components ( IBM HTTP Server, WebSphere Plugin and WebSphere Application Server / BPM ) are on the same VM.

For the record, I'm running the VM on OpenStack.

Therefore, I wanted a script that would get the hostname of the VM on which IHS/WAS is running.

This served as source: -


and this was my test script: -

 foo.jy 
import socket

hostname=socket.gethostname()
print("Hostname is " + hostname)


which I tested using Python: -

python foo.jy 

Hostname is bpm857.novalocal

and Jython: -

/opt/ibm/WebSphereProfiles/Dmgr01/bin/wsadmin.sh -lang jython -conntype none -f foo.jy 

WASX7357I: By request, this scripting client is not connected to any server process. Certain configuration and application operations will be available in local mode.
Hostname is bpm857.novalocal


This is the pukka script: -

configureBPMURL.jy

import socket

# Set variables

cellID=AdminControl.getCell()
dePath='/Cell:'+ cellID+'/BPMCellConfigExtension:/BPMDeploymentEnvironment:/'
de=AdminConfig.getid(dePath)
hostname=socket.gethostname()
print("Hostname is " + hostname)

# List existing BPM URLs

bpmurlsid=AdminConfig.getid(dePath+'BPMURLS:/')
bpmurllist=AdminUtilities.convertToList(AdminConfig.list("BPMURL", bpmurlsid))
for item in bpmurllist :
 print AdminConfig.show(item)

# Remove existing BPM URLs

for i in AdminConfig.list('BPMVirtualHostInfo').split():
 AdminConfig.remove(i)

# Create new Virtual Host

webserver_vh = AdminConfig.create('BPMVirtualHostInfo',de,[['name','webserver_vh'],['transportProtocol','https'],['hostname',hostname],['port','8443']],'virtualHosts')
AdminConfig.modify(de,[['defaultVH',webserver_vh]])

# Save and Sync

AdminConfig.save()
AdminNodeManagement.syncActiveNodes()

# Validate

print AdminConfig.show(AdminConfig.list('BPMVirtualHostInfo'))


which worked as follows: -

/opt/ibm/WebSphereProfiles/Dmgr01/bin/wsadmin.sh -lang jython -f configureBPMURL.jy 

Realm/Cell Name: <default>
Username: wasadmin
Password:         
 WASX7209I: Connected to process "dmgr" on node Dmgr using SOAP connector;  The type of process is: DeploymentManager
Hostname is bpm857.novalocal
[scenario EXTERNAL_CLIENT]
[strategies "WCCMConfigStrategy, HttpProtocolHostStrategy"]
[scenario INTERNAL_CLIENT]
[scenario RELATIVE]
[strategies RelativeUrlStrategy]
---------------------------------------------------------------
 AdminNodeManagement:        Synchronize the active nodes
 Usage: AdminNodeManagement.syncActiveNodes()
 Return: If the command is successfully invoked, a value of 1 is returned. 
---------------------------------------------------------------
 
 
Node1
[hostname bpm857.novalocal]
[name webserver_vh]
[port 8443]
[transportProtocol https]


which is nice

This was also useful: -


although the socket.gethostname() was more useful.

Friday, 10 March 2017

CTGSK3046W - IBM HTTP Server - Certificates and Permissions

I saw this: -

CTGSK3046W The key file "/tmp/ad2012.cer" could not be imported.

whilst trying to add a CA Signer certificate to a keystore using IBM HTTP Server: -

/opt/ibm/HTTPServer/bin/gskcapicmd -cert -add -file /tmp/ad2012.cer -db /opt/ibm/HTTPServer/BPM/ssl/keystore.kdb -stashed

Having checked and double-checked my command, I then tried to use openSSL to validate the certificate: -

openssl x509 -inform der -in /tmp/ad2012.cer -text -noout

Error opening Certificate /tmp/ad2012.cer
140581419276192:error:0200100D:system library:fopen:Permission denied:bss_file.c:398:fopen('/tmp/ad2012.cer','r')
140581419276192:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:400:
unable to load certificate


which was more revealing.

A quick check: -

ls -al /tmp/ad2012.cer 

-rw------- 1 cloudusr cloudusr 915 Mar 10 19:26 /tmp/ad2012.cer

confirmed that it was likely a permissions issue.

Once I fixed the permissions ( as root ): -

chmod -R 777 /tmp

we're back in the game: -

openssl x509 -inform der -in /tmp/ad2012.cer -text -noout

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            1f:92:ac:6d:1a:57:e9:b4:43:d3:81:64:ff:9e:93:d6
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: DC=com, DC=ibm, DC=uk, CN=uk-WINDOWS2012-CA
        Validity
            Not Before: Mar 10 13:15:36 2017 GMT
            Not After : Mar 10 13:25:36 2022 GMT
        Subject: DC=com, DC=ibm, DC=uk, CN=uk-WINDOWS2012-CA
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)

...

with the add operation: -

/opt/ibm/HTTPServer/bin/gskcapicmd -cert -add -file /tmp/ad2012.cer -db /opt/ibm/HTTPServer/BPM/ssl/keystore.kdb -stashed

working as one would expect: -

/opt/ibm/HTTPServer/bin/gskcapicmd -cert -list -db /opt/ibm/HTTPServer/BPM/ssl/keystore.kdb -stashed

Certificates found
* default, - personal, ! trusted, # secret key
! CN=uk-WINDOWS2012-CA,DC=uk,DC=ibm,DC=com
- bpm857.novalocal
*- bpm857.uk.ibm.com


allowing me to import my CA-signed certificate: -

/opt/ibm/HTTPServer/bin/gskcapicmd -cert -receive -file /tmp/bpm857.uk.ibm.com_ihs.cer -db /opt/ibm/HTTPServer/BPM/ssl/keystore.kdb -pw passw0rd -default_cert yes

and validate same: -

/opt/ibm/HTTPServer/bin/gskcapicmd -cert -details -db /opt/ibm/HTTPServer/BPM/ssl/keystore.kdb -stashed -label bpm857.uk.ibm.com

Label : bpm857.uk.ibm.com
Key Size : 2048
Version : X509 V3
Serial : 5b00000005a6a06a78791e1454000000000005
Issuer : CN=uk-WINDOWS2012-CA,DC=uk,DC=ibm,DC=com
Subject : CN=bpm857,DC=uk,DC=ibm,DC=com
Not Before : March 10, 2017 7:03:41 PM GMT+00:00

Not After : March 10, 2018 7:13:41 PM GMT+00:00

Public Key
    30 82 01 22 30 0D 06 09 2A 86 48 86 F7 0D 01 01
    01 05 00 03 82 01 0F 00 30 82 01 0A 02 82 01 01
    00 B1 61 3C 39 8C 63 36 4A 05 FD 72 30 20 A1 91
    C7 AE C1 FD A1 CC 08 B1 31 99 A9 E3 4A 32 B5 6A
    65 76 04 63 AF 9E 50 1A 49 76 13 08 0D 6F 0E 2C
    6F 66 1F 39 91 67 2F C2 70 22 0E BD 75 20 19 A2
    74 14 00 01 0B 12 9C 78 48 7C 43 6B A0 6B 92 9D
    F0 98 9E A6 F2 6C 3D 18 5E 5E 37 15 14 88 32 D1
    CE 9A 01 82 69 08 3B D2 75 46 DC F6 E5 DF 2C E1
    6D 48 D8 C1 62 38 28 D4 1F 99 A9 E0 50 C1 F3 F5
    AB BD EA 51 15 96 06 53 35 18 50 F5 4E 01 02 C5
    7A 19 3F B4 D9 C9 30 F5 72 C3 E9 31 8D 2A ED 8A
    67 C0 33 D5 46 87 29 A5 E0 6B 1D F1 02 28 3C 3A
    71 8D 55 5B FB 87 F8 CF 9D D1 F0 4E C0 9F 02 4D
    2C 07 1C 4A 3E 6A 8E 87 8F 0B 41 7D BF 52 B3 CF
    66 EE 99 ED 37 7B C9 08 90 D0 6B 45 92 6A 8D 50
    3D 18 16 57 6A B0 8A CB 59 21 F6 15 1E 82 1D E8
    84 9D 86 53 6F E2 07 54 60 68 40 37 EB 26 81 26
    9B 02 03 01 00 01
Public Key Type : RSA (1.2.840.113549.1.1.1)
Fingerprint : SHA1 : 
    83 1F 52 73 69 C9 BB 0D 29 CD D9 E7 D7 67 E0 EE
    15 FC 42 91
Fingerprint : MD5 : 
    BF 92 1F 65 02 07 1E 19 52 AD B1 79 D8 40 76 99
Fingerprint : SHA256 : 
    F0 7D 76 9A C7 C9 2F BD 74 A4 91 75 20 DA 01 00
    07 8C 26 95 A6 8A 1F F2 B0 AB B5 8A 6B 53 2E 3B
Extensions
    subjectAlternativeName
        dNSName: bpm857.uk.ibm.com
    SubjectKeyIdentifier
      keyIdentifier:
    45 DD 67 36 0A 15 45 EA 25 34 A7 EE 66 E4 A7 DA
    C1 7A FF AA
    AuthorityKeyIdentifier
      keyIdentifier:
    7D 5C 74 A0 48 F9 2B 97 01 6F 7D 62 28 E0 21 5A
    6E 85 39 2F
      authorityIdentifier:
      authorityCertSerialNumber:
    CRLDistributionPoints
      fullname:
      uniformResourceID:      keyIdentifier: GSKASNObject: OBJECT(tag=22, class=0)
 value: -----BEGIN HEX-----
16 40 66 69 6C 65 3A 2F 2F 2F 2F 77 69 6E 64 6F     .@file:////windo
77 73 32 30 31 32 2E 75 6B 2E 69 62 6D 2E 63 6F     ws2012.uk.ibm.co
6D 2F 43 65 72 74 45 6E 72 6F 6C 6C 2F 75 6B 2D     m/CertEnroll/uk-
57 49 4E 44 4F 57 53 32 30 31 32 2D 43 41 2E 63     WINDOWS2012-CA.c
72 6C                                               rl
-----END HEX-----


    AuthorityInfoAccess
    PKIX_AD_CA_Issuer (1.3.6.1.5.5.7.48.2)
      accessLocation:uniformResourceID: file:////windows2012.uk.ibm.com/CertEnroll/windows2012.uk.ibm.com_uk-WINDOWS2012-CA.crt
     (1.3.6.1.4.1.311.20.2)
        Value
    1E 12 00 57 00 65 00 62 00 73 00 65 00 72 00 76
    00 65 00 72
    basicConstraints
        ca = false
        pathLen = 7FFF2617ABC0
        critical
Signature Algorithm : SHA256WithRSASignature (1.2.840.113549.1.1.11)
Value
    65 3C 5F 02 BA 62 F8 28 A7 23 44 A9 87 AE 3B 47
    63 0B 32 0E CA F6 E1 88 D3 B0 05 49 00 0E A8 17
    98 75 D9 A3 DE 0A 5C CA 12 B5 CF D3 D2 A3 D5 D2
    BD 8C 0C A3 66 B5 95 6E 1D EE C0 40 32 3E 15 C0


CloudFoundry Enablement

Shared by one of my team: -


Join us for a hands-on training workshop to learn about deploying and managing applications on Cloud Foundry. We will give an overview of Cloud Foundry and how it works, including specifics relating to services, buildpacks, and architecture. We will also look at how to effectively work with Cloud Foundry in your organization.

This training is targeted at people with little or no Cloud Foundry experience but some experience delivering web-based applications. If you're curious to learn how Cloud Foundry can help you focus on development and innovation, rather than infrastructure plumbing, this is the right course for you.


This course is designed to give its students a hands on experience of designing applications for Cloud Foundry. We will give an overview of Cloud Foundry and its tools from the point of view of an application developer and how to architect polyglot applications for deployment and scaling in the cloud.

This training is targeted at developers with little hands-on Cloud Foundry experience and those who have an interest in deploying innovative, microservice-based systems into the cloud.


Have you ever wondered how to deploy and operate a platform that's designed to deploy and operate applications? Join us and find out how with a hands-on training workshop. We'll teach you how to deploy and manage the Cloud Foundry platform as well as the stateful data services that power cloud-native applications. We'll provide an operational overview of Cloud Foundry and data services, and how these can be deployed with the cluster orchestration tool, "BOSH".

This training is targeted at people with little to no Cloud Foundry BOSH experience but who have some experience managing Linux-based systems. If you're curious to learn how BOSH can help you deploy and manage Cloud Foundry and other complex systems, this is the course for you.

At last, LDAP Test Query feature added to WebSphere Application Server

So this is something that I've wanted FOREVER …

I don't know precisely when it appeared, but I just realised that WebSphere Application Server (WAS) Network Deployment 8.5.5.11 ( 8.5.5 Fixpack 11 ) now includes the ability to test LDAP connectivity: -


Specifically, this: -


which returns: -

whereas if, for example, I use the wrong Bind Password, I get: -


SECJ7340E: Exception raised trying to connect to LDAP serverException Name=javax.naming.AuthenticationException Reason=[LDAP: error code 49 - 80090308: LdapErr: DSID-0C0903D0, comment: AcceptSecurityContext error, data 52e, v2580] ***** LdapSearch Input Parameters ***** hostName=windows2012.uk.ibm.com portNumber=389 sslEnabled=false baseDN=CN=Users,DC=uk,DC=ibm bindDN=CN=LDAPBindUser,CN=Users,DC=uk,DC=ibm,DC=com bindPwd=***** searchFilter=cn=BPMUser* searchLimit=20 referral=ignore searchScope=sub 

Even better, there's a Python command AdminTask.ldapSearch which makes Scripty McScriptface happy: -

print AdminTask.ldapSearch('[-hostname windows2012.uk.ibm.com -port 389 -baseDN CN=Users,DC=uk,DC=ibm,DC=com -bindDN CN=LDAPBindUser,CN=Users,DC=uk,DC=ibm,DC=com -bindPassword Qpassw0rd -sslEnabled false -referral ignore -searchFilter cn=BPMUsers -searchLimit 20 ]')

***** LdapSearch Input Parameters *****
hostName=windows2012.uk.ibm.com
portNumber=389
sslEnabled=false
baseDN=CN=Users,DC=uk,DC=ibm,DC=com
bindDN=CN=LDAPBindUser,CN=Users,DC=uk,DC=ibm,DC=com
bindPwd=*****
searchFilter=cn=BPMUsers
searchLimit=20
referral=ignore
searchScope=sub

***** LdapSearch Results *****

------ #1
cn=BPMUsers
name=BPMUsers
instanceType=4
groupType=-2147483646
objectSid=_)2R
sAMAccountType=268435456
member=CN=BPMUser2,CN=Users,DC=uk,DC=ibm,DC=com
member=CN=BPMUser1,CN=Users,DC=uk,DC=ibm,DC=com
uSNCreated=12729
uSNChanged=12752
objectClass=top
objectClass=group
distinguishedName=CN=BPMUsers,CN=Users,DC=uk,DC=ibm,DC=com
objectCategory=CN=Group,CN=Schema,CN=Configuration,DC=uk,DC=ibm,DC=com
sAMAccountName=BPMUsers
objectGUID=HLtU
whenCreated=20170310074140.0Z
whenChanged=20170310074309.0Z
dSCorePropagationData=16010101000000.0Z

which is GREAT.

Thanks IBM :-)

*UPDATE*

I've just found this: -


which says, in part: -

The fix for this APAR is currently targeted for inclusion in
fix packs 7.0.0.41, 8.0.0.12 and 8.5.5.8

so it's been there since 8.5.5.8 ;-)