Thursday 30 March 2017

DecisionCAMP 2017 - July 13-14 2017, Birkbeck University of London, London UK


Business Rules and Decision Management Technology

Best Practices, Standards, Real-World Business Cases, and Supporting

July 13-14 2017, Birkbeck University of London, London UK

DecisionCAMP is the latest incarnation of the series of popular events for Decision Management practitioners. It started in 2008 as October RulesFest, continued in 2009-2011 as RulesFest, became IntelliFest in 2012, and DecisionCAMP in 2013-2016. See the entire history of the events. DecisionCAMP-2017 is organized by DMCommunity.org and is hosted by the RuleML+RR: International Joint Conference on Rules and Reasoning.

DecisionCAMP-2017 will include presentations from leading decision management authorities, vendors, and practitioners. The event will explore the current state in Decision Management, the real-world use of the DMN standard, and solutions to various business problems using Decision Management  tools and capabilities. The event will include a special Open Discussion "What you Like and What you Do Not Like about DMN" and a QnA Panel "Real-world Business Decision Management: Vendor and Practitioner Perspectives".

There are many benefits to attending DecisionCAMP:

• Learn about new trends in Decision Management technologies, and how they can be used to address your business problems
• Share practical results on the use of various decision management technologies in business settings
• Exchange best practices for using DMN and decision management technologies.

Monday 27 March 2017

IBM WebSphere Liberty App Accelerator

I heard about this at InterConnect last week: -

Liberty app accelerator

Easily start building apps for WebSphere

Liberty, a Java EE application server

Configure WebSphere Liberty with a set of selected technologies. Then download the project as a zip file.




Sunday 26 March 2017

IBM BPM Advanced 8.5.7 - "Unable to establish a connection to the server" seen when attempting to access Health Management

I saw this: -


within the BPM Advanced 8.5.7 Process Admin console.

What was interesting was that nothing obvious was popping up within the WAS logs ( for the Node Agent, SupCluster, AppCluster etc. ).

This was the URL that I was using to access Process Admin: -


I'm not sure what made me think this, but I then realised that the hostname - bpm857.uk.ibm.com - was actually an alias to the real hostname - bpm857.novalocal - as the server is running on an openStack PaaS platform.

When I switched the URL to: -


the Health Management console burst into life: -


which is nice.

IBM BPM 8.5.7 - ADMN0022E seen when attempting to manage Deployment Environment

I saw this: -

Caused by: javax.management.JMRuntimeException: ADMN0022E: Access is
denied for the resolve operation on ConfigService MBean because of insufficient or empty credentials.

when attempting to stop a Deployment Environment using IBM BPM Advanced 8.5.7.

Specifically, I was logged into the WebSphere Application Server (WAS) Integrated Solutions Console, as an administrator with FULL WAS authority ( access to ALL roles ), and clicked Stop against a running Deployment Environment: -


This started occurring after I'd federated Active Directory (AD) via LDAP into my WAS cell.

The federation had exposed an issue whereby I had a duplicate account - wasadmin - one within the WAS file-based registry and one within AD.

Therefore, I'd created a new file-based account ( BPMAdmin ) and granted it all the required roles e.g. deployer, operator, configurator, monitor, administrator, adminsecuritymanager, auditor.

This was the ID with which I'd logged into the ISC.

After a quick spot of Googling, I found this: -


which made me realise that I'd NOT updated a specific Java2 Authentication (J2C) Alias - CellAdminAlias - via Security -> Global Security 


This is what I had: -



and I needed to updated the User ID to the new user - BPMAdmin - and password.

Once I did this, I was then able to manage the Deployment Environment as well as the overall WAS cell.

SoapUI hanging on macOS - Not any longer


I saw an issue with SoapUI 5.3.0 hanging on my Mac this morning.

The hang would occur as soon as I'd chosen NOT to send feedback to SmartBear and to allow Eclipse to access the internet …..

It wasn't immediately obvious why … so I jumped onto Google, and found this: -

which talked me through updating: -

/Applications/SoapUI-5.3.0.app/Contents/java/app/bin/soapui.sh

and: -

/Applications/SoapUI-5.3.0.app/Contents/vmoptions.txt

effectively moving: -

-Dsoapui.browser.disabled=true

from the script to the VMp[tions file.

Which did the trick ….

Which is nice :-)

#LifeIsGood 

Wednesday 15 March 2017

JVMCFRE003 bad major version; class=org/apache/derby/jdbc/EmbeddedConnectionPoolDataSource40, offset=6

I saw this today: -

[ERROR   ] An error has occurred in trying to access data source 'jdbc/ilogDataSource': Could not lookup datasource named 'jdbc/ilogDataSource'. Check that the data source exists on the application server or contact your administrator.
An error has occurred in trying to access data source 'jdbc/ilogDataSource': Could not lookup datasource named 'jdbc/ilogDataSource'. Check that the data source exists on the application server or contact your administrator.
[ERROR   ] CWWKE0701E: FrameworkEvent ERROR Bundle:com.ibm.ws.jdbc(id=110) org.osgi.framework.ServiceException: Exception in com.ibm.ws.resource.internal.ResourceFactoryTrackerData$1.getService()
at org.eclipse.osgi.internal.serviceregistry.ServiceFactoryUse.factoryGetService(ServiceFactoryUse.java:222)
at [internal classes]
at javax.naming.InitialContext.lookup(InitialContext.java:423)
at ilog.rules.teamserver.ejb.service.dao.IlrElementDAOFactory.getInstance(IlrElementDAOFactory.java:70)
at ilog.rules.teamserver.ejb.service.IlrSessionFacadeImpl.loginInitializationWithTxnPart1(IlrSessionFacadeImpl.java:414)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at ilog.rules.teamserver.transaction.IlrTransactionalInvocationHandler.invoke(IlrTransactionalInvocationHandler.java:39)
at com.sun.proxy.$Proxy36.loginInitializationWithTxnPart1(Unknown Source)
at ilog.rules.teamserver.ejb.service.IlrLocalSessionImpl.loginInitializationWithTxnPart1(IlrLocalSessionImpl.java:64)
at ilog.rules.teamserver.ejb.service.IlrSessionFacadeImpl.login(IlrSessionFacadeImpl.java:393)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at ilog.rules.teamserver.transaction.IlrTransactionalInvocationHandler.invoke(IlrTransactionalInvocationHandler.java:39)
at com.sun.proxy.$Proxy36.login(Unknown Source)
at ilog.rules.teamserver.ejb.service.IlrLocalSessionImpl.login(IlrLocalSessionImpl.java:58)
at ilog.rules.teamserver.ejb.service.IlrLocalSessionFactory.makeSession(IlrLocalSessionFactory.java:68)
at ilog.rules.teamserver.ejb.service.IlrLocalSessionFactory.connect(IlrLocalSessionFactory.java:49)
at ilog.rules.teamserver.ejb.service.IlrLocalSessionFactory.connect(IlrLocalSessionFactory.java:59)
at com.ibm.rules.decisioncenter.web.core.UserContext.connect(UserContext.java:48)
at com.ibm.rules.decisioncenter.web.core.ApplicationInterceptor.preHandle(ApplicationInterceptor.java:117)
at org.springframework.web.servlet.HandlerExecutionChain.applyPreHandle(HandlerExecutionChain.java:134)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:913)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:851)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:953)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:844)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:575)
at [internal classes]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:829)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at [internal classes]
at com.ibm.rules.decisioncenter.web.core.filters.SecurityCheckPointFilter.doFilter(SecurityCheckPointFilter.java:95)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:207)
at [internal classes]
at com.ibm.rules.decisioncenter.web.core.filters.RemoteSessionFilter.doFilterInternal(RemoteSessionFilter.java:79)
at com.ibm.rules.decisioncenter.web.core.filters.RemoteSessionFilter.doFilter(RemoteSessionFilter.java:59)
at com.ibm.rules.decisioncenter.web.core.filters.SessionFilter.access$001(SessionFilter.java:32)
at com.ibm.rules.decisioncenter.web.core.filters.SessionFilter$1.doFilter(SessionFilter.java:73)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:106)
at com.ibm.rules.decisioncenter.web.core.filters.SessionFilter.doFilter(SessionFilter.java:70)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:207)
at [internal classes]
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:106)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:207)
at [internal classes]
at com.ibm.rules.decisioncenter.web.core.filters.HttpPUTRequestFilter.doFilterInternal(HttpPUTRequestFilter.java:65)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:106)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:207)
at [internal classes]
Caused by: java.lang.UnsupportedClassVersionError: JVMCFRE003 bad major version; class=org/apache/derby/jdbc/EmbeddedConnectionPoolDataSource40, offset=6
at java.lang.ClassLoader.defineClassImpl(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:331)
at com.ibm.ws.classloading.internal.AppClassLoader.definePackageAndClass(AppClassLoader.java:327)
... 54 more

Referencing back to a previous post: -


reminded me to do this: -

/opt/ibm/WebSphere/Liberty/usr/shared/resources/derby/derby.jar!/org/apache/derby/jdbc/EmbeddedConnectionPoolDataSource40.class

which resulted in: -

  Last modified 15-Oct-2016; size 334 bytes
  MD5 checksum 9e105007b71e5d0460499ca4564efd1b
public class org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource40 extends org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource implements javax.sql.ConnectionPoolDataSource
  Deprecated: true
  minor version: 0
  major version: 52
  flags: ACC_PUBLIC, ACC_SUPER
Constant pool:
   #1 = Methodref          #3.#14         //  org/apache/derby/jdbc/EmbeddedConnectionPoolDataSource."<init>":()V
   #2 = Class              #15            //  org/apache/derby/jdbc/EmbeddedConnectionPoolDataSource40
   #3 = Class              #16            //  org/apache/derby/jdbc/EmbeddedConnectionPoolDataSource
   #4 = Class              #17            //  javax/sql/ConnectionPoolDataSource
   #5 = Utf8               serialVersionUID
   #6 = Utf8               J
   #7 = Utf8               ConstantValue
   #8 = Long               -4368824293743156916l
  #10 = Utf8               <init>
  #11 = Utf8               ()V
  #12 = Utf8               Code
  #13 = Utf8               Deprecated
  #14 = NameAndType        #10:#11        //  "<init>":()V
  #15 = Utf8               org/apache/derby/jdbc/EmbeddedConnectionPoolDataSource40
  #16 = Utf8               org/apache/derby/jdbc/EmbeddedConnectionPoolDataSource
  #17 = Utf8               javax/sql/ConnectionPoolDataSource
{
  public org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource40();
    flags: ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0       
         1: invokespecial #1                  // Method org/apache/derby/jdbc/EmbeddedConnectionPoolDataSource."<init>":()V
         4: return        
}

The previous post took me back to this: -



I checked my JRE version: -

/opt/ibm/WebSphere/Liberty/java/java_1.7.1_64/bin/java -version

java version "1.7.0"
Java(TM) SE Runtime Environment (build pxa6470_27sr3fp40ifix-20160711_01(SR3 FP40+IX90174))
IBM J9 VM (build 2.7, JRE 1.7.0 Linux amd64-64 Compressed References 20160406_298393 (JIT enabled, AOT enabled)
J9VM - R27_Java727_SR3_20160406_0942_B298393
JIT  - tr.r13.java_20160328_114186
GC   - R27_Java727_SR3_20160406_0942_B298393_CMPRSS
J9CL - 20160406_298393)
JCL - 20160421_01 based on Oracle jdk7u101-b14


which confirmed my feelings :-)

I was using a version of Derby DB that was compiled using Java 8 whereas I'm using Java 7.

This was what I had: -


<snip>
The 10.13 release family supports the following Java and JDBC versions:

• Java SE 8 with JDBC 4.2
</snip>

so I switched to: -


<snip>
Support for Java SE 6 and Java SE 7 is being sunsetted. The 10.13 release family will not support those platforms. The 10.12 release family supports the following Java and JDBC versions:

• Java SE 6 and higher with JDBC 4.0, 4.1, and 4.2.
• Java SE 8 compact profile 2.
</snip>

Having downloaded the .ZIP, and expanded it to /tmp I grabbed the .JAR file: -

cd /opt/ibm/WebSphere/Liberty/usr/shared/resources/derby
cp /tmp/db-derby-10.12.1.1-bin/lib/derby.jar .

Job done :-)



Monday 13 March 2017

WebSphere Application Server - DNS, you can't fool it - or can you ?

I saw this: -

[3/12/17 19:55:21:158 UTC] 00000001 LogAdapter    E   DCSV9403E: Received an illegal configuration argument. Parameter MulticastInterface, value: 9.20.65.171. Exception is java.lang.Exception: Network I
nterface 9.20.65.171 was not found in local machine network interface list. Make sure that the NetworkInterface property is properly configured!
at com.ibm.rmm.mtl.transmitter.Config.<init>(Config.java:238)


while attempting to start a WebSphere Application Serve (WAS) Deployment Manager ( as part of an IBM BPM Advanced build ).

I wanted to drill into the issue, and see what hostname the offending IP address ( 9.20.65.171 ) was aliased to.

I tried/failed to use dig as it wasn't installed on my RHEL 7.3 VM.

So I installed bind-utils: -

yum install -y bind-utils

and then used dig as follows: -

dig 9.20.65.171

; <<>> DiG 9.9.4-RedHat-9.9.4-38.el7_3.2 <<>> 9.20.65.171
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 39457
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;9.20.65.171. IN A

;; AUTHORITY SECTION:
. 10800 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2017031201 1800 900 604800 86400

;; Query time: 20 msec
;; SERVER: 9.20.136.11#53(9.20.136.11)
;; WHEN: Sun Mar 12 21:07:38 UTC 2017
;; MSG SIZE  rcvd: 115


and host : -

host 9.20.65.171

171.65.20.9.in-addr.arpa domain name pointer hyc1-vm2417.hursley.ibm.com.

I also checked /etc/hosts which contained: -

# Your system has configured 'manage_etc_hosts' as True.
# As a result, if you wish for changes to this file to persist
# then you will need to either
# a.) make changes to the master file in /etc/cloud/templates/hosts.redhat.tmpl
# b.) change or remove the value of 'manage_etc_hosts' in
#     /etc/cloud/cloud.cfg or cloud-config from user-data

# The following lines are desirable for IPv4 capable hosts
127.0.0.1 bpm857.novalocal bpm857 bpm857.uk.ibm.com
127.0.0.1 localhost.localdomain localhost
127.0.0.1 localhost4.localdomain4 localhost4

# The following lines are desirable for IPv6 capable hosts
::1 bpm857.novalocal bpm857
::1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6

9.20.65.75 oracle.uk.ibm.com oracle
9.20.65.140 windows2012.uk.ibm.com windows2012
9.20.65.171 bpm857.uk.ibm.com bpm857

I revisited an old post of mine: -


and resurrected an old Java class: -

hostStuff.java

import java.net.InetAddress;
import java.net.UnknownHostException;

public class hostStuff
{
public static void main(String[] args)
{

try
{
InetAddress address = InetAddress.getLocalHost();
System.out.println("My IP address ( via InetAddress.getLocalHost() ) is " + address.toString());
System.out.println("My hostname ( via InetAddress.getHostName() ) is " + address.getHostName());
System.out.println("My hostname ( via InetAddress.getCanonicalHostname() ) is  " + address.getCanonicalHostName());
}
catch (UnknownHostException e)
{
       System.out.println("I'm sorry. I don't know my own name.");
}
}
}


which, when run, returned: -

My IP address ( via InetAddress.getLocalHost() ) is bpm857.uk.ibm.com/127.0.0.1
My hostname ( via InetAddress.getHostName() ) is bpm857.uk.ibm.com
My hostname ( via InetAddress.getCanonicalHostname() ) is  bpm857.novalocal

I suspect that WAS is executing InetAddress.getCanonicalHostname() under the covers, which is the real hostname of the box - bpm857.novalocal - as dictated by OpenStack, whereas I'm trying to spoof WAS to use a different name.

So I "hacked" /etc/hosts : -

# Your system has configured 'manage_etc_hosts' as True.
# As a result, if you wish for changes to this file to persist
# then you will need to either
# a.) make changes to the master file in /etc/cloud/templates/hosts.redhat.tmpl
# b.) change or remove the value of 'manage_etc_hosts' in
#     /etc/cloud/cloud.cfg or cloud-config from user-data

# The following lines are desirable for IPv4 capable hosts
127.0.0.1 bpm857.uk.ibm.com bpm857
127.0.0.1 localhost.localdomain localhost
127.0.0.1 localhost4.localdomain4 localhost4

# The following lines are desirable for IPv6 capable hosts
::1 bpm857.novalocal bpm857
::1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6

9.20.65.75 oracle.uk.ibm.com oracle
9.20.65.140 windows2012.uk.ibm.com windows2012
9.20.65.171 bpm857.uk.ibm.com bpm857


effectively removing reference to bpm857.novalocal which my class confirmed: -

My IP address ( via InetAddress.getLocalHost() ) is bpm857.uk.ibm.com/127.0.0.1
My hostname ( via InetAddress.getHostName() ) is bpm857.uk.ibm.com
My hostname ( via InetAddress.getCanonicalHostname() ) is  bpm857.uk.ibm.com

I tried to start the Deployment Manager, but that again failed with the same exception as before.

I further "hacked" /etc/hosts : -

# Your system has configured 'manage_etc_hosts' as True.
# As a result, if you wish for changes to this file to persist
# then you will need to either
# a.) make changes to the master file in /etc/cloud/templates/hosts.redhat.tmpl
# b.) change or remove the value of 'manage_etc_hosts' in
#     /etc/cloud/cloud.cfg or cloud-config from user-data

# The following lines are desirable for IPv4 capable hosts
127.0.0.1 localhost.localdomain localhost
127.0.0.1 localhost4.localdomain4 localhost4

# The following lines are desirable for IPv6 capable hosts
::1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6

9.20.65.75 oracle.uk.ibm.com oracle
9.20.65.140 windows2012.uk.ibm.com windows2012
9.20.65.171 bpm857.uk.ibm.com bpm857


but to no avail.

In the end, I bowed to the inevitable and stuck with the host / domain that OpenStack was allocating, especially given that my VM is going to be connecting to other VMs on the same OpenStack box, as well as to the outside world.

Therefore, I reverted my WAS configuration to use the bpm857.novalocal host/domain name internally, and continued to reference it externally as bpm857.uk.ibm.com.

Thankfully, when I rebooted the VM, the /etc/hosts file was reverted back to its original state, 

Now, when I run my hostStuff class, I get: -

My IP address ( via InetAddress.getLocalHost() ) is bpm857.novalocal/127.0.0.1
My hostname ( via InetAddress.getHostName() ) is bpm857.novalocal
My hostname ( via InetAddress.getCanonicalHostname() ) is  bpm857.novalocal


and I've reverted my response file accordingly: -

cat /tmp/ResponseFiles/Advanced-PS-ThreeClusters-Oracle.properties |grep -i hostname

bpm.de.psProcessCenterHostname=
bpm.dmgr.hostname=bpm857.novalocal
bpm.de.node.1.hostname=bpm857.novalocal
bpm.de.db.1.hostname=oracle.novalocal
bpm.de.db.2.hostname=oracle.novalocal
bpm.de.db.3.hostname=oracle.novalocal
bpm.de.db.4.hostname=oracle.novalocal


So we're back in the game ….

The moral of the story ? Ensure that you use host/domain names that are valid for your environment, networking with your network folks, as required.

Also, never use localhost for WAS hostnames :-)

Sunday 12 March 2017

Weirdness - The database is not consistent with the BPM Product version

So we saw this issue last week: -

[3/9/17 7:06:35:804 UTC] 00000001 WsServerImpl  E   WSVR0009E: Error occurred during startup
com.ibm.ws.exception.RuntimeError: The database [jndi = jdbc/PerformanceDB] version [8.5.7.201612] is not consistent with the BPM Product version [8.5.7.0], please ensure the database is updated successfully before starting server.


during the build of a new IBM Business Process Manager (BPM) Advanced 8.5.7 Deployment Environment with an Oracle 12c database.

The problem appeared to be an inconsistency between database tables within Oracle and the expectations of the BPM product, specifically the LSW_SYSTEM_SCHEMA tables, of which there are two: -

PSUSER.LSW_SYSTEM_SCHEMA
PDWUSER.LSW_SYSTEM_SCHEMA

So, here's the thing …

These tables are defined when one first builds the BPM Deployment Environment using BPMConfig.sh: -

/opt/ibm/WebSphere/AppServer/bin/BPMConfig.sh -create -de /mnt/ResponseFiles/ Advanced-PS-ThreeClusters-Oracle.properties

specifically, here: -

/opt/ibm/WebSphereProfiles/Dmgr01/dbscripts/PSCell1.De1/Oracle/orcl/psuser/createSchema_Advanced.sql: psuser.LSW_SYSTEM_SCHEMA WHERE PROPNAME = 'DatabaseSchemaVersion' ; 
/opt/ibm/WebSphereProfiles/Dmgr01/dbscripts/PSCell1.De1/Oracle/orcl/psuser/createSchema_Advanced.sql: VALUES ('DatabaseSchemaVersion',

and: -

/opt/ibm/WebSphereProfiles/Dmgr01/dbscripts/PSCell1.De1/Oracle/orcl/pdwuser/createSchema_Advanced.sql: pdwuser.LSW_SYSTEM_SCHEMA WHERE PROPNAME = 'DatabaseSchemaVersion' ; 
/opt/ibm/WebSphereProfiles/Dmgr01/dbscripts/PSCell1.De1/Oracle/orcl/pdwuser/createSchema_Advanced.sql: VALUES ('DatabaseSchemaVersion',

thanks to fgrep : -

fgrep -R DatabaseSchemaVersion /opt/ibm/WebSphereProfiles/Dmgr01/dbscripts/

Note that the exception related to the Performance Data Warehouse (PDW) database/schema, because we were starting the SupCluster, within which the PDW applications run.

When we saw this issue last week, the problem was that the tables in Oracle contained the value: -

8.5.7

( namely three columns / digits ), as evidenced: -

select * from PSUSER.LSW_SYSTEM_SCHEMA;

PROPNAME PROPVALUE
DatabaseSchemaVersion 8.5.7

select * from PDWUSER.LSW_SYSTEM_SCHEMA;

PROPNAME PROPVALUE
DatabaseSchemaVersion 8.5.7


whereas BPM appeared to be expected four columns / digits.

To prove / circumvent the issue, our friendly Oracle DBA deleted the current row: -

delete from PDWUSER.LSW_SYSTEM_SCHEMA where PROPNAME='DatabaseSchemaVersion';

and inserted a new row: -

insert into PDWUSER.LSW_SYSTEM_SCHEMA(PROPNAME,PROPVALUE) VALUES('DatabaseSchemaVersion','8.5.7.0');

and validated the change: -

select PROPNAME,PROPVALUE from PDWUSER.LSW_SYSTEM_SCHEMA where PROPNAME='DatabaseSchemaVersion';

PROPNAME PROPVALUE
DatabaseSchemaVersion 8.5.7.0


Once we did this, the SupCluster started without exception.

The strange thing ?

I can't recreate the same problem :-(

I'm using the same software: -

/opt/ibm/InstallationManager/eclipse/tools/imcl listInstalledPackages

com.ibm.cic.agent_1.8.6000.20161118_1611
com.ibm.bpm.ADV.v85_8.5.7000.20160301_1551
com.ibm.websphere.ND.v85_8.5.5011.20161206_1434
com.ibm.websphere.IHS.v85_8.5.5011.20161206_1434
com.ibm.websphere.PLG.v85_8.5.5011.20161206_1434

and followed the same installation process; specifically installing BPM Advanced 8.5.0.0 and patching to 8.5.7.0 at the same time ( plus also adding the Q1 2016 cumulative fix pack - CF2016.03

Specifically, this is what I see in the tables created by BPConfig.sh : -

/opt/ibm/WebSphereProfiles/Dmgr01/dbscripts/PSCell1.De1/Oracle/orcl/psuser/createSchema_Advanced.sql

DECLARE
v_table_count NUMBER;
  BEGIN
   SELECT COUNT(*) INTO v_table_count FROM
    psuser.LSW_SYSTEM_SCHEMA WHERE PROPNAME = 'DatabaseSchemaVersion' ;
     IF (v_table_count = 0) THEN
      INSERT INTO
    psuser.LSW_SYSTEM_SCHEMA("PROPNAME",
 "PROPVALUE")
VALUES ('DatabaseSchemaVersion',
 '8.5.7') ;
END IF ;
END ;


/opt/ibm/WebSphereProfiles/Dmgr01/dbscripts/PSCell1.De1/Oracle/orcl/pdwuser/createSchema_Advanced.sql

DECLARE
v_table_count NUMBER;
  BEGIN
   SELECT COUNT(*) INTO v_table_count FROM
    pdwuser.LSW_SYSTEM_SCHEMA WHERE PROPNAME = 'DatabaseSchemaVersion' ;
     IF (v_table_count = 0) THEN
      INSERT INTO
    pdwuser.LSW_SYSTEM_SCHEMA("PROPNAME",
 "PROPVALUE")
VALUES ('DatabaseSchemaVersion',
 '8.5.7') ;
END IF ;
END ;


So, as far as I can establish, we shouldn't have needed to apply the modification in Oracle :-(

Whilst I'm not 100% sure what went wrong, I do, at least, have a mitigation if it occurs again.

For the client in question, we're going for a clean installation of BPM Advanced 8.5.7.0 plus the Q4 2016 cumulative fix pack - CF2016.12.

For the record, I had seen a similar problem before: -


That time around, I'd NOT run the appropriate createSchema_Advanced script.

Also, the IBM Knowledge Centre reports: -


• Make sure that there is one row in the LSW_SYSTEM_SCHEMA table of the Process Server or Performance Data Warehouse database with the name DatabaseSchemaVersion. Make sure that the value is the same as the value of the IBM BPM version of your current profile.
• If the row does not exist, insert a new row with the correct value. If the row does not exist, insert a new row with the correct value. The value must be a 3-digit version number, for example, 8.5.6.
• If the row exists but the value is not the same as your current profile, you might not have finished migration. Make sure to run the database upgrade step to upgrade your database.
• To disable version validation, set a WebSphere variable in the cell scope. Log in to the administrative console and go to Environment > WebSphere variables. Under cell scope, create a variable named SKIPSTDDBVERSIONCHECK and set its value to true. Restart the servers. The version validation is skipped during server start.

which is pretty spot-on.

Saturday 11 March 2017

Scripting in Python and Jython with added OS commands

I was writing a generic ( use anywhere ) script to add a BPM URL to my IBM BPM Advanced 8.5.7 environment.

For me, all of the components ( IBM HTTP Server, WebSphere Plugin and WebSphere Application Server / BPM ) are on the same VM.

For the record, I'm running the VM on OpenStack.

Therefore, I wanted a script that would get the hostname of the VM on which IHS/WAS is running.

This served as source: -


and this was my test script: -

 foo.jy 
import socket

hostname=socket.gethostname()
print("Hostname is " + hostname)


which I tested using Python: -

python foo.jy 

Hostname is bpm857.novalocal

and Jython: -

/opt/ibm/WebSphereProfiles/Dmgr01/bin/wsadmin.sh -lang jython -conntype none -f foo.jy 

WASX7357I: By request, this scripting client is not connected to any server process. Certain configuration and application operations will be available in local mode.
Hostname is bpm857.novalocal


This is the pukka script: -

configureBPMURL.jy

import socket

# Set variables

cellID=AdminControl.getCell()
dePath='/Cell:'+ cellID+'/BPMCellConfigExtension:/BPMDeploymentEnvironment:/'
de=AdminConfig.getid(dePath)
hostname=socket.gethostname()
print("Hostname is " + hostname)

# List existing BPM URLs

bpmurlsid=AdminConfig.getid(dePath+'BPMURLS:/')
bpmurllist=AdminUtilities.convertToList(AdminConfig.list("BPMURL", bpmurlsid))
for item in bpmurllist :
 print AdminConfig.show(item)

# Remove existing BPM URLs

for i in AdminConfig.list('BPMVirtualHostInfo').split():
 AdminConfig.remove(i)

# Create new Virtual Host

webserver_vh = AdminConfig.create('BPMVirtualHostInfo',de,[['name','webserver_vh'],['transportProtocol','https'],['hostname',hostname],['port','8443']],'virtualHosts')
AdminConfig.modify(de,[['defaultVH',webserver_vh]])

# Save and Sync

AdminConfig.save()
AdminNodeManagement.syncActiveNodes()

# Validate

print AdminConfig.show(AdminConfig.list('BPMVirtualHostInfo'))


which worked as follows: -

/opt/ibm/WebSphereProfiles/Dmgr01/bin/wsadmin.sh -lang jython -f configureBPMURL.jy 

Realm/Cell Name: <default>
Username: wasadmin
Password:         
 WASX7209I: Connected to process "dmgr" on node Dmgr using SOAP connector;  The type of process is: DeploymentManager
Hostname is bpm857.novalocal
[scenario EXTERNAL_CLIENT]
[strategies "WCCMConfigStrategy, HttpProtocolHostStrategy"]
[scenario INTERNAL_CLIENT]
[scenario RELATIVE]
[strategies RelativeUrlStrategy]
---------------------------------------------------------------
 AdminNodeManagement:        Synchronize the active nodes
 Usage: AdminNodeManagement.syncActiveNodes()
 Return: If the command is successfully invoked, a value of 1 is returned. 
---------------------------------------------------------------
 
 
Node1
[hostname bpm857.novalocal]
[name webserver_vh]
[port 8443]
[transportProtocol https]


which is nice

This was also useful: -


although the socket.gethostname() was more useful.

Friday 10 March 2017

CTGSK3046W - IBM HTTP Server - Certificates and Permissions

I saw this: -

CTGSK3046W The key file "/tmp/ad2012.cer" could not be imported.

whilst trying to add a CA Signer certificate to a keystore using IBM HTTP Server: -

/opt/ibm/HTTPServer/bin/gskcapicmd -cert -add -file /tmp/ad2012.cer -db /opt/ibm/HTTPServer/BPM/ssl/keystore.kdb -stashed

Having checked and double-checked my command, I then tried to use openSSL to validate the certificate: -

openssl x509 -inform der -in /tmp/ad2012.cer -text -noout

Error opening Certificate /tmp/ad2012.cer
140581419276192:error:0200100D:system library:fopen:Permission denied:bss_file.c:398:fopen('/tmp/ad2012.cer','r')
140581419276192:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:400:
unable to load certificate


which was more revealing.

A quick check: -

ls -al /tmp/ad2012.cer 

-rw------- 1 cloudusr cloudusr 915 Mar 10 19:26 /tmp/ad2012.cer

confirmed that it was likely a permissions issue.

Once I fixed the permissions ( as root ): -

chmod -R 777 /tmp

we're back in the game: -

openssl x509 -inform der -in /tmp/ad2012.cer -text -noout

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            1f:92:ac:6d:1a:57:e9:b4:43:d3:81:64:ff:9e:93:d6
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: DC=com, DC=ibm, DC=uk, CN=uk-WINDOWS2012-CA
        Validity
            Not Before: Mar 10 13:15:36 2017 GMT
            Not After : Mar 10 13:25:36 2022 GMT
        Subject: DC=com, DC=ibm, DC=uk, CN=uk-WINDOWS2012-CA
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)

...

with the add operation: -

/opt/ibm/HTTPServer/bin/gskcapicmd -cert -add -file /tmp/ad2012.cer -db /opt/ibm/HTTPServer/BPM/ssl/keystore.kdb -stashed

working as one would expect: -

/opt/ibm/HTTPServer/bin/gskcapicmd -cert -list -db /opt/ibm/HTTPServer/BPM/ssl/keystore.kdb -stashed

Certificates found
* default, - personal, ! trusted, # secret key
! CN=uk-WINDOWS2012-CA,DC=uk,DC=ibm,DC=com
- bpm857.novalocal
*- bpm857.uk.ibm.com


allowing me to import my CA-signed certificate: -

/opt/ibm/HTTPServer/bin/gskcapicmd -cert -receive -file /tmp/bpm857.uk.ibm.com_ihs.cer -db /opt/ibm/HTTPServer/BPM/ssl/keystore.kdb -pw passw0rd -default_cert yes

and validate same: -

/opt/ibm/HTTPServer/bin/gskcapicmd -cert -details -db /opt/ibm/HTTPServer/BPM/ssl/keystore.kdb -stashed -label bpm857.uk.ibm.com

Label : bpm857.uk.ibm.com
Key Size : 2048
Version : X509 V3
Serial : 5b00000005a6a06a78791e1454000000000005
Issuer : CN=uk-WINDOWS2012-CA,DC=uk,DC=ibm,DC=com
Subject : CN=bpm857,DC=uk,DC=ibm,DC=com
Not Before : March 10, 2017 7:03:41 PM GMT+00:00

Not After : March 10, 2018 7:13:41 PM GMT+00:00

Public Key
    30 82 01 22 30 0D 06 09 2A 86 48 86 F7 0D 01 01
    01 05 00 03 82 01 0F 00 30 82 01 0A 02 82 01 01
    00 B1 61 3C 39 8C 63 36 4A 05 FD 72 30 20 A1 91
    C7 AE C1 FD A1 CC 08 B1 31 99 A9 E3 4A 32 B5 6A
    65 76 04 63 AF 9E 50 1A 49 76 13 08 0D 6F 0E 2C
    6F 66 1F 39 91 67 2F C2 70 22 0E BD 75 20 19 A2
    74 14 00 01 0B 12 9C 78 48 7C 43 6B A0 6B 92 9D
    F0 98 9E A6 F2 6C 3D 18 5E 5E 37 15 14 88 32 D1
    CE 9A 01 82 69 08 3B D2 75 46 DC F6 E5 DF 2C E1
    6D 48 D8 C1 62 38 28 D4 1F 99 A9 E0 50 C1 F3 F5
    AB BD EA 51 15 96 06 53 35 18 50 F5 4E 01 02 C5
    7A 19 3F B4 D9 C9 30 F5 72 C3 E9 31 8D 2A ED 8A
    67 C0 33 D5 46 87 29 A5 E0 6B 1D F1 02 28 3C 3A
    71 8D 55 5B FB 87 F8 CF 9D D1 F0 4E C0 9F 02 4D
    2C 07 1C 4A 3E 6A 8E 87 8F 0B 41 7D BF 52 B3 CF
    66 EE 99 ED 37 7B C9 08 90 D0 6B 45 92 6A 8D 50
    3D 18 16 57 6A B0 8A CB 59 21 F6 15 1E 82 1D E8
    84 9D 86 53 6F E2 07 54 60 68 40 37 EB 26 81 26
    9B 02 03 01 00 01
Public Key Type : RSA (1.2.840.113549.1.1.1)
Fingerprint : SHA1 : 
    83 1F 52 73 69 C9 BB 0D 29 CD D9 E7 D7 67 E0 EE
    15 FC 42 91
Fingerprint : MD5 : 
    BF 92 1F 65 02 07 1E 19 52 AD B1 79 D8 40 76 99
Fingerprint : SHA256 : 
    F0 7D 76 9A C7 C9 2F BD 74 A4 91 75 20 DA 01 00
    07 8C 26 95 A6 8A 1F F2 B0 AB B5 8A 6B 53 2E 3B
Extensions
    subjectAlternativeName
        dNSName: bpm857.uk.ibm.com
    SubjectKeyIdentifier
      keyIdentifier:
    45 DD 67 36 0A 15 45 EA 25 34 A7 EE 66 E4 A7 DA
    C1 7A FF AA
    AuthorityKeyIdentifier
      keyIdentifier:
    7D 5C 74 A0 48 F9 2B 97 01 6F 7D 62 28 E0 21 5A
    6E 85 39 2F
      authorityIdentifier:
      authorityCertSerialNumber:
    CRLDistributionPoints
      fullname:
      uniformResourceID:      keyIdentifier: GSKASNObject: OBJECT(tag=22, class=0)
 value: -----BEGIN HEX-----
16 40 66 69 6C 65 3A 2F 2F 2F 2F 77 69 6E 64 6F     .@file:////windo
77 73 32 30 31 32 2E 75 6B 2E 69 62 6D 2E 63 6F     ws2012.uk.ibm.co
6D 2F 43 65 72 74 45 6E 72 6F 6C 6C 2F 75 6B 2D     m/CertEnroll/uk-
57 49 4E 44 4F 57 53 32 30 31 32 2D 43 41 2E 63     WINDOWS2012-CA.c
72 6C                                               rl
-----END HEX-----


    AuthorityInfoAccess
    PKIX_AD_CA_Issuer (1.3.6.1.5.5.7.48.2)
      accessLocation:uniformResourceID: file:////windows2012.uk.ibm.com/CertEnroll/windows2012.uk.ibm.com_uk-WINDOWS2012-CA.crt
     (1.3.6.1.4.1.311.20.2)
        Value
    1E 12 00 57 00 65 00 62 00 73 00 65 00 72 00 76
    00 65 00 72
    basicConstraints
        ca = false
        pathLen = 7FFF2617ABC0
        critical
Signature Algorithm : SHA256WithRSASignature (1.2.840.113549.1.1.11)
Value
    65 3C 5F 02 BA 62 F8 28 A7 23 44 A9 87 AE 3B 47
    63 0B 32 0E CA F6 E1 88 D3 B0 05 49 00 0E A8 17
    98 75 D9 A3 DE 0A 5C CA 12 B5 CF D3 D2 A3 D5 D2
    BD 8C 0C A3 66 B5 95 6E 1D EE C0 40 32 3E 15 C0


CloudFoundry Enablement

Shared by one of my team: -


Join us for a hands-on training workshop to learn about deploying and managing applications on Cloud Foundry. We will give an overview of Cloud Foundry and how it works, including specifics relating to services, buildpacks, and architecture. We will also look at how to effectively work with Cloud Foundry in your organization.

This training is targeted at people with little or no Cloud Foundry experience but some experience delivering web-based applications. If you're curious to learn how Cloud Foundry can help you focus on development and innovation, rather than infrastructure plumbing, this is the right course for you.


This course is designed to give its students a hands on experience of designing applications for Cloud Foundry. We will give an overview of Cloud Foundry and its tools from the point of view of an application developer and how to architect polyglot applications for deployment and scaling in the cloud.

This training is targeted at developers with little hands-on Cloud Foundry experience and those who have an interest in deploying innovative, microservice-based systems into the cloud.


Have you ever wondered how to deploy and operate a platform that's designed to deploy and operate applications? Join us and find out how with a hands-on training workshop. We'll teach you how to deploy and manage the Cloud Foundry platform as well as the stateful data services that power cloud-native applications. We'll provide an operational overview of Cloud Foundry and data services, and how these can be deployed with the cluster orchestration tool, "BOSH".

This training is targeted at people with little to no Cloud Foundry BOSH experience but who have some experience managing Linux-based systems. If you're curious to learn how BOSH can help you deploy and manage Cloud Foundry and other complex systems, this is the course for you.

At last, LDAP Test Query feature added to WebSphere Application Server

So this is something that I've wanted FOREVER …

I don't know precisely when it appeared, but I just realised that WebSphere Application Server (WAS) Network Deployment 8.5.5.11 ( 8.5.5 Fixpack 11 ) now includes the ability to test LDAP connectivity: -


Specifically, this: -


which returns: -

whereas if, for example, I use the wrong Bind Password, I get: -


SECJ7340E: Exception raised trying to connect to LDAP serverException Name=javax.naming.AuthenticationException Reason=[LDAP: error code 49 - 80090308: LdapErr: DSID-0C0903D0, comment: AcceptSecurityContext error, data 52e, v2580] ***** LdapSearch Input Parameters ***** hostName=windows2012.uk.ibm.com portNumber=389 sslEnabled=false baseDN=CN=Users,DC=uk,DC=ibm bindDN=CN=LDAPBindUser,CN=Users,DC=uk,DC=ibm,DC=com bindPwd=***** searchFilter=cn=BPMUser* searchLimit=20 referral=ignore searchScope=sub 

Even better, there's a Python command AdminTask.ldapSearch which makes Scripty McScriptface happy: -

print AdminTask.ldapSearch('[-hostname windows2012.uk.ibm.com -port 389 -baseDN CN=Users,DC=uk,DC=ibm,DC=com -bindDN CN=LDAPBindUser,CN=Users,DC=uk,DC=ibm,DC=com -bindPassword Qpassw0rd -sslEnabled false -referral ignore -searchFilter cn=BPMUsers -searchLimit 20 ]')

***** LdapSearch Input Parameters *****
hostName=windows2012.uk.ibm.com
portNumber=389
sslEnabled=false
baseDN=CN=Users,DC=uk,DC=ibm,DC=com
bindDN=CN=LDAPBindUser,CN=Users,DC=uk,DC=ibm,DC=com
bindPwd=*****
searchFilter=cn=BPMUsers
searchLimit=20
referral=ignore
searchScope=sub

***** LdapSearch Results *****

------ #1
cn=BPMUsers
name=BPMUsers
instanceType=4
groupType=-2147483646
objectSid=_)2R
sAMAccountType=268435456
member=CN=BPMUser2,CN=Users,DC=uk,DC=ibm,DC=com
member=CN=BPMUser1,CN=Users,DC=uk,DC=ibm,DC=com
uSNCreated=12729
uSNChanged=12752
objectClass=top
objectClass=group
distinguishedName=CN=BPMUsers,CN=Users,DC=uk,DC=ibm,DC=com
objectCategory=CN=Group,CN=Schema,CN=Configuration,DC=uk,DC=ibm,DC=com
sAMAccountName=BPMUsers
objectGUID=HLtU
whenCreated=20170310074140.0Z
whenChanged=20170310074309.0Z
dSCorePropagationData=16010101000000.0Z

which is GREAT.

Thanks IBM :-)

*UPDATE*

I've just found this: -


which says, in part: -

The fix for this APAR is currently targeted for inclusion in
fix packs 7.0.0.41, 8.0.0.12 and 8.5.5.8

so it's been there since 8.5.5.8 ;-)

Reminder - Insufficient ulimit Value Causes Native OutOfMemory

This from IBM: -

An out of memory may be observed on a system running WebSphere Application Server on Linux or AIX that is due to ulimit restrictions on number of process/threads per user. Further investigation may reveal a "Failed to create a thread:" message within the generated javacore which would indicate a native out of memory issue has been encountered. The cause of the problem may be an insufficient ulimit setting. While this type of issue can occur on any level of Linux, or AIX, the issue is most likely to be seen in systems running multiple instances of WebSphere Application Server using one userid. The following will outline how to identify if a process ulimit is the culprit and what WebSphere Application Server Support recommends to fix the case.


Thanks to IBM_CTS on Twitter :-)


Thursday 9 March 2017

WebSphere User Group - London - Thursday 30 March 2017

This popped up in my inbox today: -

It's only 3 weeks to go until the next WebSphere User Group event in London - it's on Thursday, March 30th at IBM South Bank.

Agenda Topics:

The provisional agenda is now available! Take a look here. Based on user feedback we have moved a bit of the focus to topics like DevOps, Docker, Microservices and Cloud as well as providing the latest updates in the Application Server, Integration and Messaging spaces that this event is know for.

This year we have restructured the day a little: less concurrent tracks but slightly shorter presentations to ensure that you won't miss the key topics. We will also be running some open discussion / drop in sessions for key technologies, where you can come and talk to other users or to the IBM team and ask any question!

Registration:

The date - March 30th - is only a few weeks away so why not register now?  
Just go to http://www.websphereusergroup.co.uk/wug/meeting/43/ and click the "Please register here" link near the top, or click here to go directly to registration.  We'll send out more information closer to the time once the agenda is nearly final but also watch the website where you'll soon start to see the draft agenda as it develops.  Please note that, as with last years event, the registration link will take you to IBM, who are handling the registrations for this event, which should provide a smoother and more reliable registration experience for you.

Contact us:

If you have any questions, comments, please email us: chairman@websphereusergroup.co.uk

I'll be there, will you ?

Wednesday 8 March 2017

Customize the style of IBM BPM coaches - A pattern and a hierarchical approach

I feel like a proud father :-)

This was authored, and peer-reviewed, by members of my team :-)


This article focuses on how to customize responsive coaches and gives you a pattern for maintaining the customizations across an entire enterprise. If you don't want to make major customizations to the style of your coaches, but you still want responsive user interfaces that work well on all devices, see the Access IBM BPM from anywhere series.

Tuesday 7 March 2017

ORA-12514, TNS:listener does not currently know of service requested in connect descriptor DSRA0010E: SQL State = 08006, Error Code = 12,514

So I saw this: -



The test connection operation failed for data source BPM Business Process Choreographer data source on server nodeagent at node Node1 with the following exception: java.sql.SQLException: Listener refused the connection with the following error: ORA-12514, TNS:listener does not currently know of service requested in connect descriptor DSRA0010E: SQL State = 08006, Error Code = 12,514. View JVM logs for further details.

when testing a JDBC data source connection between WebSphere Application Server 8.5.5.11 and Oracle 12c.

This is what appears in the WAS Node Agent log ( SystemOut.log ) : -


[3/7/17 14:47:23:940 UTC] 00000086 DataSourceCon E   DSRA8040I: Failed to connect to the DataSource "".  Encountered java.sql.SQLException: Listener refused the connection with the following error:
ORA-12514, TNS:listener does not currently know of service requested in connect descriptor
  DSRA0010E: SQL State = 08006, Error Code = 12,514
...

Caused by: java.lang.Exception: Listener refused the connection with the following error:
ORA-12514, TNS:listener does not currently know of service requested in connect descriptor

The URL configured within the data source was: -

jdbc:oracle:thin:@oracle.novalocal:1521/orcl

and I knew that the WAS box could resolve the hostname of the Oracle box, and vice versa.

I'd also ensured that the listener port ( 1521 ) was accessible between the two boxes, which I'd tested via Telnet (!) : -

telnet oracle.novalocal 1521

which proved the network connection ……..

Or did it ?

On the Oracle box, I started to dig into the problem, by checking the listener: -

lsnrctl status listener

LSNRCTL for Linux: Version 12.2.0.1.0 - Production on 07-MAR-2017 05:46:29

Copyright (c) 1991, 2016, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=oracle.novalocal)(PORT=1521)))
STATUS of the LISTENER
------------------------
Alias                     listener
Version                   TNSLSNR for Linux: Version 12.2.0.1.0 - Production
Start Date                07-MAR-2017 05:44:10
Uptime                    0 days 0 hr. 2 min. 18 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /home/oracle/app/oracle/product/12.2.0/dbhome_1/network/admin/listener.ora
Listener Log File         /home/oracle/app/oracle/diag/tnslsnr/oracle/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oracle.novalocal)(PORT=1521)))
Services Summary...
Service "orcl.novalocal" has 1 instance(s).
  Instance "orcl", status READY, has 1 handler(s) for this service...
Service "orclXDB.novalocal" has 1 instance(s).
  Instance "orcl", status READY, has 1 handler(s) for this service...
The command completed successfully

and then looking at the Listener trace: -

tail -f /home/oracle/app/oracle/diag/tnslsnr/oracle/listener/trace/listener.log

...
07-MAR-2017 05:44:56 * (CONNECT_DATA=(CID=(PROGRAM=JDBC Thin Client)(HOST=__jdbc__)(USER=wasadmin))(SERVICE_NAME=orcl)) * (ADDRESS=(PROTOCOL=tcp)(HOST=9.20.65.171)(PORT=32860)) * establish * orcl * 12514
TNS-12514: TNS:listener does not currently know of service requested in connect descriptor


Note the port listed in the trace: -

32860

Ah-ha, I thought, and added that port into the firewall rules that sit between the two boxes ( I'm using OpenStack so we have a Security Group and some Rules.

Same error as before, and this time the Listener trace showed: -

07-MAR-2017 05:51:53 * (CONNECT_DATA=(CID=(PROGRAM=JDBC Thin Client)(HOST=__jdbc__)(USER=wasadmin))(SERVICE_NAME=orcl)) * (ADDRESS=(PROTOCOL=tcp)(HOST=9.20.65.171)(PORT=32882)) * establish * orcl * 12514
TNS-12514: TNS:listener does not currently know of service requested in connect descriptor

In other words, the port number is changing ….

So I went round and round the rabbit hole …

… before realising that, if Oracle is showing the port range that WAS is using, I must be getting past the OpenStack "firewall" 

Then I found and read this: -


which said, in part: -

I had this issue and the fix was to make sure in tnsnames.ora the SERVICE_NAME is a valid service name in your database. To find out valid service names, you can use the following query in oracle:

select value from v$parameter where name='service_names'

This led me to check the Listener using tnsping : -

tnsping `hostname`

TNS Ping Utility for Linux: Version 12.2.0.1.0 - Production on 07-MAR-2017 15:55:48

Copyright (c) 1997, 2016, Oracle.  All rights reserved.

Used parameter files:
/home/oracle/app/oracle/product/12.2.0/dbhome_1/network/admin/sqlnet.ora

Used EZCONNECT adapter to resolve the alias
Attempting to contact (DESCRIPTION=(CONNECT_DATA=(SERVICE_NAME=))(ADDRESS=(PROTOCOL=TCP)(HOST=::1)(PORT=1521))(ADDRESS=(PROTOCOL=TCP)(HOST=9.20.65.75)(PORT=1521)))
OK (0 msec)


and then check the Service Name: -

sqlplus / as sysdba

select value from v$parameter where name='service_names';

VALUE
————————————————————————————————————————
orcl.novalocal

at which point the screams could be heard half-way across London.

Yes, the Service Name in Oracle did NOT match that in the WAS JDBC Data Source :-(

Once I changed it to: -

jdbc:oracle:thin:@oracle.novalocal:1521/orcl.novalocal

it just worked



[3/7/17 16:00:22:394 UTC] 00000086 DSConfigurati I   DSRA8203I: Database product name : Oracle
[3/7/17 16:00:22:395 UTC] 00000086 DSConfigurati I   DSRA8204I: Database product version : Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production
[3/7/17 16:00:22:396 UTC] 00000086 DSConfigurati I   DSRA8205I: JDBC driver name  : Oracle JDBC driver
[3/7/17 16:00:22:396 UTC] 00000086 DSConfigurati I   DSRA8206I: JDBC driver version  : 12.1.0.1.0
[3/7/17 16:00:22:398 UTC] 00000086 DSConfigurati I   DSRA8025I: Successfully connected to DataSource.


So I know where I went wrong on the WAS / BPM side.

When I created the Deployment Environment, I specified orcl as the Database Name: -

bpm.de.db.1.databaseName=orcl

whereas it should've been orcl.novalocal 

However, I'm suspecting that I messed up the database configuration using dbca on the Oracle side.

Still, c'est la vie ...

For the record, the JDBC URL could / should have been: -

jdbc:oracle:thin:@//oracle.novalocal:1521/orcl.novalocal

i.e. the use of a double slash ( // ) before the host/domain name wasn't relevant / important.

PS For the record, I could've also run this command in SQLPlus: -

Show parameter service_name  

NAME     TYPE VALUE
------------------------------------ ----------- ------------------------------
service_names     string orcl.novalocal

*UPDATE*

I appear to have successfully updated the Service Name in Oracle, thanks to this: -

Cannot change service name for Oracle

This is what I did: -

sqlplus / as sysdba

SQL> alter system set db_domain='' scope=spfile;

System altered.

SQL> alter system register;

System altered.

SQL> shutdown
Database closed.
Database dismounted.
ORACLE instance shut down.

SQL> startup
ORACLE instance started.

Total System Global Area 2516582400 bytes
Fixed Size    8623832 bytes
Variable Size  687868200 bytes
Database Buffers 1811939328 bytes
Redo Buffers    8151040 bytes
Database mounted.
Database opened.

SQL> Show parameter service_name  

NAME     TYPE VALUE
------------------------------------ ----------- ------------------------------
service_names     string orcl

SQL> Exit

Disconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

lsnrctl status listener

LSNRCTL for Linux: Version 12.2.0.1.0 - Production on 08-MAR-2017 05:41:43

Copyright (c) 1991, 2016, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=oracle.novalocal)(PORT=1521)))
STATUS of the LISTENER
------------------------
Alias                     listener
Version                   TNSLSNR for Linux: Version 12.2.0.1.0 - Production
Start Date                07-MAR-2017 16:33:35
Uptime                    0 days 13 hr. 8 min. 8 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /home/oracle/app/oracle/product/12.2.0/dbhome_1/network/admin/listener.ora
Listener Log File         /home/oracle/app/oracle/diag/tnslsnr/oracle/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=oracle.novalocal)(PORT=1521)))
Services Summary...
Service "orcl" has 1 instance(s).
  Instance "orcl", status READY, has 1 handler(s) for this service...
Service "orclXDB" has 1 instance(s).
  Instance "orcl", status READY, has 1 handler(s) for this service...
The command completed successfully

Once I did this, I was able to revert my JDBC data sources back to this: -

jdbc:oracle:thin:@//oracle.novalocal:1521/orcl

and everything *appears* to work.

Now to start my clusters ....

Visual Studio Code - Wow 🙀

Why did I not know that I can merely hit [cmd] [p]  to bring up a search box allowing me to search my project e.g. a repo cloned from GitHub...