java.lang.UnsatisfiedLinkError: JCAM_Crypto_JNI (libCCLCore.so: cannot open shared object file: No such file or directory)

This caught me out during my scripted installation of IBM Business Monitor 8.0.1 ( with a back-end DB2 LUW 9.7.0.5 database ) on Red Hat Enterprise Linux.

During the startup of my clustered environment, I saw this in the SystemOut.log: -

….

[31/01/13 15:51:27:379 GMT] 00000015 FfdcProvider  W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on /opt/IBM/WebSphere/AppServer/profiles/AppSrv01/logs/ffdc/PCSR011.Support_858d66b1_13.01.31_15.51.27.3254262725509703725002.txt com.ibm.ws.webcontainer.servlet.ServletInstance.init 181
[31/01/13 15:51:27:381 GMT] 00000015 servlet       E com.ibm.ws.webcontainer.servlet.ServletWrapper init Uncaught.init.exception.thrown.by.servlet
[31/01/13 15:51:27:387 GMT] 00000015 webapp        E com.ibm.ws.webcontainer.webapp.WebApp commonInitializationFinally SRVE0266E: Error occured while initializing servlets: {0}
                                 javax.servlet.ServletException: SRVE0207E: Uncaught initialization exception created by servlet
        at com.ibm.ws.webcontainer.servlet.ServletWrapper.init(ServletWrapper.java:398)
        at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.init(ServletWrapperImpl.java:168)
        at com.ibm.ws.webcontainer.servlet.ServletWrapper.loadOnStartupCheck(ServletWrapper.java:1351)
        at com.ibm.ws.webcontainer.webapp.WebApp.doLoadOnStartupActions(WebApp.java:609)
        at com.ibm.ws.webcontainer.webapp.WebApp.commonInitializationFinally(WebApp.java:578)
        at com.ibm.ws.webcontainer.webapp.WebAppImpl.initialize(WebAppImpl.java:421)
        at com.ibm.ws.webcontainer.webapp.WebGroupImpl.addWebApplication(WebGroupImpl.java:88)
        at com.ibm.ws.webcontainer.VirtualHostImpl.addWebApplication(VirtualHostImpl.java:169)
        at com.ibm.ws.webcontainer.WSWebContainer.addWebApp(WSWebContainer.java:749)
        at com.ibm.ws.webcontainer.WSWebContainer.addWebApplication(WSWebContainer.java:634)
        at com.ibm.ws.webcontainer.component.WebContainerImpl.install(WebContainerImpl.java:422)
        at com.ibm.ws.webcontainer.component.WebContainerImpl.start(WebContainerImpl.java:714)
        at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:1160)
        at com.ibm.ws.runtime.component.DeployedApplicationImpl.fireDeployedObjectStart(DeployedApplicationImpl.java:1369)
        at com.ibm.ws.runtime.component.DeployedModuleImpl.start(DeployedModuleImpl.java:639)
        at com.ibm.ws.runtime.component.DeployedApplicationImpl.start(DeployedApplicationImpl.java:967)
        at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:766)
        at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:2153)
        at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:445)
        at com.ibm.ws.runtime.component.CompositionUnitImpl.start(CompositionUnitImpl.java:123)
        at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:388)
        at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.access$500(CompositionUnitMgrImpl.java:116)
        at com.ibm.ws.runtime.component.CompositionUnitMgrImpl$CUInitializer.run(CompositionUnitMgrImpl.java:994)
        at com.ibm.wsspi.runtime.component.WsComponentImpl$_AsynchInitializer.run(WsComponentImpl.java:349)
        at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1690)
Caused by: java.lang.UnsatisfiedLinkError: JCAM_Crypto_JNI (libCCLCore.so: cannot open shared object file: No such file or directory)
        at java.lang.ClassLoader.loadLibraryWithPath(ClassLoader.java:1053)
        at java.lang.ClassLoader.loadLibraryWithClassLoader(ClassLoader.java:1017)
        at java.lang.System.loadLibrary(System.java:509)
        at com.cognos.accman.jcam.crypto.jni.JNISystemProperties.<clinit>(JNISystemProperties.java:27)
        at java.lang.J9VMInternals.initializeImpl(Native Method)
        at java.lang.J9VMInternals.initialize(J9VMInternals.java:228)
        at com.cognos.accman.jcam.crypto.SystemProtectionSession._getRawKey(SystemProtectionSession.java:244)
        at com.cognos.accman.jcam.crypto.SystemProtectionSession.<init>(SystemProtectionSession.java:57)
        at com.cognos.accman.jcam.crypto.misc.Configuration.<init>(Configuration.java:70)
        at com.cognos.accman.jcam.crypto.misc.Configuration.getInstanceWithDefaultConfig(Configuration.java:113)
        at com.cognos.accman.jcam.crypto.misc.JVMEnvironment.isFIPSSupportedOS(JVMEnvironment.java:212)
        at com.cognos.accman.jcam.crypto.CAMFactory.checkFIPSSupport(CAMFactory.java:2814)
        at com.cognos.accman.jcam.crypto.CAMFactory.initialize(CAMFactory.java:125)
        at com.cognos.indications.LogIPFControl.initCAMCrypto(LogIPFControl.java:566)
        at com.cognos.indications.LogIPFControl.initialize(LogIPFControl.java:255)
        at com.cognos.indications.LogIPFControl.start(LogIPFControl.java:235)
        at com.cognos.p2plb.config.ConfigStoreServlet.init(ConfigStoreServlet.java:122)
        at com.cognos.pogo.isolation.ServletWrapper.initServlet(ServletWrapper.java:121)
        at com.cognos.pogo.isolation.ServletWrapper.init(ServletWrapper.java:101)
        at com.ibm.ws.webcontainer.servlet.ServletWrapper.init(ServletWrapper.java:336)
        ... 24 more

[31/01/13 15:51:27:397 GMT] 00000015 webcontainer  I com.ibm.ws.webcontainer.VirtualHostImpl addWebApplication SRVE0250I: Web Module IBM Cognos has been bound to default_host[*:8444,*:8443,*:443].
[31/01/13 15:51:27:466 GMT] 00000015 ApplicationMg A   WSVR0221I: Application started: IBM Cognos

….

Following this Technote: -

Unable to run cogconfig.sh: Can't find library JCAM_Crypto_JNI

I used the ldd command against the Linux library that was failing to load - libCCLCore.so : -

$ locate libCCLCore.so

/opt/IBM/WebSphere/AppServer/cognos/bin/libCCLCore.so
/opt/IBM/WebSphere/AppServer/cognos/bin64/libCCLCore.so
/opt/IBM/WebSphere/AppServer/cognos/cgi-bin/libCCLCore.so

$ ldd /opt/IBM/WebSphere/AppServer/cognos/bin/libCCLCore.so

linux-gate.so.1 =>  (0x00644000)
libz.so.1 => /lib/libz.so.1 (0x001c0000)
libicucogi18n.so.40 => not found
libicucoguc.so.40 => not found
libX11.so.6 => /usr/lib/libX11.so.6 (0x00e3b000)
libdl.so.2 => /lib/libdl.so.2 (0x00ba7000)
libnsl.so.1 => /lib/libnsl.so.1 (0x00a76000)
libpthread.so.0 => /lib/libpthread.so.0 (0x006b3000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00395000)
libm.so.6 => /lib/libm.so.6 (0x0014d000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00177000)
libc.so.6 => /lib/libc.so.6 (0x001d4000)
libxcb.so.1 => /usr/lib/libxcb.so.1 (0x00195000)
/lib/ld-linux.so.2 (0x008ca000)
libXau.so.6 => /usr/lib/libXau.so.6 (0x001b3000)

$ ldd /opt/IBM/WebSphere/AppServer/cognos/bin64/libCCLCore.so

linux-vdso.so.1 =>  (0x00007fff04aac000)
libz.so.1 => /lib64/libz.so.1 (0x00007ffbf50c3000)
libicucogi18n.so.40 => not found
libicucoguc.so.40 => not found
libX11.so.6 => /usr/lib64/libX11.so.6 (0x00007ffbf4d83000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007ffbf4b7e000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x00007ffbf4965000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ffbf4748000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007ffbf4441000)
libm.so.6 => /lib64/libm.so.6 (0x00007ffbf41bd000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007ffbf3fa7000)
libc.so.6 => /lib64/libc.so.6 (0x00007ffbf3c13000)
libxcb.so.1 => /usr/lib64/libxcb.so.1 (0x00007ffbf39f8000)
/lib64/ld-linux-x86-64.so.2 (0x00000032d4400000)
libXau.so.6 => /usr/lib64/libXau.so.6 (0x00007ffbf37f4000)

$ ldd /opt/IBM/WebSphere/AppServer/cognos/cgi-bin/libCCLCore.so

linux-gate.so.1 =>  (0x006cb000)
libz.so.1 => /lib/libz.so.1 (0x00e73000)
libicucogi18n.so.40 => not found
libicucoguc.so.40 => not found
libX11.so.6 => /usr/lib/libX11.so.6 (0x00952000)
libdl.so.2 => /lib/libdl.so.2 (0x00cf3000)
libnsl.so.1 => /lib/libnsl.so.1 (0x00da1000)
libpthread.so.0 => /lib/libpthread.so.0 (0x002d2000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x002ed000)
libm.so.6 => /lib/libm.so.6 (0x0088d000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x003d8000)
libc.so.6 => /lib/libc.so.6 (0x003f6000)
libxcb.so.1 => /usr/lib/libxcb.so.1 (0x0058c000)
/lib/ld-linux.so.2 (0x008ca000)
libXau.so.6 => /usr/lib/libXau.so.6 (0x005aa000)

I went around this loop for a while, using the locate and ldd commands to track down the libicucogi18n.so.40 and libicucoguc.so.40 libraries, but to no avail.

I then did some Googling, as one does, and found this developerWorks forum post: -

Oracle Data Source Connect Fails libcogudaor

which led me to this Technote: -

Unable to test an Oracle 10g data source connection: RQP-DEF-0068 UDA-SQL-0432

Now I'm not using Oracle but the latter article said something: -

The LD_LIBRARY_PATH does not reference the 32-bit library for the Oracle Client. The 64-bit client is being referenced instead

that made me think.

I checked the IBM Business Monitor 8.0.1 Installation Guide, which said: -

Following that, I updated the LD_LIBRARY_PATH property ( Servers > Clusters > WebSphere application server clusters > PCSR01.Support > Cluster members > PCSR011.Support > Process definition > Environment Entries > LD_LIBRARY_PATH ) for the PCSR01.Support cluster member ( that which runs the Cognos BI code ) from: -

/opt/IBM/WebSphere/AppServer/profiles/AppSrv01/cognos/PCSR011.Support/bin64

to: -

/opt/IBM/WebSphere/AppServer/profiles/AppSrv01/cognos/PCSR011.Support/bin64:/opt/ibm/db2/V9.7/lib32/

and restarted my cluster.

As they say in the movies, Job Done !

Next step is to script the same using wsadmin and Jython.

Security Bulletin: An IBM Business Process Manager SSL connection can be established without host name verification: CVE-2012-5785

I'm reposting this from this IBM Flash: -

Security Bulletin: An IBM Business Process Manager SSL connection can be established without host name verification: CVE-2012-5785

Abstract

A Secure Sockets Layer (SSL) connection can be established without host name verfication, which an make the connection vulnerable to a man-in-the-middle attack.

Content

While obtaining an SSL connection, the IBM Business Process Management (BPM) system does not validate the host name of the target connection against the SubjectDN of the certificate. This situation can make the connection vulnerable to a man-in-the-middle attack.

CVE ID: 2012-5785
CVSS Base Score: 4.3
CVSS Temporal Score: See http://xforce.iss.net/xforce/xfdb/79830 for the current score.
CVSS Environmental Score*: Undefined
CVSS Vector: (AV:N/AC:M/Au:N/C:N/I:P/A:N)

REMEDIATION

To eliminate a man-in-the-middle attack, apply Interim Fixes JR45329, JR45216, and JR45071, or apply a Fix Pack that contains these APARS. These changes verify the host name against the certificate SubjectDN value. Using the following links, download the interim fixes from IBM Fix Central for IBM Integration Designer, Business Space (IBM Business Monitor) and your applicable IBM Business Process Manager product:

• IBM Integration Designer: APAR JR45329

• Business Space: APAR JR45216

• IBM Business Process Manager Standard: APAR JR45071

• IBM Business Process Manager Express: APAR JR45071

• IBM Business Process Manager Advanced: APAR JR45071

Please please refer the IBM Flash for further details, including the recommended fixes.

Patching IBM Business Process Manager Advanced 8.0.1 on Linux

This is an aide memoire as I know I will need it before too long: -

Download Required Fixes

At the time of writing, there's only two iFixes for IBM BPM Advanced 8.0.1 on IBM Fix Central.

*UPDATE* I need to include a reference to this iFix Flash: -

Security Bulletin: An IBM Business Process Manager SSL connection can be established without host name verification: CVE-2012-5785

Install Fixes

$ mkdir /tmp/bpm801fixes
$ mkdir /tmp/bpm801fixes/IFJR44937
$ mkdir /tmp/bpm801fixes/IFJR45071

$ unzip ~/8.0.1.0-WS-BPM-IFJR44937.zip -d /tmp/bpm801fixes/IFJR44937/

$ unzip ~/8.0.1.0-WS-BPM-IFJR45071.zip -d /tmp/bpm801fixes/IFJR45071/

$ /opt/IBM/InstallationManager/eclipse/tools/imcl install 8.0.1.0-WS-BPM-IFJR44937 -repositories /tmp/bpm801fixes/IFJR44937/ -installationDirectory /opt/IBM/WebSphere/AppServer -log ~/IFJR44937.txt

Installed 8.0.1.0-WS-BPM-IFJR44937_8.0.1000.20121203_1324 to the /opt/IBM/WebSphere/AppServer directory.

$ /opt/IBM/InstallationManager/eclipse/tools/imcl install 8.0.1.0-WS-BPM-IFJR45071 -repositories /tmp/bpm801fixes/IFJR45071/ -installationDirectory /opt/IBM/WebSphere/AppServer -log ~/IFJR45071.txt

Installed 8.0.1.0-WS-BPM-IFJR45071_8.0.1000.20121218_1447 to the /opt/IBM/WebSphere/AppServer directory.

Job Done!

IBM Process Designer - Not playing nicely with IBM Process Center ( BPM Advanced 8.0.1 )

I've seen this a few times now, and wanted to get my initial thoughts down on "paper".

As part of an IBM Business Process Manager Advanced 8.0.1 installation, there's normally a requirement to use the IBM Process Designer (IPD) developer client to create and deploy processes up to the Process Center server.

BPM makes this even easier by including a link to the Process Designer tool from the Process Center page: -

https://bpmpc.uk.ibm.com:8443/ProcessCenter/repository/com.lombardisoftware.repository.Repository/Repository.jsp

One thing that has been bugging me is that the downloaded Process Designer bundle: -

-rw-r--r--@  1 hayd  staff   423M 30 Jan 09:41 IBM Process Designer.zip

( which includes Eclipse )

doesn't seem to automagically connect to the right Process Center URL.

Once one expands the downloaded ZIP file, there's a configuration file - eclipse.ini e.g. /Users/hayd/Downloads/IBM Process Designer/eclipse.ini which contains the following line: -

-install
file:.
-configuration
configuration
-name
IBM BPM
-consoleLog
-dir
ltr
-clean
-vm
./AppClient/java/jre/bin/javaw.exe
-vmargs
-Xms128m
-Xmx512m
-XX:PermSize=128m
-XX:MaxPermSize=128m
-Djavax.net.ssl.keyStoreType=PKCS12
-Djavax.net.ssl.keyStore=./etc/key.p12
-Djavax.net.ssl.keyStorePassword=WebAS
-Djavax.net.ssl.trustStoreType=PKCS12
-Djavax.net.ssl.trustStore=./etc/trust.p12
-Djavax.net.ssl.trustStorePassword=WebAS
-Dcom.ibm.CORBA.ConfigURL="file:resources/sas.client.props"
-Dcom.ibm.CORBA.FragmentSize="0"
-Dcom.ibm.SSL.ConfigURL="file:resources/ssl.client.props"
-Djava.security.auth.login.config="file:resources/wsjaas_client.conf"
-Djava.naming.factory.initial=com.ibm.websphere.naming.WsnInitialContextFactory
-Dcom.ibm.bpm.processcenter.url=https://bpmpc.uk.ibm.com:8443
-Djava.ext.dirs="./AppClient/java/jre/lib/ext;./AppClient/lib;./AppClient/plugins"
-Dcom.lombardisoftware.core.TWEnvironment.environmentName=AUTHORING_ENVIRONMENT
-DentityExpansionLimit=2147483647

which looks good.

However, upon startup, IPD was still trying to connect to the wrong port with the wrong protocol during the authentication process.

Using the above example, IPD would try and authenticate against: -

rhel6.uk.ibm.com:9080

rather than: -

bpmpc.uk.ibm.com:8443

Talking with a learned colleague in Germany, it looks like IPD is making a request to the Process Center on the right URL - bpmpc.uk.ibm.com:8443 - *BUT* is then retrieving the wrong bootstrap host/port combination: -

the initial call is to /repo, which will respond with the server's bootstrap address, which then will be used for EJB and JMS communication.
It that bootstrap address contained an unresolvable host name, you'd see the described behaviour.

He also directed me to a new ( to me ) configuration file: -

/opt/IBM/WebSphere/AppServer/profiles/PCDMProfile/config/cells/PCCELL/nodes/PCNODE1/servers/PCSR011.AppTarget/process-center/TeamWorksConfiguration.running.xml

which contains a whole slew of WRONG host/port combinations: -

    <images-prefix>http://rhel6.uk.ibm.com:9080/teamworks</images-prefix>
    <portal-prefix>http://rhel6.uk.ibm.com:9080/portal</portal-prefix>
    <repository-prefix>http://rhel6.uk.ibm.com:9080/ProcessCenter</repository-prefix>
    <servlet-prefix>http://rhel6.uk.ibm.com:9080/teamworks</servlet-prefix>
    <webapi-prefix>http://rhel6.uk.ibm.com:9080/webapi</webapi-prefix>
      <base-url>http://rhel6.uk.ibm.com:9080/teamworks/webservices</base-url>
TeamWorks Link:  http://rhel6.uk.ibm.com:9080/portal/jsp/getProcessDetails.do?bpdInstanceId=<#= tw.system.process.instanceId #>
    <portal-prefix>http://rhel6.uk.ibm.com:9080/portal</portal-prefix>
    <process-admin-prefix>http://rhel6.uk.ibm.com:9080/ProcessAdmin</process-admin-prefix>
    <teamworks-webapp-prefix>http://rhel6.uk.ibm.com:9080/teamworks</teamworks-webapp-prefix>
      <default-namespace-uri>http://rhel6.uk.ibm.com:9080/schema/</default-namespace-uri>
    <coach-designer-xsl-url>http://rhel6.uk.ibm.com:9080/teamworks/coachdesigner/transform/CoachDesigner.xsl</coach-designer-xsl-url>
      <provider-url>corbaname:iiop:rhel6.uk.ibm.com:2810</provider-url>
    <jndi-url>corbaname:iiop:rhel6.uk.ibm.com:2810</jndi-url>
        <url>jdbc:db2://rhel6.uk.ibm.com:50000/BPMDB</url>
        <client-link>http://rhel6.uk.ibm.com:9080/teamworks</client-link>
    <repository-server-url>http://rhel6.uk.ibm.com:9080/ProcessCenter</repository-server-url>
    <server-host>rhel6.uk.ibm.com</server-host>
      <env key="java.naming.provider.url" value="corbaname:iiop:rhel6.uk.ibm.com:2810" />

I've yet to find out if I can automagically update this file during the installation and configuration phase BUT at least I know where to look.

Meantime, here's some required reading: -

Configuration file overview and explanation for Lombardi Teamworks, WebSphere Lombardi Edition (WLE), and the IBM Business Process Manager (BPM) products

Changing the database information for Process Server and the Performance Data Warehouse for the IBM Business Process Manager (BPM) products

*UPDATE* Reading the last link, it looks like TeamWorksConfiguration.running.xml gets its configuration from 99Local.xml and 100Custom.xml, so that's the first place I'm going to look :-)

*UPDATE AGAIN*

Thanks to some help from my mate, Bob, it's now working OK.

It's all down to 100Custom.xml AND the need to perform a FULL resynchronization of the cell configuration.

This is the DM Profile cell/node/server copy: -

-rw-r--r-- 1 wasadmin wasadmins 3880 Jan 29 16:09 /opt/IBM/WebSphere/AppServer/profiles/PCDMProfile/config/cells/PCCELL/nodes/PCNODE1/servers/PCSR011.AppTarget/process-center/config/100Custom.xml

and this is the Node Profile cell/node/server copy: -

-rw-r--r-- 1 wasadmin wasadmins 2757 Jan 29 13:52 /opt/IBM/WebSphere/AppServer/profiles/PCN1Profile/config/cells/PCCELL/nodes/PCNODE1/servers/PCSR011.AppTarget/process-center/config/100Custom.xml
I did a full resynch. and this is what I now have: -

-rw-r--r-- 1 wasadmin wasadmins 3880 Jan 29 16:09 /opt/IBM/WebSphere/AppServer/profiles/PCN1Profile/config/cells/PCCELL/nodes/PCNODE1/servers/PCSR011.AppTarget/process-center/config/100Custom.xml

So the full resynch. was the missing link :-)

After restarting the AppTarget cluster, I'm now seeing that the Node Profile copy of TeamWorksConfiguration.running.xml is getting updated: -

-rw-r--r-- 1 wasadmin wasadmins 373599 Jan 30 17:13 /opt/IBM/WebSphere/AppServer/profiles/PCN1Profile/config/cells/PCCELL/nodes/PCNODE1/servers/PCSR011.AppTarget/process-center/TeamWorksConfiguration.running.xml

and it's got the correct entries from 100Custom.xml: -

...
    https://bpm801.uk.ibm.com:8443/teamworks
    https://bpm801.uk.ibm.com:8443/portal
    https://bpm801.uk.ibm.com:8443/ProcessCenter
    https://bpm801.uk.ibm.com:8443/teamworks
    true
    https://bpm801.uk.ibm.com:8443/webapi
...

Now to download a new copy of Process Designer, and see whether it connects OK.

IBM Business Process Manager Advanced 8.0.1 - Bootstrapping not …. strapping ?

I saw this earlier: -

INFO:Error creating bean with name 'ejbCore.RepositoryServicesCore' defined in class path resource [registry.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.lombardisoftware.server.ejb.repositoryservices.RepositoryServicesCore]: Constructor threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ejbCore.PersistenceServicesCore' defined in class path resource [registry.xml]: Cannot resolve reference to bean 'handlersMap' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'handlersMap': Cannot create inner bean 'com.lombardisoftware.server.ejb.persistence.PSDefaultHandler#675fd7ca' of type [com.lombardisoftware.server.ejb.persistence.PSDefaultHandler] while setting bean property 'sourceMap' with key [TypedStringValue: value [Task], target type [null]]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.lombardisoftware.server.ejb.persistence.PSDefaultHandler#675fd7ca#1' defined in class path resource [registry.persistence.xml]: Cannot resolve reference to bean 'dao.task' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dao.task' defined in class path resource [registry.persistence.xml]: Instantiation of bean failed; nested exception is java.lang.NoClassDefFoundError: com.lombardisoftware.server.ejb.persistence.dao.TaskDAO (initialization failure)
SEVERE:

in: -

/opt/IBM/WebSphere/AppServer/profiles/PCDMProfile/logs/bootstrapProcesServerData.PCSR01.AppTarget.Tue-Jan-29-14.26.29-2013.log 

using the bootstrapProcessServerData.sh command.

Now I know that this worked, as I'd done the same thing earlier.

I had to dig back through the console log for the session in which I was running the bootstrapProcessServerData command, and found: -

…

Caused by: org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.lombardisoftware.bpd.runtime.engine.message.DefaultMessageRoutingCache]: Constructor threw exception; nested exception is org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [select propvalue from lsw_system where propkey=?]; nested exception is com.ibm.db2.jcc.am.SqlSyntaxErrorException: DB2 SQL Error: SQLCODE=-204, SQLSTATE=42704, SQLERRMC=DB2INST1.LSW_SYSTEM, DRIVER=4.11.69
…

At this point, I realised that ( DOH! ), I hadn't actually created the relevant DB2 database tables - DB2INST1.LSW_SYSTEM etc.

I quickly resolved this by executing: -

$ db2 -tvf /opt/IBM/WebSphere/AppServer/profiles/PCDMProfile/dbscripts/ProcessServer/DB2/BPMDB/createTable_ProcessServer.sql 

and: -

…

The bootstrapping of data completed.....
'BootstrapProcessServerData admin command completed.....'

…

What was annoying is that there was no obvious mention of the issue in any of the WAS logs e.g. /opt/IBM/WebSphere/AppServer/profiles/PCDMProfile/logs, /opt/IBM/WebSphere/AppServer/profiles/PCDMProfile/ffdc etc.

IBM Business Process Manager Advanced 8.0.1 - Database Schemas for the Service Integration bus Messaging Engines

Just writing this down for future reference.

Whilst in BPM Standard, we only have two schemas ( schemae ? ) for the SIbus Messaging Engines - - with Advanced, we have a few more: -

Schema CMNBM00

JNDI Name of JDBC Data Source jdbc/com.ibm.ws.sib/PCSR01.Messaging-BPC.PCCELL.Bus
JDBC Data Source Business Process Choreographer ME data source 
Database CMNDB

Schema CMNCM00

JNDI Name of JDBC Data Source jdbc/com.ibm.ws.sib/PCSR01.Messaging-CEI.PCCELL.BUS
JDBC Data Source CEI ME data source 
Database CMNDB

Schema CMNPE00

JNDI Name of JDBC Data Source jdbc/com.ibm.ws.sib/twperfsvr_bus
JDBC Data Source Performance Data Warehouse ME data source
Database CMNDB

Schema CMNPS00

JNDI Name of JDBC Data Source jdbc/com.ibm.ws.sib/twprocsvr_bus
JDBC Data Source Process Server ME data source 
Database CMNDB

Schema CMNSA00

JNDI Name of JDBC Data Source jdbc/com.ibm.ws.sib/PCSR01.Messaging-SCA.APPLICATION.PCCELL.Bus
JDBC Data Source SCA Application Bus ME data source 
Database CMNDB

Schema CMNSS00

JNDI Name of JDBC Data Source jdbc/com.ibm.ws.sib/PCSR01.Messaging-SCA.SYSTEM.PCCELL.Bus
JDBC Data Source SCA System Bus ME data source
Database CMNDB

The SQL scripts required to create these six schemas can be generated using the sibDDLGenerator.sh script, as follows: -

$ cd /opt/IBM/WebSphere/AppServer/profiles/PCDMProfile/bin/sibDDLGenerator.sh

$ ./sibDDLGenerator.sh -system db2 -version 9.7 -platform unix -schema CMNBM00 >> ~/createBPM801Schemas.sql -statementend ";"

$ ./sibDDLGenerator.sh -system db2 -version 9.7 -platform unix -schema CMNCM00 >> ~/createBPM801Schemas.sql -statementend ";"
$ ./sibDDLGenerator.sh -system db2 -version 9.7 -platform unix -schema CMNPE00 >> ~/createBPM801Schemas.sql -statementend ";"
$ ./sibDDLGenerator.sh -system db2 -version 9.7 -platform unix -schema CMNPS00 >> ~/createBPM801Schemas.sql -statementend ";"
$ ./sibDDLGenerator.sh -system db2 -version 9.7 -platform unix -schema CMNSA00 >> ~/createBPM801Schemas.sql -statementend ";"
$ ./sibDDLGenerator.sh -system db2 -version 9.7 -platform unix -schema CMNSS00 >> ~/createBPM801Schemas.sql -statementend ";"

WSVR0016W for pdq.jar and pdqmgmr.jar during creation of an IBM BPM Advanced 8.0.1 Deployment Manager profile

I am using IBM Business Process Manager Advanced 8.0.1 on Red Hat Enterprise Linux 6.3 (x86-64). During the installation, I have created a Deployment Manager ( Process Center ) profile using the following template: -

/opt/IBM/WebSphere/AppServer/profileTemplates/BPM/dmgr.procctr.adv

During the start-up of the DMGR profile, I see the following two exceptions: -

[29/01/13 12:57:10:993 GMT] 00000000 ResourceMgrIm W   WSVR0016W: Classpath entry, ${PUREQUERY_PATH}/pdq.jar, in Resource, DB2 Using IBM JCC Driver (XA), located at /opt/IBM/WebSphere/AppServer/profiles/PCDMProfile/config/cells/PCCELL/resources.xml has an invalid variable
[29/01/13 12:57:10:998 GMT] 00000000 ResourceMgrIm W   WSVR0016W: Classpath entry, ${PUREQUERY_PATH}/pdqmgmt.jar, in Resource, DB2 Using IBM JCC Driver (XA), located at /opt/IBM/WebSphere/AppServer/profiles/PCDMProfile/config/cells/PCCELL/resources.xml has an invalid variable

These two JAR files - pdq.jar and pdqmgmr.jar - appear to relate to the IBM Optim pureQuery run-time.

As far as I am aware, this isn't something that IBM BPM Advanced 8.0.1 requires.

The two JARs are not installed with the underlying WebSphere Application Server Network Deployment 8.0.0.5 product, or with the IBM BPM Advanced 8.0.1 code.

This is what I have installed: -

$ /opt/IBM/InstallationManager/eclipse/tools/imcl listInstalledPackages

com.ibm.cic.agent_1.6.0.20120831_1216

com.ibm.websphere.IHS.v80_8.0.5.20121022_1902

com.ibm.bpm.ADV.V80_8.0.1000.20121102_2136

com.ibm.websphere.ND.v80_8.0.5.20121022_1902

com.ibm.websphere.PLG.v80_8.0.5.20121022_1902

If I remove the two JARs from the class path, either via the WAS Integrated Solutions Console, or via a modification of: -

/opt/IBM/WebSphere/AppServer/profiles/PCDMProfile/config/cells/PCCELL/resources.xml

...
    <classpath>${PUREQUERY_PATH}/pdq.jar</classpath>
    <classpath>${PUREQUERY_PATH}/pdqmgmt.jar</classpath>
...

the exceptions do not occur.

I've raised a PMR, and we'll see what happens :-)