WebSphere Liberty Profile on macOS - SuperSnooping

So I've run this on Liberty multiple times, including most recently on my Mac, natively.

This is what I have: -

Check Liberty Version

~/Downloads/wlp/bin/server version

objc[43120]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/bin/java (0x10761e4c0) and /Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre/lib/libinstrument.dylib (0x1077184e0). One of the two will be used. Which one is undefined.
WebSphere Application Server 16.0.0.2 (1.0.13.cl160220160526-2258) on Java HotSpot(TM) 64-Bit Server VM, version 1.8.0_112-b16 (en_GB)

Check server.xml

cat ~/Downloads/wlp/usr/servers/defaultServer/server.xml

<server description="new server">

    <!-- Enable features -->
    <featureManager>
        <feature>webProfile-7.0</feature>
        <feature>adminCenter-1.0</feature>
        <feature>localConnector-1.0</feature>
    </featureManager>

    <classloading useJarUrls="true"/>

    <quickStartSecurity userName="admin" userPassword="{xor}Lz4sLCgwLTs="/>

    <keyStore id="defaultKeyStore" password="{xor}EzY9Oi0rJg=="/>

    <!-- To access this server from a remote client add a host attribute to the following element, e.g. host="*" -->
    <httpEndpoint httpPort="9080" httpsPort="9443" id="defaultHttpEndpoint"/>
                  
    <!-- Automatically expand WAR files and EAR files -->
    <applicationManager autoExpand="true"/>

    <remoteFileAccess>
        <writeDir>${server.config.dir}</writeDir>
    </remoteFileAccess>


    <applicationMonitor updateTrigger="mbean"/>

Check Dropins

ls -al ~/Downloads/wlp/usr/servers/defaultServer/dropins/

total 24
drwxr-x---   3 davidhay  staff    102 27 Jan 18:10 .
drwxr-x---  10 davidhay  staff    340  1 Sep 10:02 ..
-rw-------   1 davidhay  staff  11486  9 Aug  2004 SuperSnoopWeb.war

Start Liberty

~/Downloads/wlp/bin/server start

Watch the logs

tail -f ~/Downloads/wlp/usr/servers/defaultServer/logs/messages.log 

product = WebSphere Application Server 16.0.0.2 (wlp-1.0.13.cl160220160526-2258)
wlp.install.dir = /Users/davidhay/Downloads/wlp/
java.home = /Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre
java.version = 1.8.0_112
java.runtime = Java(TM) SE Runtime Environment (1.8.0_112-b16)
os = Mac OS X (10.12.3; x86_64) (en_GB)
process = 42629@Davids-MacBook-Pro-4.local
...
[27/01/17 18:10:59:702 GMT] 00000026 com.ibm.ws.webcontainer.osgi.webapp.WebGroup                 I SRVE0169I: Loading Web Module: SuperSnoopWeb.
[27/01/17 18:10:59:702 GMT] 00000026 com.ibm.ws.webcontainer                                      I SRVE0250I: Web Module SuperSnoopWeb has been bound to default_host.
[27/01/17 18:10:59:702 GMT] 00000026 com.ibm.ws.http.internal.VirtualHostImpl                     A CWWKT0016I: Web application available (default_host): http://localhost:9080/SuperSnoopWeb/
[27/01/17 18:10:59:704 GMT] 00000026 com.ibm.ws.app.manager.AppMessageHelper                      A CWWKZ0001I: Application SuperSnoopWeb started in 0.423 seconds.
...
[27/01/17 18:10:59:712 GMT] 00000019 com.ibm.ws.kernel.feature.internal.FeatureManager            A CWWKF0011I: The server defaultServer is ready to run a smarter planet.

Test SuperSnoop

http://localhost:9080/SuperSnoopWeb/SuperSnoop

Watch the logs

tail -f ~/Downloads/wlp/usr/servers/defaultServer/logs/messages.log 

...
[27/01/17 18:12:05:806 GMT] 00000039 com.ibm.ws.session.WASSessionCore                            I SESN0176I: A new session context will be created for application key default_host/SuperSnoopWeb
[27/01/17 18:12:05:816 GMT] 00000039 com.ibm.ws.util                                              I SESN0172I: The session manager is using the Java default SecureRandom implementation for session ID generation.
[27/01/17 18:12:05:827 GMT] 00000039 com.ibm.ws.jsp                                               I JSPG8502I: The value of the JSP attribute jdkSourceLevel is "15".
[27/01/17 18:12:05:916 GMT] 00000039 com.ibm.ws.cache.CacheServiceImpl                            I DYNA1056I: Dynamic Cache (object cache) initialized successfully.
[27/01/17 18:12:06:055 GMT] 00000039 com.ibm.ws.webcontainer.servlet                              I SRVE0242I: [SuperSnoopWeb] [/SuperSnoopWeb] [SuperSnoop]: Initialization successful.
[27/01/17 18:12:06:056 GMT] 00000039 SystemOut                                                    O SuperSnoop running
...

I also downloaded the latest WLP: -

-rw-r--r--@  1 davidhay  staff  69984067 27 Jan 18:21 wlp-webProfile7-16.0.0.4.zip

...
WebSphere Application Server 16.0.0.4 (1.0.15.cl160420161113-0206) on Java HotSpot(TM) 64-Bit Server VM, version 1.8.0_112-b16 (en_GB)
...

from here: -

Get started with Liberty

and again SuperSnoopServlet worked without problems, with a default server.xml : -

<?xml version="1.0" encoding="UTF-8"?>
<server description="new server">

    <!-- Enable features -->
    <featureManager>
        <feature>webProfile-7.0</feature>
    </featureManager>

    <!-- To access this server from a remote client add a host attribute to the following element, e.g. host="*" -->
    <httpEndpoint id="defaultHttpEndpoint"
                  httpPort="9080"
                  httpsPort="9443" />
                  
    <!-- Automatically expand WAR files and EAR files -->
    <applicationManager autoExpand="true"/>

Fun, fun, fun

IBM WebSphere Liberty Profile - To Java or not to Java

One of my friends asked me about the implications of installing or not installing Java when one installs WebSphere Liberty Profile (WLP)

This is what I told him ...

It depends upon what you actually download and install. For example, I downloaded this: -

IBM WebSphere Application Server Liberty Core V8.5.5 for Multiplatform Multilingual (installed using IBM Installation Manager)  CIK2VML

from, as per this: -

How to download WebSphere Application Server Liberty Core V8.5.5 from Passport Advantage Online

and installed it using a response file -  installWLP.rsp : -

<?xml version='1.0' encoding='UTF-8'?>
<agent-input>
  <server>
    <repository location='/mnt/hgfs/DownloadDirector/WLP' temporary='true'/>
  </server>
  <profile id='WebSphere Liberty V8.5' installLocation='/opt/ibm/WebSphere/Liberty'>
    <data key='cic.selector.arch' value='x86_64'/>
  </profile>
  <install>
    <!-- IBM WebSphere Application Server Liberty Core 8.5.5.0 -->
    <offering profile='WebSphere Liberty V8.5' id='com.ibm.websphere.liberty.v85' version='8.5.5000.20130514_1313' features='liberty,embeddablecontainer'/>
  </install>
  <preference name='com.ibm.cic.common.core.preferences.eclipseCache' value='/opt/ibm/IMShared'/>
</agent-input>

This provides me with the WLP runtime, but NO JDK/JRE. This means that, when I start a server: -

/opt/ibm/WebSphere/Liberty/bin/server start

Starting server defaultServer.
Server defaultServer started with process ID 23307.

cat /opt/ibm/WebSphere/Liberty/usr/servers/defaultServer/logs/console.log 

Server defaultServer created.
Launching defaultServer (WebSphere Application Server 8.5.5.0/wlp-1.0.3.20130510-0831) on OpenJDK 64-Bit Server VM, version 1.7.0_121-mockbuild_2016_11_01_00_38-b00 (en_US)
[AUDIT   ] CWWKE0001I: The server defaultServer has been launched.
[AUDIT   ] CWWKZ0058I: Monitoring dropins for applications. 
[AUDIT   ] CWWKF0011I: The server defaultServer is ready to run a smarter planet.

cat /opt/ibm/WebSphere/Liberty/usr/servers/defaultServer/logs/messages.log 

********************************************************************************
product = WebSphere Application Server 8.5.5.0 (wlp-1.0.3.20130510-0831)
wlp.install.dir = /opt/ibm/WebSphere/Liberty/
java.home = /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.121.x86_64/jre
java.version = 1.7.0_121
java.runtime = OpenJDK Runtime Environment (1.7.0_121-mockbuild_2016_11_01_00_38-b00)
os = Linux (2.6.32-642.13.1.el6.x86_64; amd64) (en_US)
********************************************************************************
[1/19/17 19:47:15:189 GMT] 00000001 com.ibm.ws.logging.internal.TraceSpecification               I TRAS0018I: The trace state has been changed. The new trace state is *=info.
[1/19/17 19:47:15:203 GMT] 00000001 com.ibm.ws.kernel.launch.internal.FrameworkManager           A CWWKE0001I: The server defaultServer has been launched.
[1/19/17 19:47:16:372 GMT] 00000001 com.ibm.ws.kernel.launch.internal.FrameworkManager           I CWWKE0002I: The kernel started after 1.419 seconds
[1/19/17 19:47:16:432 GMT] 00000015 com.ibm.ws.kernel.feature.internal.FeatureManager            I CWWKF0007I: Feature update started.
[1/19/17 19:47:17:445 GMT] 00000019 com.ibm.ws.tcpchannel.internal.TCPChannel                    I CWWKO0219I: TCP Channel defaultHttpEndpoint has been started and is now listening for requests on host localhost  (IPv4: 127.0.0.1) port 9080.
[1/19/17 19:47:17:561 GMT] 0000000d com.ibm.ws.app.manager.internal.monitor.DropinMonitor        A CWWKZ0058I: Monitoring dropins for applications. 
[1/19/17 19:47:17:603 GMT] 00000015 com.ibm.ws.kernel.feature.internal.FeatureManager            I CWWKF0008I: Feature update completed in 1.229 seconds.
[1/19/17 19:47:17:603 GMT] 00000015 com.ibm.ws.kernel.feature.internal.FeatureManager            A CWWKF0011I: The server defaultServer is ready to run a smarter planet.

To run WLP, I'm using whatever JRE available to me.

I can see this as follows: -

which java

/usr/bin/java

`which java` -version

java version "1.7.0_121"
OpenJDK Runtime Environment (rhel-2.6.8.1.el6_8-x86_64 u121-b00)
OpenJDK 64-Bit Server VM (build 24.121-b00, mixed mode)

This is the JDK that comes with my Linux ( Red Hat Enterprise Linux 6.8 )

If I remove the JDKs from my box: -

rpm -e java-1.7.0-openjdk-1.7.0.121-2.6.8.1.el6_8.x86_64
rpm -e java-1.6.0-openjdk-1.6.0.40-1.13.12.6.el6_8.x86_64

I can't start WLP: -

/opt/ibm/WebSphere/Liberty/bin/server start

/opt/ibm/WebSphere/Liberty/bin/server: line 708: java: command not found

Therefore, I either need to rely upon the OS to provide a JDK ( which is, perhaps, an unsafe dependency ) or install an IBM JDK, such as: -

IBM WebSphere SDK Java Technology Edition V7.0 (1 of 2) (for WebSphere Application Server Liberty Core V8.5.5) Multiplatform Multilingual
IBM WebSphere SDK Java Technology Edition V7.0 (2 of 2) (for WebSphere Application Server Liberty Core V8.5.5) Multiplatform Multilingual

Whilst I download the latter, I installed the OpenJDK 1.8 onto my box: -

which java

/usr/bin/java

`which java` -version

openjdk version "1.8.0_111"
OpenJDK Runtime Environment (build 1.8.0_111-b15)
OpenJDK 64-Bit Server VM (build 25.111-b15, mixed mode)

Interestingly, Liberty doesn't want to start; most likely because it's not tested/supported with Java 8: -

/opt/ibm/WebSphere/Liberty/bin/server start

OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
Starting server defaultServer.

If I put Java 1.7 back on the box, we're golden: -

cat /opt/ibm/WebSphere/Liberty/usr/servers/defaultServer/logs/console.log

Launching defaultServer (WebSphere Application Server 8.5.5.0/wlp-1.0.3.20130510-0831) on OpenJDK 64-Bit Server VM, version 1.7.0_121-mockbuild_2016_11_01_00_38-b00 (en_US)
[AUDIT   ] CWWKE0001I: The server defaultServer has been launched.
[AUDIT   ] CWWKZ0058I: Monitoring dropins for applications. 
[AUDIT   ] CWWKF0011I: The server defaultServer is ready to run a smarter planet.

In terms of the JDK / SDK ( Java Software Development Kit ) vs. the JRE ( Java Runtime Environment ), broadly speaking, a SDK or JDK provides one with the ability to compile Java classes e.g.: -

/opt/ibm/WebSphere/AppServer/java/bin/javac HelloWorld.java

whereas a JRE provides one with the ability to execute a compiled Java class e.g. : -

/opt/ibm/WebSphere/AppServer/java/bin/java HelloWorld

which returns: -

Hello World! 

Bottom line, WLP *does* require Java; unlike WAS ND, it's not installed by default, but WLP will use a SDK/JRE if available, or you can mandate and thus install an IBM SDK of your choosing ( Java 6 or Java 7 for WLP 8.5 and Java 8 for WLP 9 - or the equivalent 16.XXX codebase ).

An OS such as Linux can have multiple SDKs or JREs installed; as per my example, I had Java 1.6 ( aka Java 6 ) and Java 1.7 ( aka Java 7 ) installed, plus various others ( I previously had WAS ND, ODM 8.7 and ODM 8.8 installed, all of which included JREs ).

So there's no issue in having multiple JREs in different paths; it's just a matter of specifying the path ….

As an update, having downloaded Java 7, as per my previous: -

-rw-r--r--@  1 davidhay  staff  1031566304 19 Jan 21:12 SDK_JAVA_TE_V7.0_1OF2_WASLC_V8.5.5.zip
-rw-r--r--@  1 davidhay  staff   734081206 19 Jan 20:36 SDK_JAVA_TE_V7.0_2OF2_WASLC_V8.5.5.zip

I've installed it using IIM and a response file - installJava7.rsp 

<?xml version='1.0' encoding='UTF-8'?>
<agent-input>
  <server>
    <repository location='/mnt/hgfs/DownloadDirector/Java' temporary='true'/>
  </server>
  <profile id='WebSphere Liberty V8.5' installLocation='/opt/ibm/WebSphere/Liberty'>
    <data key='cic.selector.arch' value='x86_64'/>
  </profile>
  <install>
    <!-- IBM WebSphere SDK Java Technology Edition Version 7.0 for Liberty 7.0.4.1 -->
    <offering profile='WebSphere Liberty V8.5' id='com.ibm.websphere.liberty.IBMJAVA.v70' version='7.0.4001.20130510_2103' features='com.ibm.sdk.7'/>
  </install>
  <preference name='com.ibm.cic.common.core.preferences.eclipseCache' value='/opt/ibm/IMShared'/>
</agent-input>

Having installed this, WLP now uses the IBM SDK rather than the system default: -

cat /opt/ibm/WebSphere/Liberty/usr/servers/defaultServer/logs/messages.log
 
********************************************************************************
product = WebSphere Application Server 8.5.5.0 (wlp-1.0.3.20130510-0831)
wlp.install.dir = /opt/ibm/WebSphere/Liberty/
java.home = /opt/ibm/WebSphere/Liberty/java/java_1.7_64/jre
java.version = 1.7.0
java.runtime = Java(TM) SE Runtime Environment (pxa6470sr4fp1ifx-20130423_02 (SR4 FP1+IV38579+IV38399+IV40208))
os = Linux (2.6.32-642.13.1.el6.x86_64; amd64) (en_US)
********************************************************************************
[1/19/17 21:14:54:717 GMT] 00000001 com.ibm.ws.logging.internal.TraceSpecification               I TRAS0018I: The trace state has been changed. The new trace state is *=info.
[1/19/17 21:14:54:749 GMT] 00000001 com.ibm.ws.kernel.launch.internal.FrameworkManager           A CWWKE0001I: The server defaultServer has been launched.
[1/19/17 21:14:56:613 GMT] 00000001 com.ibm.ws.kernel.launch.internal.FrameworkManager           I CWWKE0002I: The kernel started after 2.432 seconds
[1/19/17 21:14:56:670 GMT] 0000001d com.ibm.ws.kernel.feature.internal.FeatureManager            I CWWKF0007I: Feature update started.
[1/19/17 21:14:57:742 GMT] 00000021 com.ibm.ws.tcpchannel.internal.TCPChannel                    I CWWKO0219I: TCP Channel defaultHttpEndpoint has been started and is now listening for requests on host localhost  (IPv4: 127.0.0.1) port 9080.
[1/19/17 21:14:58:110 GMT] 00000016 com.ibm.ws.app.manager.internal.monitor.DropinMonitor        A CWWKZ0058I: Monitoring dropins for applications. 
[1/19/17 21:14:58:171 GMT] 0000001d com.ibm.ws.kernel.feature.internal.FeatureManager            I CWWKF0008I: Feature update completed in 1.556 seconds.
[1/19/17 21:14:58:171 GMT] 0000001d com.ibm.ws.kernel.feature.internal.FeatureManager            A CWWKF0011I: The server defaultServer is ready to run a smarter planet.

This is what I have installed: -

/opt/ibm/InstallationManager/eclipse/tools/imcl listInstalledPackages -features -long

/opt/ibm/InstallationManager/eclipse : com.ibm.cic.agent_1.8.5001.20161016_1705 : IBM®Installation Manager : 1.8.5.1 : 
/opt/ibm/WebSphere/Liberty : com.ibm.websphere.liberty.IBMJAVA.v70_7.0.4001.20130510_2103 : IBM WebSphere SDK Java Technology Edition Version 7.0 for Liberty : 7.0.4.1 : 
/opt/ibm/WebSphere/Liberty : com.ibm.websphere.liberty.v85_8.5.5000.20130514_1313 : IBM WebSphere Application Server Liberty Core : 8.5.5.0 : embeddablecontainer

Note that I needed to install WLP before Java 7, or I get this: -

/opt/ibm/InstallationManager/eclipse/tools/imcl -input installJava7.rsp -acceptLicense

ERROR: The following errors were generated while installing.
  CRIMA1071E ERROR: The installation package 'IBM WebSphere SDK Java Technology Edition Version 7.0 for Liberty' requires components supplied by other packages.
  The required components might be supplied by the following installation packages:
  Package: IBM WebSphere Application Server Liberty 
  Package: IBM WebSphere Application Server Liberty Trial 
  Package: IBM WebSphere Application Server Liberty - Express 
  Package: IBM WebSphere Application Server Liberty - Express Trial 
  Package: IBM WebSphere Application Server Liberty Network Deployment 
  Package: IBM WebSphere Application Server Liberty Network Deployment Trial 
  Package: IBM WebSphere Application Server Liberty Core 
  Package: IBM WebSphere Application Server Liberty for z/OS 
    ERROR: The following additional components are required in installation location 'WebSphere Liberty V8.5':
      ERROR: In root installation context:
        ERROR: Installation context "com.ibm.websphere.liberty.java.installcontext" was not resolved
    ERROR: The following additional components are required in installation location 'WebSphere Liberty V8.5':
      ERROR: In root installation context:
        ERROR: Installation context "com.ibm.websphere.liberty.java.installcontext" was not resolved
    ERROR: In root installation context:
      ERROR: Installation context "com.ibm.websphere.liberty.java.installcontext" was not resolved

If I do it the right way around: -

/opt/ibm/InstallationManager/eclipse/tools/imcl -input installWLP.rsp -acceptLicense

Installed com.ibm.websphere.liberty.v85_8.5.5000.20130514_1313 to the /opt/ibm/WebSphere/Liberty directory.

/opt/ibm/InstallationManager/eclipse/tools/imcl -input installJava7.rsp -acceptLicense

Installed com.ibm.websphere.liberty.IBMJAVA.v70_7.0.4001.20130510_2103 to the /opt/ibm/WebSphere/Liberty directory.

we're golden.

If I do both in one response file - installEverything.rsp

<?xml version='1.0' encoding='UTF-8'?>
<agent-input>
  <server>
    <repository location='/mnt/hgfs/DownloadDirector/WLP' temporary='true'/>
    <repository location='/mnt/hgfs/DownloadDirector/Java' temporary='true'/>
  </server>
  <profile id='WebSphere Liberty V8.5' installLocation='/opt/ibm/WebSphere/Liberty'>
    <data key='cic.selector.arch' value='x86_64'/>
  </profile>
  <install>
    <!-- IBM WebSphere Application Server Liberty Core 8.5.5.0 -->
    <offering profile='WebSphere Liberty V8.5' id='com.ibm.websphere.liberty.v85' version='8.5.5000.20130514_1313' features='liberty,embeddablecontainer'/>
    <offering profile='WebSphere Liberty V8.5' id='com.ibm.websphere.liberty.IBMJAVA.v70' version='7.0.4001.20130510_2103' features='com.ibm.sdk.7'/>
  </install>
  <preference name='com.ibm.cic.common.core.preferences.eclipseCache' value='/opt/ibm/IMShared'/>
</agent-input>

/opt/ibm/InstallationManager/eclipse/tools/imcl -input installEverything.rsp -acceptLicense

Installed com.ibm.websphere.liberty.v85_8.5.5000.20130514_1313 to the /opt/ibm/WebSphere/Liberty directory.
Installed com.ibm.websphere.liberty.IBMJAVA.v70_7.0.4001.20130510_2103 to the /opt/ibm/WebSphere/Liberty directory.

again, we're good to go.

WebSphere Developer Tools - Also does Liberty on Docker

Following my earlier post: -

WebSphere Developer Tools on macOS - It's been a while

I did not know ( but do now ) that the WebSphere Developer Tools also supports the WebSphere Liberty Profile on a Docker container: -

which is good to know :-)

WebSphere Developer Tools on macOS - It's been a while

I had a need to tinker with some Java in the context of WebSphere Application Server (WAS) Network Deployment (ND) 8.5.

I'm using a Mac, with WAS ND 8.5.5.11 on Red Hat Enterprise Linux 6.6 on VMware, as well as WebSphere Liberty Profile 8.5.59 on Docker natively on macOS.

 Having downloaded the latest and greatest Eclipse ( Neon ) from here ( actually from IBM Bluemix here ) I then dived across to the WebSphere Developer Tools page here: 

IBM WebSphere Application Server traditional V8.5x Developer Tools

Whilst the plugin does apparently support Neon: -

When I tried to drag/drop this into Eclipse, it didn't want to install ( and I did NOT grab the error message, more fool me ).

However, the Eclipse Market Place plugin DID provide me with the appropriate Neon version of the tools: -

Specifically, this is what I installed

IBM WebSphere Application Server traditional V8.5x Developer Tools

so we're good to go :-)

IBM Operational Decision Manager 8.7.0.0 and IBM WebSphere Application Server 8.5.5.11 - Perhaps Not BFFs

I tried / failed to install IBM Operational Decision Manager (ODM) Rules 8.7.0.0 onto a newly installed copy of WebSphere Application Server (WAS) 8.5.5.11 ( aka Fixpack 11 ) today.

This is what I was trying to install: -

/opt/ibm/InstallationManager/eclipse/tools/imcl listAvailablePackages -repositories /mnt/hgfs/Software/Repo/ODM87/Product/DecisionServerRules/,/mnt/hgfs/Software/Repo/ODM87/Product/ProfileTemplateRules

com.ibm.websphere.odm.ds.rules.v87_8.7.0.20141114_0949
com.ibm.websphere.odm.pt.rules.v87_8.7.0.20141114_1039

Having run this command: -

/opt/ibm/InstallationManager/eclipse/tools/imcl -input /mnt/hgfs/Software/Repo/ResponseFiles/installODM87.rsp -acceptLicense

I saw this: -

ERROR: The following errors were generated while installing.
  CRIMA1209E ERROR: Validation failed for property user.wodm_was_home.
    ERROR: The product is unable to determine which edition of WebSphere Application Server is installed at the selected location. Base or Network Deployment edition of version 8.5.5.0 or later is required, depending on the features selected.

    Explanation: The product is unable to determine which edition of WebSphere Application Server is installed at the selected location. Base or Network Deployment edition of version 8.5.5.0 or later is required, depending on the features selected.

    User Action: 

I checked, and ensure that WAS *was* installed: -

/opt/ibm/WebSphere/AppServer/bin/versionInfo.sh

WVER0010I: Copyright (c) IBM Corporation 2002, 2012; All rights reserved.
WVER0012I: VersionInfo reporter version 1.15.1.48, dated 2/8/12

--------------------------------------------------------------------------------
IBM WebSphere Product Installation Status Report
--------------------------------------------------------------------------------

Report at date and time January 10, 2017 1:55:15 PM GMT

Installation
--------------------------------------------------------------------------------
Product Directory        /opt/ibm/WebSphere/AppServer
Version Directory        /opt/ibm/WebSphere/AppServer/properties/version
DTD Directory            /opt/ibm/WebSphere/AppServer/properties/version/dtd
Log Directory            /home/wasadmin/var/ibm/InstallationManager/logs

Product List
--------------------------------------------------------------------------------
ND                       installed

Installed Product
--------------------------------------------------------------------------------
Name                  IBM WebSphere Application Server Network Deployment
Version               8.5.5.11
ID                    ND
Build Level           cf111649.01
Build Date            12/6/16
Package               com.ibm.websphere.ND.v85_8.5.5011.20161206_1434
Architecture          x86-64 (64 bit)
Installed Features    IBM 64-bit WebSphere SDK for Java
                      WebSphere Application Server Full Profile
                      EJBDeploy tool for pre-EJB 3.0 modules
                      Embeddable EJB container
                      Stand-alone thin clients and resource adapters

--------------------------------------------------------------------------------
End Installation Status Report
————————————————————————————————————————

I even tried installing ODM 8.7.1.1: -

/opt/ibm/InstallationManager/eclipse/tools/imcl listAvailablePackages -repositories /mnt/hgfs/Software/Repo/ODM87/Product/DecisionServerRules/,/mnt/hgfs/Software/Repo/ODM87/Product/ProfileTemplateRules,/mnt/hgfs/Software/Repo/ODM87/Fixes/DS/updates,/mnt/hgfs/Software/Repo/ODM87/Fixes/PTDS/updates

com.ibm.websphere.odm.ds.rules.v87_8.7.0.20141114_0949
com.ibm.websphere.odm.pt.rules.v87_8.7.0.20141114_1039
com.ibm.websphere.odm.ds.rules.v87_8.7.0.20141114_0949
com.ibm.websphere.odm.ds.rules.v87_8.7.1001.20151006_2017
com.ibm.websphere.odm.pt.rules.v87_8.7.0.20141114_1039
com.ibm.websphere.odm.pt.rules.v87_8.7.1001.20151007_0005

but to no avail.

I then switched back to an older version of WAS - 8.5.5.10 - and it just worked: -

/opt/ibm/InstallationManager/eclipse/tools/imcl -input /mnt/hgfs/Software/Repo/ResponseFiles/installODM87.rsp -acceptLicense

Installed com.ibm.websphere.odm.ds.rules.v87_8.7.1001.20151006_2017 to the /opt/ibm/ODM87 directory.
Installed com.ibm.websphere.odm.pt.rules.v87_8.7.1001.20151007_0005 to the /opt/ibm/ODM87 directory.

Interestingly, ODM 8.7.1 on WAS 8.5.5.11 is supported as per the System Requirements document, but I have something that works for me :-)

Interesting, WebSphere Application Server 8.5.5.11 and Java 6

I saw this: -

ERROR: Support for using Java SE 6 with WebSphere Application Server ends in April 2018.

Java SE 8 is the recommended Java SDK because it provides the latest features and security updates. You can continue to use Java SE 6, but no service can be provided after the end of support date, which could expose your environment to security risks.

You must specify the default Java SDK version on the 'user.wasjava' property. 

To install the Java 8 SDK, specify '-properties user.wasjava=java8'. 

To install the Java 6 SDK, specify '-properties user.wasjava=java6'. 

whilst trying to install the latest WebSphere Application Server (WAS) version ( 8.5.5 Fixpack 11 aka 8.5.5.11 ) today, via this command: -

/opt/ibm/InstallationManager/eclipse/tools/imcl -input /mnt/hgfs/Software/Repo/ResponseFiles/installWAS855.rsp -acceptLicense

Given that I was using a response file, it was easy enough to fix, by adding an additional data.key to installWAS855.rsp: -

…

<profile id='IBM WebSphere Application Server V8.5' installLocation='/opt/ibm/WebSphere/AppServer'>
<data key='eclipseLocation' value='/opt/ibm/WebSphere/AppServer'/>
<data key='user.import.profile' value='false'/>
<data key='cic.selector.os' value='linux'/>
<data key='cic.selector.arch' value='x86'/>
<data key='cic.selector.ws' value='gtk'/>
<data key='cic.selector.nl' value='en'/>
<data key='user.wasjava' value='java6'/>
</profile>
…

Now I see this: -


Installed com.ibm.websphere.ND.v85_8.5.5011.20161206_1434 to the /opt/ibm/WebSphere/AppServer directory.
WARNING: Your default Java SDK is Java SE 6. After April 2018, service for Java SE 6 can no longer be provided, which could expose your environment to security risks.

To stay current and reduce the risk of security exposures, update to Java SE 8 using one of the following options:
- Install WebSphere Application Server 8.5.5.11 or later with Java SE 8.
- Install the optional IBM WebSphere SDK, Java Technology Edition, Version 8 package to an existing package group that contains WebSphere Application Server Version 8.5.
- Migrate to WebSphere Application Server traditional Version 9.   

Alternatively, to use Java SE 7, install the optional IBM WebSphere SDK, Java Technology Edition, Version 7 or 7.1 package to the package group that contains WebSphere Application Server Version 8.5.

Note that I'm explicitly choosing to use Java6, other Java versions are available :-)

More background from this: -

Java SE 6 end of service in WebSphere Application Server V8.5

<snip>

WebSphere Application Server v8.5 traditional

Starting in version 8.5.5.11, you must install either Java SE 6 or Java SE 8 as the version of Java SE contained in the /java and /java64 directories in WebSphere Application Server and used by default during server and node configuration. For existing installations that are being updated, your current Java level remains the same. For example, if you previously installed version 8.5.5.10 with Java SE 6, and you update to version 8.5.5.11, your Java version remains Java SE 6. For information about how to migrate to Java SE 8, see Migrating to Java Platform, Standard Edition 7 or 8.

You can install Java SE 8 by using the GUI, the command line or by using response files. To install Java SE 8 using the command line or response files, you must specify the Java version by using the user.wasjava property.

• To install Java SE 8, specify -properties user.wasjava=java8.
• To install Java SE 6, specify -properties user.wasjava=java6.

</snip>

Synology NAS - Where's my Unzip command gorn ?

I hit a wee snag earlier today, whilst attempting to unpack some IBM software on my NAS.

Being a command-line junkie, I'd SSH'd into the box: -

Synology NAS - From My Mac, Via SSH

Synology NAS - Broke SSH but Telnet saved me

Synology NAS - More SSH Loveliness - Permissions and ACLs

and was attempting to use the unzip command, as per this example: -

unzip ../../IIM18/agent.installer.linux.gtk.x86_64_1.8.5001.20161016_1705.zip

which, alas, returned: -

-sh: unzip: command not found

Thankfully, the Synology forum came to my rescue, with this: -

Unzip?

which led me to the fact that Synology have replaced zip with 7Zip, as per this: -

which 7z

/bin/7z

so I now need to do this: -

7z x ../../IIM18/agent.installer.linux.gtk.x86_64_1.8.5001.20161016_1705.zip 

to unpack my file, and I'm good to go.

Thankfully, they've left tar alone, so I can still do this: -

tar xvf ../../ODM87/Product/DEC_SVR_RU_WIN_32_64_BITS_V8.7_ML.tar

which is nice :-)

For the record, this is what I'm running on the NAS: -

uname -a

Linux DiskStation 3.2.40 #8451 SMP Fri Dec 9 10:49:00 CST 2016 armv7l GNU/Linux synology_armadaxp_ds414

cat /etc/VERSION 

majorversion="6"
minorversion="0"
productversion="6.0.2"
buildphase="hotfix"
buildnumber="8451"
smallfixnumber="7"
builddate="2016/12/20"
buildtime="05:10:39"

IBM BPM 8.5.7 - Key Assist ( or how do I do XXX ? )

I'm not sure if it's a mild case of CDO ( surely the letters ODC aren't in order ! ), but I was struggling to align some elements in a Business Process Definition.

Thus, I was looking for the zoom controls, and stumbled across Key Assis

This is, of course, a feature of Eclipse, and I've written about similar capabilities in IBM Notes

Anyway, here's the Key Assist panes for Process Designer ( I'm using 8.5.7 on Windows 7 )

For the record, the zoom controls are right there :-)

IBM BPM 8.5.7 - Web Process Designer - On YouTube

On YouTube here: -

IBM BPM 8.5.7 Build From Scratch part 1

Intro to IBM BPM 8.5.7 - Intro, build from scratch, process modeling, variables, playbacks, snapshots, default UI, new web based process editor.

IBM Business Process Manager (BPM) 8.5.7: Building a Business Process

We will build a Mortgage Approval process application that can run in IBM BPM. We'll take an iterative approach to building the solution. It's all done in a web browser, connected to IBM BPM in an IBM cloud.

IBM Business Process Manager 8.5.7 - Web Process Designer

This caught me out, albeit briefly.

I'm using the very latest IBM BPM 8.5.7 with the December 2016 update, and am running through a tutorial: -

Hiring Tutorial: Create the process

One thing that I noticed was that, even if I created the Process Application in the Eclipse-based Process Designer tool, I was still automatically directed to the web-based Process Designer tool when I created a new Process.

I looked here: -

Where to edit Process Designer artifacts

and found this table: -

which highlighted where I was going wrong.

I was creating a Process as opposed to a Business Process Definition (BPD).

Whilst one can create a BPD using the Eclipse PD tool, one can only create a Process using the web-based PD tool.

Easy when you know how ...

Synology NAS - More SSH Loveliness - Permissions and ACLs

Following on from my earlier posts: -

Synology NAS - From My Mac, Via SSH

Synology NAS - Broke SSH but Telnet saved me

I've gone a few steps further in my understanding.

I've now got to a point where I can access the NAS using a user other than root or admin.

Having created a new user via the Web UI ( DaveHay ) which was a member of the users and administrators groups, I went through the same steps as before: -

Client-side ( macOS )

Generate a public/private key

ssh-keygen -t rsa -b 4096 -f foobar -N passw0rd

Generating public/private rsa key pair.
Your identification has been saved in foobar.
Your public key has been saved in foobar.pub.
The key fingerprint is:
SHA256:w7rpoqt07lMZNhT9GVdCOpRKEunRq9+zGb6+YHl8kC4 davidhay@Davids-GhostRider-4.local
The key's randomart image is:
+---[RSA 4096]----+
|     o*  .oo..   |
|     = +.o...    |
|    o + +o+      |
|     = + oo      |
|    . = So       |
|     + .+..      |
| . .. oE.= .     |
|. o.. .+=o+      |
|..+=.o+ .B=      |
+----[SHA256]-----+

Copy the public key to the clipboard

pbcopy < foobar.pub 

Server-side ( Synology )

Logged in as admin

ssh admin@diskstation

admin@diskstation's password: 

Switched to the root user

admin@DiskStation:~$ sudo bash
Password: 

Switched to the DaveHay user

su - DaveHay

( NOTE the above steps are required because I deliberately didn't give the user a password, as I only ever want to authenticate via a public/private key )

Create .ssh subdirectory

mkdir ~/.ssh

Create authorised keys file in .ssh

vi ~/.ssh/authorized_keys

Add public key from clipboard

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCz6Nd1Zugpjbsaz0ceF8WK5ps7SExiV6bR3ITtufFd0jp+ZyIhGJY+iRMzqslGEGcrYHGWzZRUGwq+dT4rikm/3yI2usHUI7TE2pFXS0SVI0jdsSp76Yos7lTVdcRJVlVaXG6nCKPYY3zfLrgmNXwDArYUHkVotBuKeF19lXR5Uu5DvxWUCsXz1APuRaX6oylmmk9QgZGClqdn4rrPjzKguwSZpUIOFRIfIbJiEIKvfu1vrEF45QlAoxvx4BQ0Mqew7Dv9Nt/s5ByGs7w/YHwJiWDpbGx0KCMiaeuwLjuj8N/dxfh6DIllqKzEXRCniftU6hXDULKLLoQx8WZoU90kvRLob27SjcVDrdM6C1Q0yQ2OGY0/OjKl2QjFk99LmZbCvLA5hb46eQBJviM1l9BBlf6eBq0qQtADKGV2UfZb43Z32rYObyqPqQjnfYiAk1CdECtJUCCGPdXbviPfDOYKaXgseBCnLNpnAislcmvI0YsuKKTo3xz16PFvhyJel+5EEbIpZaRQTQNDPjpXqr2pzhP5vcKuh09Z/w7lFZ0oRP47SACryYgbQzTowDthJ135kW00AsGMMEP9Yz2HjqQLdZZv0NL0KZgGIxaFHXpshPuCOWK3MmYtEqoJtcSDr++JtLU+/59/b3N+BqZxYuFSoOEUMhiee3k7VMq1ZNT5/Q== davidhay@Davids-GhostRider-4.local

Client-side ( macOS )

Attempt to connect using private key

ssh -i ~/foobar DaveHay@diskstation

which immediately prompted me for a password: -

DaveHay@diskstation's password: 

As before, I went back into the Synology, and updated the directory / file permissions for the newly created .ssh subdirectory

Server-side ( Synology )

( As DaveHay, having logged on as admin and switched user via su - DaveHay )

Check current state

ls -al -R ~

.:
total 20
drwxrwxrwx+ 3 DaveHay users 4096 Jan  5 18:40 .
drwxrwxrwx+ 7 root    root  4096 Jan  5 18:32 ..
drwxrwxrwx+ 2 DaveHay users 4096 Jan  5 18:40 .ssh
-rwxrwxrwx+ 1 DaveHay users  669 Jan  5 18:40 .viminfo

./.ssh:
total 12
drwxrwxrwx+ 2 DaveHay users 4096 Jan  5 18:40 .
drwxrwxrwx+ 3 DaveHay users 4096 Jan  5 18:40 ..
-rwxrwxrwx+ 1 DaveHay users  762 Jan  5 18:40 authorized_keys

Set the .ssh subdirectory to 700

chmod 700 ~/.ssh

Set the authorized_keys file to 644 

chmod 644 ~/.ssh/authorized_keys

Source: http://ss64.com/bash/chmod.html

Check new state

ls -al -R ~

/var/services/homes/DaveHay:
total 20
drwxrwxrwx+ 3 DaveHay users 4096 Jan  5 18:40 .
drwxrwxrwx+ 7 root    root  4096 Jan  5 18:32 ..
drwx------  2 DaveHay users 4096 Jan  5 18:40 .ssh
-rwxrwxrwx+ 1 DaveHay users  669 Jan  5 18:40 .viminfo

/var/services/homes/DaveHay/.ssh:
total 12
drwx------  2 DaveHay users 4096 Jan  5 18:40 .
drwxrwxrwx+ 3 DaveHay users 4096 Jan  5 18:40 ..
-rw-r--r--  1 DaveHay users  762 Jan  5 18:40 authorized_keys

Attempt to connect using private key

ssh -i ~/foobar DaveHay@diskstation

which immediately prompted me for a password: -

DaveHay@diskstation's password: 

:-(

I added some debugging: -

ssh -v -i ~/foobar DaveHay@diskstation

which showed: -

…

debug1: Authentications that can continue: publickey,password
debug1: Offering RSA public key: foobar
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: password
DaveHay@diskstation's password: 
…

ssh -vv -i ~/foobar DaveHay@diskstation

which showed: -

…

debug1: Authentications that can continue: publickey,password
debug1: Offering RSA public key: foobar
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey,password
debug2: we did not send a packet, disable method
debug1: Next authentication method: password
DaveHay@diskstation's password: 
...

ssh -vv -i ~/foobar DaveHay@diskstation

…

debug1: Authentications that can continue: publickey,password
debug1: Offering RSA public key: foobar
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred: ,password
debug3: authmethod_is_enabled password
debug1: Next authentication method: password
DaveHay@diskstation's password: 

…

Something I read online made me think about extended attributes, over and above the usual Unix permissions.

I re-visited the current state: -

ls -al -R ~

/var/services/homes/DaveHay:
total 20
drwxrwxrwx+ 3 DaveHay users 4096 Jan  5 18:40 .
drwxrwxrwx+ 7 root    root  4096 Jan  5 18:32 ..
drwx------  2 DaveHay users 4096 Jan  5 18:40 .ssh
-rwxrwxrwx+ 1 DaveHay users  669 Jan  5 18:40 .viminfo

/var/services/homes/DaveHay/.ssh:
total 12
drwx------  2 DaveHay users 4096 Jan  5 18:40 .
drwxrwxrwx+ 3 DaveHay users 4096 Jan  5 18:40 ..
-rw-r--r--  1 DaveHay users  762 Jan  5 18:40 authorized_keys

Yes, it was the additional plus character that made me wonder; +

As root I checked the permissions for the DaveHay user: -

ls -ale /volume1/homes/DaveHay/

total 20
drwx------+ 3 DaveHay users 4096 Jan  5 18:40 .
 [0] user:DaveHay:allow:rwxpdDaARWcCo:fd-- (level: 0)
 [1] user:DaveHay:allow:rwxpdDaARWc--:fd-- (level: 1)
 [2] user::allow:rwxpdDaARWc--:fd-- (level: 1)
 [3] user::allow:rwxpdDaARWc--:fd-- (level: 1)

d--x--x--x+ 7 root    root  4096 Jan  5 18:32 ..
drwx------  2 DaveHay users 4096 Jan  5 18:40 .ssh
-rwx------+ 1 DaveHay users  669 Jan  5 18:40 .viminfo
 [0] user:DaveHay:allow:rwxpdDaARWcCo:---- (level: 1)
 [1] user:DaveHay:allow:rwxpdDaARWc--:---- (level: 2)
 [2] user::allow:rwxpdDaARWc--:---- (level: 2)
 [3] user::allow:rwxpdDaARWc--:---- (level: 2)

and used chmod to recursively set ALL the permissions on the DaveHay user's home directory: -

chmod -R 700 /volume1/homes/DaveHay/

which removes the special attributes ( I think these may be the ACLs added by the NAS itself when I created the new user ), and validated: -

ls -ale /volume1/homes/DaveHay/

total 20
drwx------  3 DaveHay users 4096 Jan  5 18:40 .
d--x--x--x+ 7 root    root  4096 Jan  5 18:32 ..
drwx------  2 DaveHay users 4096 Jan  5 18:40 .ssh
-rwx------  1 DaveHay users  669 Jan  5 18:40 .viminfo

In other words, the extended attributes for the user have gone, apart from the parent directory ( /volume1/homes ) which is fine.

I re-tested my SSH connection: -

ssh -i foobar DaveHay@diskstation

Enter passphrase for key 'foobar': 
DaveHay@DiskStation:~$ 

In other words, I'm only now being presented with a request for the passphrase for the private key, rather than the password for the DaveHay user.

So, it was a long journey, but an enjoyable one :-)

As ever, #LifeIsGood

Synology NAS - Broke SSH but Telnet saved me

I was fiddling about with SSH on my Synology DS414 NAS, with the intention of setting up password-less authentication ( about which more to follow ).

Having updated the SSHD configuration ( /etc/ssh/sshd_config ), I then found that I wasn't able to restart the SSH service, via the following command: -

synoservicectl --restart sshd

Whilst SSH stopped, it refused to restart.

Which meant that I was locked out ….

Happily, I still had access via the Web UI meaning that I could enable the oh-so-secure Telnet service

Whilst I wouldn't use Telnet ordinarily, it provided me with a nice little "back door" ( perhaps an inappropriate choice of words ).

I was then able to telnet into the box, and sort out the configuration i.e. reverse out the changes to sshd_config.

However, things still weren't rosy. Whilst SSHD was apparently running, I was seeing: -

Permission denied, please try again.
Connection to diskstation closed.

when I tried to ssh in.

Thankfully, I still had the Telnet option, so I checked the normal Unix logs: -

dmesg

[15363.072410] init: sshd main process (32286) terminated with status 255
[15363.079132] init: sshd main process ended, respawning
[15363.543730] init: sshd main process (32305) terminated with status 255
[15363.550438] init: sshd main process ended, respawning
[15364.004256] init: sshd main process (32324) terminated with status 255
[15364.010969] init: sshd main process ended, respawning
[15364.469980] init: sshd main process (32343) terminated with status 255
[15364.476658] init: sshd main process ended, respawning
[15364.933677] init: sshd main process (32363) terminated with status 255
[15364.940675] init: sshd main process ended, respawning
[15365.414906] init: sshd main process (32384) terminated with status 255
[15365.421621] init: sshd respawning too fast, stopped

which wasn't totally helpful.

So I revisited the Web UI, and saw this: -

Yep, you guessed it, I'd disabled the service, so it wasn't ever going to start :-)

So, I'm now back to a working SSH situation - time to go play with SSL keys again :-)

That'll be the next blog post ….

IBM Notes on Mac - Dude, where's my toolbar ?

It's been a while since last I posted about IBM Notes, even though it's the one tool that I use each and every day.

This time around, it's the toolbars that we use within Notes that were causing me grief.

For the record, I'm using Notes 9.0.1 on macOS Sierra

I'm not a big fan of out-of-context toolbars ( I find those in Office 2016 to be rather annoying ) but I do like certain toolbar features when I need them.

Specifically I'm looking at the Editing toolbar that appears when one is …. editing a document in Notes

What I was finding was that the toolbar did NOT appear unless I unchecked the Show Toolbars Only When Editing option via View > Toolbar

This meant that the toolbar appeared ALL the time, even when I was NOT editing

I did note that, with Show Toolbars Only When Editing checked, the option Hide All Toolbars was also automatically checked, which felt contra-intuitive.

Then I dug further ….

Looking in Notes Preferences ( [cmd] [,]  or IBM Notes > Preferences ), I found this

Once I checked Show toolbar that applies to my current task I got precisely what I wanted

To summarise, this is what I now have

which is nice

IBM BPM - Tinkering with Process Instance Cleanup - aka BPMProcessInstancesPurge

Following my previous post: -

IBM BPM - Tinkering with Process Instance Cleanup - aka BPMProcessInstancesCleanup

I've had a play with the BPMProcessInstancesPurge command.

It's WAY more powerful, and definitely a case of Caveat Emptor, Your Mileage May Vary and Here Be Dragons.

This command allows one to remove *ALL* instances from a Process Server, regardless of status.

But it does require one to Use The Force.

Now read on ….

Validate status of instances - active, completed, terminated, suspended

db2 "select inst.EXECUTION_STATUS, count(*) as total from LSW_BPD_INSTANCE inst, LSW_TASK task where task.BPD_INSTANCE_ID = inst.BPD_INSTANCE_ID group by inst.EXECUTION_STATUS"

EXECUTION_STATUS TOTAL      

              1.           1

              2.           1

              4.           1

              6.           1

Clear ALL Terminated instances

print AdminTask.BPMProcessInstancesPurge('[-instanceStatus TERMINATED]')

The BPMProcessInstancesPurge command passed.

Check results in AppCluster logs

tail -f /opt/ibm/WebSphereProfiles/AppSrv01/logs/AppClusterMember1/SystemOut.log

[04/01/17 17:04:25:612 GMT] 000001eb CommandServic I PALAdminCommands otherProcessInstancesPurge Entering

[04/01/17 17:04:25:616 GMT] 000001eb Log           I   PAL: About to delete the process instances matching the filter criteria: 

Statuses: [4]

Maximum duration: 0

Transaction slice: 10

[04/01/17 17:04:25:616 GMT] 000001eb Log           I   PAL: Number of qualifying instances before deletion: 1

[04/01/17 17:04:25:687 GMT] 000001eb Log           I   PAL: A deletion job is running. Progress: Deleted 1 instance(s).

[04/01/17 17:04:25:687 GMT] 000001eb Log           I   PAL: Every instance has been deleted. The deletion job finished.

[04/01/17 17:04:25:691 GMT] 000001eb Log           I   PAL: Successfully deleted the process instances matching the filter criteria: 

Statuses: [4]

Maximum duration: 0

Transaction slice: 10

[04/01/17 17:04:25:692 GMT] 000001eb Log           I   PAL: Number of qualifying instances before deletion: 1

[04/01/17 17:04:25:695 GMT] 000001eb Log           I   PAL: Number of qualifying instances after deletion: 0

[04/01/17 17:04:25:698 GMT] 000001eb CommandServic I PALAdminCommands otherProcessInstancesPurge Exiting

[04/01/17 17:04:34:110 GMT] 00000124 ProcessIndexB I   CWLLG0757I: The purge process was completed successfully. 1 tasks and 1 instances were successfully removed from the search index.

Validate that there are no longer any Terminated instances ( status code 4 )

db2 "select inst.EXECUTION_STATUS, count(*) as total from LSW_BPD_INSTANCE inst, LSW_TASK task where task.BPD_INSTANCE_ID = inst.BPD_INSTANCE_ID group by inst.EXECUTION_STATUS"

EXECUTION_STATUS TOTAL      

              1.           1

              2.           1

              6.           1

Attempt to clear ALL instances

print AdminTask.BPMProcessInstancesPurge('[-instanceStatus ALL]')

The BPMProcessInstancesPurge command passed.

wsadmin>print AdminTask.BPMProcessInstancesPurge('[-instanceStatus ALL]')

WASX7015E: Exception running command: "AdminTask.BPMProcessInstancesPurge('[-instanceStatus ALL]')"; exception information:

java.lang.Exception: java.lang.Exception: The instance status 'ALL' is a non end state. The '-force' flag is mandatory to delete instances in this state.

Attempt to clear ALL instances - use the force

print AdminTask.BPMProcessInstancesPurge('[-instanceStatus ALL -force]')

The BPMProcessInstancesPurge command passed.

Check results in AppCluster logs

tail -f /opt/ibm/WebSphereProfiles/AppSrv01/logs/AppClusterMember1/SystemOut.log

[04/01/17 17:08:42:748 GMT] 000001ec CommandServic I PALAdminCommands otherProcessInstancesPurge Entering

[04/01/17 17:08:42:753 GMT] 000001ec Log           I   PAL: About to delete the process instances matching the filter criteria: 

Statuses: [2, 4, 3, 1, 5, 6]

Maximum duration: 0

Transaction slice: 10

[04/01/17 17:08:42:754 GMT] 000001ec Log           I   PAL: Number of qualifying instances before deletion: 3

[04/01/17 17:08:42:963 GMT] 000001ec Log           I   PAL: A deletion job is running. Progress: Deleted 3 instance(s).

[04/01/17 17:08:42:963 GMT] 000001ec Log           I   PAL: Every instance has been deleted. The deletion job finished.

[04/01/17 17:08:42:970 GMT] 000001ec Log           I   PAL: Successfully deleted the process instances matching the filter criteria: 

Statuses: [2, 4, 3, 1, 5, 6]

Maximum duration: 0

Transaction slice: 10

[04/01/17 17:08:42:970 GMT] 000001ec Log           I   PAL: Number of qualifying instances before deletion: 3

[04/01/17 17:08:42:975 GMT] 000001ec Log           I   PAL: Number of qualifying instances after deletion: 0

[04/01/17 17:08:42:978 GMT] 000001ec CommandServic I PALAdminCommands otherProcessInstancesPurge Exiting

[04/01/17 17:08:49:143 GMT] 00000124 ProcessIndexB I   CWLLG0757I: The purge process was completed successfully. 3 tasks and 3 instances were successfully removed from the search index.

Validate the database

db2 "select inst.EXECUTION_STATUS, count(*) as total from LSW_BPD_INSTANCE inst, LSW_TASK task where task.BPD_INSTANCE_ID = inst.BPD_INSTANCE_ID group by inst.EXECUTION_STATUS"

EXECUTION_STATUS TOTAL      

  0 record(s) selected.

It goes without saying that the force should be used with caution - with great power, comes great responsibility ( just because you can, doesn't mean you should )

IBM BPM - Tinkering with Process Instance Cleanup - aka BPMProcessInstancesCleanup

The context here is that I'm looking at the mechanisms for house-keeping an IBM Business Process Manager (BPM) environment, and am specifically focused upon process instances.

I'm using BPM Advanced 8.5.7 ( aka  8.5.7.201612 ).

So we have some work for the database administrator ( here I'm using DB2 10.5 on Linux, other databases are available ): -

Connect to BPM DB

db2 connect to bpmdb

Set Current Schema to db2user1

db2 set current schema db2user1

Check Execution Status

db2 "select STATUS_ID, NAME from LSW_BPD_STATUS_CODES"

       1. Active                                                                                                                                                                                                  

       2. Completed                                                                                                                                                                                               

       3. Failed                                                                                                                                                                                                  

       4. Terminated                                                                                                                                                                                              

       5. Did_not_Start                                                                                                                                                                                           

       6. Suspended                         

Summarise Status

db2 "select inst.EXECUTION_STATUS, count(*) as total from LSW_BPD_INSTANCE inst, LSW_TASK task where task.BPD_INSTANCE_ID = inst.BPD_INSTANCE_ID group by inst.EXECUTION_STATUS"

EXECUTION_STATUS TOTAL      

---------------- -----------

              1.           1

              2.           1

              4.           1

              6.           1

This means that we have four instances in total; one Active, one Completed, one Terminated and one Suspended.

Check for Active Instances

db2 "select BPD_INSTANCE_ID,SNAPSHOT_ID from LSW_BPD_INSTANCE where EXECUTION_STATUS = '1'"

BPD_INSTANCE_ID SNAPSHOT_ID                         

            11. 3053758b-8764-4c1c-854a-ad0982143313

Check for Completed Instances

db2 "select BPD_INSTANCE_ID,SNAPSHOT_ID from LSW_BPD_INSTANCE where EXECUTION_STATUS = '2'"

BPD_INSTANCE_ID SNAPSHOT_ID                         

             4. 3053758b-8764-4c1c-854a-ad0982143313

Check for Terminated Instances

db2 "select BPD_INSTANCE_ID,SNAPSHOT_ID from LSW_BPD_INSTANCE where EXECUTION_STATUS = '4'"

BPD_INSTANCE_ID SNAPSHOT_ID                         

             6. 3053758b-8764-4c1c-854a-ad0982143313

             7. 3053758b-8764-4c1c-854a-ad0982143313

             8. 3053758b-8764-4c1c-854a-ad0982143313

             9. 3053758b-8764-4c1c-854a-ad0982143313

Check for Suspended Instances

db2 "select BPD_INSTANCE_ID,SNAPSHOT_ID from LSW_BPD_INSTANCE where EXECUTION_STATUS = '6'"

BPD_INSTANCE_ID SNAPSHOT_ID                         

            10. f1659d94-2365-4903-8a90-9fa62f3ccc97

Correlate Snapshots to Process Applications

db2 "select NAME,ACRONYM from LSW_SNAPSHOT where SNAPSHOT_ID = '3053758b-8764-4c1c-854a-ad0982143313'"

NAME ACRONYM

Procurement Sample v8570 PSV8570 

db2 "select NAME,ACRONYM from LSW_SNAPSHOT where SNAPSHOT_ID = 'f1659d94-2365-4903-8a90-9fa62f3ccc97'"

NAME ACRONYM

Responsive Hiring Sample v8570 RHSV8_1                                                                                                                                                                                                                                                         

With the details of the snapshots, applications and instances, we now pass control to the BPM administrator: -

Start WSAdmin Client ( note; this is against the AppCluster SOAP port )

/opt/ibm/WebSphereProfiles/Dmgr01/bin/wsadmin.sh -host bpm857.uk.ibm.com -port 8881 -lang jython -user wasadmin -password passw0rd

List Installed Process Applications

print AdminTask.BPMListProcessApplications()

...

Name: Hiring Sample

Acronym: HSS

Description: Hiring Sample

Toolkit: false

Name: Hiring Sample Advanced

Acronym: HSAV1

Description: 

Toolkit: false

Name: Procurement Sample

Acronym: STPPS1

Description: 

Toolkit: false

...

Show Snapshot Details of Chosen Process Application

print AdminTask.BPMShowProcessApplication('[-containerAcronym HSAV1]')

Name: Hiring Sample Advanced

Acronym: HSAV1

Description: 

Toolkit: false

Tracks:

Track Name: Main

Track Acronym: Main

Default: true

Tip: 

Created On: 2016-12-30 14:01:25.987

Created By: User.9

State: State[Inactive]

Capability: Capability[Advanced]

No of running instances: 0

List of Snapshots: 

Name: Advanced Hiring Sample v8570

Acronym: AHSV857

Created On: 2016-12-30 14:01:25.987

Created By: User.9

Is Default: false

State: State[Undeployed]

Capability: Capability[Advanced]

No of running instances: 0

print AdminTask.BPMShowSnapshot('[-containerAcronym HSAV1 -containerSnapshotAcronym AHSV857 ]')

Name: Advanced Hiring Sample v8570

Acronym: AHSV857

Created On: 2016-12-30 14:01:25.987

Created By: User.9

Is Default: false

State: State[Undeployed]

Capability: Capability[Advanced]

Theme: BPM Theme

No of running instances: 0

Dependency:

Toolkit Acronym: TWSYS

Toolkit Name: System Data

Toolkit Track Acronym: Main

Toolkit Track Name: Main

Snapshot Name: 8.5.7.0

Snapshot Acronym: 8.5.7.0

Created On: 2016-12-30 14:00:39.906

Created By: User.9

Dependency:

Toolkit Acronym: SYSC

Toolkit Name: Coaches

Toolkit Track Acronym: Main

Toolkit Track Name: Main

Snapshot Name: 8.5.7.0

Snapshot Acronym: 8.5.7.0

Created On: 2016-12-30 14:00:47.195

Created By: User.9

Dependency:

Toolkit Acronym: SYSRC

Toolkit Name: Responsive Coaches

Toolkit Track Acronym: Main

Toolkit Track Name: Main

Snapshot Name: 8.5.7.0

Snapshot Acronym: 8.5.7.0

Created On: 2016-12-30 14:00:50.024

Created By: User.9

print AdminTask.BPMShowProcessApplication('[-containerAcronym HSS]')

Name: Hiring Sample

Acronym: HSS

Description: Hiring Sample

Toolkit: false

Tracks:

Track Name: Main

Track Acronym: Main

Default: true

Tip: 

Created On: 2016-12-30 14:01:22.959

Created By: User.9

State: State[Inactive]

Capability: Capability[Standard]

No of running instances: 0

List of Snapshots: 

Name: Responsive Hiring Sample v8570

Acronym: RHSV8_1

Created On: 2016-12-30 14:01:22.959

Created By: User.9

Is Default: false

State: State[Inactive]

Capability: Capability[Standard]

No of running instances: 0

print AdminTask.BPMShowSnapshot('[-containerAcronym HSS -containerSnapshotAcronym RHSV8_1 ]')

Name: Responsive Hiring Sample v8570

Acronym: RHSV8_1

Created On: 2016-12-30 14:01:22.959

Created By: User.9

Is Default: false

State: State[Inactive]

Capability: Capability[Standard]

Theme: BPM Theme

No of running instances: 0

Dependency:

Toolkit Acronym: TWSYS

Toolkit Name: System Data

Toolkit Track Acronym: Main

Toolkit Track Name: Main

Snapshot Name: 8.5.7.0

Snapshot Acronym: 8.5.7.0

Created On: 2016-12-30 14:00:39.906

Created By: User.9

Dependency:

Toolkit Acronym: SYSRC

Toolkit Name: Responsive Coaches

Toolkit Track Acronym: Main

Toolkit Track Name: Main

Snapshot Name: 8.5.7.0

Snapshot Acronym: 8.5.7.0

Created On: 2016-12-30 14:00:50.024

Created By: User.9

print AdminTask.BPMShowProcessApplication('[-containerAcronym STPPS1]')

Name: Procurement Sample

Acronym: STPPS1

Description: 

Toolkit: false

Tracks:

Track Name: Main

Track Acronym: Main

Default: true

Tip: 

Created On: 2016-12-30 14:01:30.466

Created By: User.9

State: State[Inactive]

Capability: Capability[Advanced]

No of running instances: 0

List of Snapshots: 

Name: Procurement Sample v8570

Acronym: PSV8570

Created On: 2016-12-30 14:01:30.466

Created By: User.9

Is Default: false

State: State[Undeployed]

Capability: Capability[Advanced]

No of running instances: 0

print AdminTask.BPMShowSnapshot('[-containerAcronym STPPS1 -containerSnapshotAcronym PSV8570 ]')

Name: Procurement Sample v8570

Acronym: PSV8570

Created On: 2016-12-30 14:01:30.466

Created By: User.9

Is Default: false

State: State[Undeployed]

Capability: Capability[Advanced]

Theme: BPM Theme

No of running instances: 0

Dependency:

Toolkit Acronym: TWSYS

Toolkit Name: System Data

Toolkit Track Acronym: Main

Toolkit Track Name: Main

Snapshot Name: 8.5.7.0

Snapshot Acronym: 8.5.7.0

Created On: 2016-12-30 14:00:39.906

Created By: User.9

Dependency:

Toolkit Acronym: SYSRC

Toolkit Name: Responsive Coaches

Toolkit Track Acronym: Main

Toolkit Track Name: Main

Snapshot Name: 8.5.7.0

Snapshot Acronym: 8.5.7.0

Created On: 2016-12-30 14:00:50.024

Created By: User.9

Run Instance Cleanup for Hire Sample ( note; all instances, regardless of status )

print AdminTask.BPMProcessInstancesCleanup('[-containerAcronym HSS -containerSnapshotAcronym RHSV8_1 ]')

The BPMProcessInstancesCleanup command passed.

Check results in AppCluster Logs

tail -f /opt/ibm/WebSphereProfiles/AppSrv01/logs/AppClusterMember1/SystemOut.log

[04/01/17 09:58:05:333 GMT] 000001eb CommandServic I PALAdminCommands otherProcessInstancesCleanup Entering

[04/01/17 09:58:05:355 GMT] 000001eb Log           I   PAL: About to start the BPMProcessInstancesCleanup with parameters:  -containerAcronym Hiring Sample -containerSnapshotAcronym RHSV8_1 -instanceStatus ALL  -maximumduration 0 -transactionSlice 1

[04/01/17 09:58:05:355 GMT] 000001eb Log           I   PAL: Number of qualifying instances before deletion: 1

[04/01/17 09:58:05:543 GMT] 000001eb Log           I   PAL: A deletion job is running. Progress: Deleted 1 instance(s).

[04/01/17 09:58:05:544 GMT] 000001eb Log           I   PAL: Every instance has been deleted. The deletion job finished.

[04/01/17 09:58:05:564 GMT] 000001eb Log           I   PAL: Successfully ran the BPMProcessInstancesCleanup command with parameters:  -containerAcronym Hiring Sample -containerSnapshotAcronym RHSV8_1 -instanceStatus ALL  -maximumduration 0 -transactionSlice 1

[04/01/17 09:58:05:564 GMT] 000001eb Log           I   PAL: Number of qualifying instances before deletion: 1

[04/01/17 09:58:05:569 GMT] 000001eb Log           I   PAL: Number of qualifying instances after deletion: 0

[04/01/17 09:58:05:573 GMT] 000001eb CommandServic I PALAdminCommands otherProcessInstancesCleanup Exiting

[04/01/17 09:58:15:288 GMT] 00000124 ProcessIndexB I   CWLLG0757I: The purge process was completed successfully. 2 tasks and 1 instances were successfully removed from the search index.

Run Instance Cleanup for Hire Sample Advanced ( note; all instances, regardless of status )

print AdminTask.BPMProcessInstancesCleanup('[-containerAcronym HSAV1 -containerSnapshotAcronym AHSV857 ]')

The BPMProcessInstancesCleanup command passed.

Check results in AppCluster Logs

tail -f /opt/ibm/WebSphereProfiles/AppSrv01/logs/AppClusterMember1/SystemOut.log

[04/01/17 10:03:46:320 GMT] 000001ec CommandServic I PALAdminCommands otherProcessInstancesCleanup Entering

[04/01/17 10:03:46:340 GMT] 000001ec Log           I   PAL: About to start the BPMProcessInstancesCleanup with parameters:  -containerAcronym Hiring Sample Advanced -containerSnapshotAcronym AHSV857 -instanceStatus ALL  -maximumduration 0 -transactionSlice 1

[04/01/17 10:03:46:340 GMT] 000001ec Log           I   PAL: Number of qualifying instances before deletion: 1

[04/01/17 10:03:46:422 GMT] 000001ec Log           I   PAL: A deletion job is running. Progress: Deleted 1 instance(s).

[04/01/17 10:03:46:423 GMT] 000001ec Log           I   PAL: Every instance has been deleted. The deletion job finished.

[04/01/17 10:03:46:433 GMT] 000001ec Log           I   PAL: Successfully ran the BPMProcessInstancesCleanup command with parameters:  -containerAcronym Hiring Sample Advanced -containerSnapshotAcronym AHSV857 -instanceStatus ALL  -maximumduration 0 -transactionSlice 1

[04/01/17 10:03:46:434 GMT] 000001ec Log           I   PAL: Number of qualifying instances before deletion: 1

[04/01/17 10:03:46:439 GMT] 000001ec Log           I   PAL: Number of qualifying instances after deletion: 0

[04/01/17 10:03:46:444 GMT] 000001ec CommandServic I PALAdminCommands otherProcessInstancesCleanup Exiting

[04/01/17 10:03:55:324 GMT] 00000124 ProcessIndexB I   CWLLG0757I: The purge process was completed successfully. 3 tasks and 1 instances were successfully removed from the search index.

Run Instance Cleanup for Procurement Sample; only for Terminated instances

print AdminTask.BPMProcessInstancesCleanup('[-containerAcronym STPPS1 -containerSnapshotAcronym PSV8570 -instanceStatus CANCELED -instanceID [ 6 7 8 9 ]]')

The BPMProcessInstancesCleanup command passed.

Check results in AppCluster Logs

tail -f /opt/ibm/WebSphereProfiles/AppSrv01/logs/AppClusterMember1/SystemOut.log

[04/01/17 16:18:02:778 GMT] 000001ec CommandServic I PALAdminCommands otherProcessInstancesCleanup Entering

[04/01/17 16:18:02:800 GMT] 000001ec Log           I   PAL: About to start the BPMProcessInstancesCleanup with parameters:  -containerAcronym Procurement Sample -containerSnapshotAcronym PSV8570 -instanceStatus CANCELED  -instanceID [6, 7, 8, 9] -maximumduration 0 -transactionSlice 1

[04/01/17 16:18:02:801 GMT] 000001ec Log           I   PAL: Number of qualifying instances before deletion: 4

[04/01/17 16:18:02:883 GMT] 000001ec Log           I   PAL: A deletion job is running. Progress: Deleted 1 instance(s).

[04/01/17 16:18:02:907 GMT] 000001ec Log           I   PAL: A deletion job is running. Progress: Deleted 1 instance(s).

[04/01/17 16:18:02:919 GMT] 000001ec Log           I   PAL: A deletion job is running. Progress: Deleted 1 instance(s).

[04/01/17 16:18:02:931 GMT] 000001ec Log           I   PAL: A deletion job is running. Progress: Deleted 1 instance(s).

[04/01/17 16:18:02:931 GMT] 000001ec Log           I   PAL: Every instance has been deleted. The deletion job finished.

[04/01/17 16:18:02:947 GMT] 000001ec Log           I   PAL: Successfully ran the BPMProcessInstancesCleanup command with parameters:  -containerAcronym Procurement Sample -containerSnapshotAcronym PSV8570 -instanceStatus CANCELED  -instanceID [6, 7, 8, 9] -maximumduration 0 -transactionSlice 1

[04/01/17 16:18:02:948 GMT] 000001ec Log           I   PAL: Number of qualifying instances before deletion: 4

[04/01/17 16:18:02:954 GMT] 000001ec Log           I   PAL: Number of qualifying instances after deletion: 0

[04/01/17 16:18:02:959 GMT] 000001ec CommandServic I PALAdminCommands otherProcessInstancesCleanup Exiting

[04/01/17 16:18:08:806 GMT] 00000124 ProcessIndexB I   CWLLG0757I: The purge process was completed successfully. 4 tasks and 4 instances were successfully removed from the search index.

Run Instance Cleanup for Responsive Hiring Sample; only for Suspended Instances

print AdminTask.BPMProcessInstancesCleanup('[-containerAcronym HSS -containerSnapshotAcronym RHSV8_1 -instanceStatus ALL -instanceID [ 10 ]]')

The BPMProcessInstancesCleanup command passed.

Check results in AppCluster Logs

tail -f /opt/ibm/WebSphereProfiles/AppSrv01/logs/AppClusterMember1/SystemOut.log

[04/01/17 16:35:30:072 GMT] 000001eb CommandServic I PALAdminCommands otherProcessInstancesCleanup Entering

[04/01/17 16:35:30:089 GMT] 000001eb Log           I   PAL: The process instance with the ID '10' is in the state '6' which is not a specified state and will not be deleted.

[04/01/17 16:35:30:096 GMT] 000001eb CommandServic I PALAdminCommands otherProcessInstancesCleanup Exiting

[04/01/17 16:35:38:927 GMT] 00000124 ProcessIndexB I   CWLLG0757I: The purge process was completed successfully. 0 tasks and 0 instances were successfully removed from the search index.

This shows that one can NOT cleanup instances that are suspended; the instance either needs to be resumed, and then completed, OR terminated, before cleanup can take place

Run Instance Cleanup for Procurement Sample; only for Active Instances

print AdminTask.BPMProcessInstancesCleanup('[-containerAcronym STPPS1 -containerSnapshotAcronym PSV8570 -instanceStatus ALL -instanceID [ 11 ]]')

The BPMProcessInstancesCleanup command passed.

Check results in AppCluster Logs

tail -f /opt/ibm/WebSphereProfiles/AppSrv01/logs/AppClusterMember1/SystemOut.log

[04/01/17 16:46:24:475 GMT] 000001eb CommandServic I PALAdminCommands otherProcessInstancesCleanup Entering

[04/01/17 16:46:24:489 GMT] 000001eb Log           I   PAL: The process instance with the ID '11' is in the state '1' which is not a specified state and will not be deleted.

[04/01/17 16:46:24:494 GMT] 000001eb CommandServic I PALAdminCommands otherProcessInstancesCleanup Exiting

[04/01/17 16:46:34:029 GMT] 00000124 ProcessIndexB I   CWLLG0757I: The purge process was completed successfully. 0 tasks and 0 instances were successfully removed from the search index.

This shows that one can NOT cleanup instances that are active; the instance either needs to be completed, OR terminated, before cleanup can take place

My next mission is to look at the BPMProcessInstancesPurge command, which supersedes BPMProcessInstancesCleanup, which has been deprecated.

That's tomorrow's job ….

With thanks to this: -

BPMProcessInstancesCleanup command (deprecated)

BPM Tasks and Process Instances Clean up – purging data from Process Server DB