RAR - Why does my profile augmentation fail ?

During a build of an IBM Operational Decision Manager (ODM) Rules 8.9.2.1 environment, I was trying ( and failing ) to augment an existing WebSphere Application Server (WAS) profile - the Deployment Manager - Dmgr01 - with ODM Decision Server (RES) capabilities: -

/opt/ibm/WebSphere/AppServer/bin/manageprofiles.sh -augment -profileName Dmgr01 -templatePath /opt/ibm/WebSphere/AppServer/profileTemplates/odm/decisionserver/management/ -odmHome /opt/ibm/ODM89/

simply returned: -

...
/opt/ibm/WebSphere/AppServer/bin/manageprofiles.sh -augment -profileName Dmgr01 -templatePath /opt/ibm/WebSphere/AppServer/profileTemplates/odm/decisionserver/management/ -odmHome /opt/ibm/ODM89/
INSTCONFFAILED: Profile augmentation failed. For more information, consult /opt/ibm/WebSphere/AppServer/logs/manageprofiles/Dmgr01_augment.log.
…

I checked the log file: -

view /opt/ibm/WebSphere/AppServer/logs/manageprofiles/Dmgr01_augment.log

which showed: -

...
<record>
  <date>2018-10-31T09:47:29</date>
  <millis>1540979249438</millis>
  <sequence>599</sequence>
  <logger>com.ibm.ws.install.configmanager.actionengine.ant.utils.ANTLogToCmtLogAdapter</logger>
  <level>WARNING</level>
  <class>com.ibm.ws.install.configmanager.actionengine.ant.utils.ANTLogToCmtLogAdapter</class>
  <method>messageLogged</method>
  <thread>1</thread>
  <message>wsadmin task failed with return code :105</message>
</record>
...
<record>
  <date>2018-10-31T09:47:24</date>
  <millis>1540979244994</millis>
  <sequence>576</sequence>
  <logger>com.ibm.ws.install.configmanager.actionengine.ant.utils.AntTaskToLogToConfigManagersLogFiles</logger>
  <level>INFO</level>
  <class>/opt/ibm/WebSphere/AppServer/profileTemplates/odm/decisionserver/management/actions/decisionServerSanityCheck.ant:7: </class>
  <method>decisionServerSanityCheck</method>
  <thread>1</thread>
  <message>Calling decisionServerSanityCheck.py with ODM_HOME=/opt/ibm/ODM89/</message>
</record>
...
<record>
  <date>2018-10-31T09:47:24</date>
  <millis>1540979244994</millis>
  <sequence>577</sequence>
  <logger>com.ibm.ws.install.configmanager.actionengine.ant.utils.AntTaskToLogToConfigManagersLogFiles</logger>
  <level>INFO</level>
  <class>/opt/ibm/WebSphere/AppServer/profileTemplates/odm/decisionserver/management/actions/decisionServerSanityCheck.ant:9: </class>
  <method>decisionServerSanityCheck</method>
  <thread>1</thread>
  <message>${WSADMIN_EXECUTABLE} -conntype NONE -f /opt/ibm/WebSphere/AppServer/profileTemplates/odm/decisionserver/management/actions/scripts/decisionServerSanityCheck.py /opt/ibm/WebSphere/AppServer/profileTemplates/odm/decisionserver/management/actions/scripts /opt/ibm/ODM89/</message>
</record>
...
<record>
  <date>2018-10-31T09:47:29</date>
  <millis>1540979249457</millis>
  <sequence>664</sequence>
  <logger>com.ibm.ws.install.configmanager.ConfigManager</logger>
  <level>WARNING</level>
  <class>com.ibm.ws.install.configmanager.ConfigManager</class>
  <method>executeAllActionsFound</method>
  <thread>1</thread>
  <message>Fatal configuration action failed: com.ibm.ws.install.configmanager.actionengine.ConfigAction-/opt/ibm/WebSphere/AppServer/profileTemplates/odm/decisionserver/management/actions/decisionServerSanityCheck.ant</message>
</record>
<record>
  <date>2018-10-31T09:47:29</date>
  <millis>1540979249457</millis>
  <sequence>665</sequence>
  <logger>com.ibm.ws.install.configmanager.ConfigManager</logger>
  <level>SEVERE</level>
  <class>com.ibm.ws.install.configmanager.ConfigManager</class>
  <method>launch</method>
  <thread>1</thread>
  <message>ConfigManager action execution failed on a fatal action</message>
</record>
<record>
  <date>2018-10-31T09:47:29</date>
  <millis>1540979249457</millis>
  <sequence>666</sequence>
  <logger>com.ibm.ws.install.configmanager.ConfigManager</logger>
  <level>INFO</level>
  <class>com.ibm.ws.install.configmanager.ConfigManager</class>
  <method>launch</method>
  <thread>1</thread>
  <message>Returning with return code: INSTCONFFAILED</message>
</record>
<record>
  <date>2018-10-31T09:47:29</date>
  <millis>1540979249457</millis>
  <sequence>667</sequence>
  <logger>com.ibm.wsspi.profile.WSProfile</logger>
  <level>SEVERE</level>
  <class>com.ibm.wsspi.profile.WSProfile</class>
  <method>doAugment</method>
  <thread>1</thread>
  <message>Augmentation of profile Dmgr01 at path /opt/ibm/WebSphere/AppServer/profiles/Dmgr01 using profile template /opt/ibm/WebSphere/AppServer/profileTemplates/odm/decisionserver/management failed.</message>
</record>
…

I dug into the Deployment Manager logs further, and found this: -

cat  /opt/ibm/WebSphere/AppServer/logs/manageprofiles/Dmgr01/odm/decisionServerSanityCheck.wsadmin.log

which was WAY more useful: -

...
WASX7357I: By request, this scripting client is not connected to any server process. Certain configuration and application operations will be available in local mode.
WASX7303I: The following options are passed to the scripting environment and are available as arguments that are stored in the argv variable: "[/opt/ibm/WebSphere/AppServer/profileTemplates/odm/decisionserver/management/actions/scripts, /opt/ibm/ODM89]"
Processing sanatiy check before augmentation
ERROR: The specified ODM_HOME does not look correct, unable to find jrules-res-xu-WAS85.rar!
Traceback (most recent call last):
  File "<string>", line 41, in ?
RuntimeError: ERROR: The specified ODM_HOME does not look correct!
WASX7017E: Exception received while running file "/opt/ibm/WebSphere/AppServer/profileTemplates/odm/decisionserver/management/actions/scripts/decisionServerSanityCheck.py"; exception information: com.ibm.bsf.BSFException: exception from Jython:
Traceback (innermost last):
  File "<string>", line 87, in ?
RuntimeError: An error occur during sanity checks
…

This is what I have installed: -

/opt/ibm/InstallationManager/eclipse/tools/imcl listInstalledPackages -features -long

/opt/ibm/InstallationManager/eclipse : com.ibm.cic.agent_1.8.9001.20180709_1302 : IBM® Installation Manager : 1.8.9.1 : 
/opt/ibm/WebSphere/AppServer : com.ibm.websphere.ND.v85_8.5.5014.20180802_1018 : IBM WebSphere Application Server Network Deployment : 8.5.5.14 : com.ibm.sdk.6_64bit,ejbdeploy,embeddablecontainer,thinclient
/opt/ibm/ODM89 : com.ibm.websphere.odm.dc.v89_8.9.2001.20180723_1215 : Decision Center : 8.9.2.1 : Decision Center,com.ibm.wbdm.dts.was.feature
/opt/ibm/ODM89 : com.ibm.websphere.odm.ds.rules.v89_8.9.2001.20180723_1224 : Decision Server Rules : 8.9.2.1 : com.ibm.wds.rules.res.feature,com.ibm.wds.rules.res.was.feature
/opt/ibm/ODM89 : com.ibm.websphere.odm.pt.dc.v89_8.9.2001.20180723_1240 : Decision Center profile templates for WebSphere Application Server   : 8.9.2.1 : 
/opt/ibm/ODM89 : com.ibm.websphere.odm.pt.rules.v89_8.9.2001.20180723_1243 : Decision Server Rules profile templates for WebSphere Application Server  : 8.9.2.1 : 

and yet this is what I have in terms of WAS85 artefacts for the RES: -

ls /opt/ibm/ODM89/executionserver/applicationservers/WebSphere85/

-rw-r--r-- 1 wasadmin wasadmins 14674397 Jul 20 18:23 jrules-res-htds-WAS85.ear
-rw-r--r-- 1 wasadmin wasadmins 30996584 Jul 20 18:23 jrules-res-management-WAS85.ear
-rw-r--r-- 1 wasadmin wasadmins  2136520 Jul 20 18:23 jrules-res-mdb-WAS85.jar
-rw-r--r-- 1 wasadmin wasadmins  1559934 Jul 20 18:23 jrules-res-session-ejb3-WAS85.jar
-rw-r--r-- 1 wasadmin wasadmins 67130079 Jul 20 18:23 jrules-ssp-WAS85.ear

Note the absence of said RAR file.

I dug around within the source media, and found this: -

./disk1/ProfileTemplateRules/native/com.ibm.websphere.odm.rules.res.was_8.9.1000.20171018-1755.zip

which I unzipped to a temporary directory: -

unzip ./disk1/ProfileTemplateRules/native/com.ibm.websphere.odm.rules.res.was_8.9.1000.20171018-1755.zip -d /tmp/snafu

Archive:  ./disk1/ProfileTemplateRules/native/com.ibm.websphere.odm.rules.res.was_8.9.1000.20171018-1755.zip
   creating: /tmp/snafu/executionserver/
   creating: /tmp/snafu/executionserver/applicationservers/
   creating: /tmp/snafu/executionserver/applicationservers/WebSphere85/
  inflating: /tmp/snafu/executionserver/applicationservers/WebSphere85/jrules-res-htds-WAS85.ear  
  inflating: /tmp/snafu/executionserver/applicationservers/WebSphere85/jrules-res-management-WAS85.ear  
  inflating: /tmp/snafu/executionserver/applicationservers/WebSphere85/jrules-res-mdb-WAS85.jar  
  inflating: /tmp/snafu/executionserver/applicationservers/WebSphere85/jrules-res-session-ejb3-WAS85.jar  
  inflating: /tmp/snafu/executionserver/applicationservers/WebSphere85/jrules-res-xu-WAS85.rar  
  inflating: /tmp/snafu/executionserver/applicationservers/WebSphere85/jrules-ssp-WAS85.ear  

I then grabbed the missing RAR - jrules-res-xu-WAS85.rar - and placed it here: -

/opt/ibm/ODM89/executionserver/applicationservers/WebSphere85/

-rw-r--r-- 1 wasadmin wasadmins 14674397 Jul 20 18:23 jrules-res-htds-WAS85.ear
-rw-r--r-- 1 wasadmin wasadmins 30996584 Jul 20 18:23 jrules-res-management-WAS85.ear
-rw-r--r-- 1 wasadmin wasadmins  2136520 Jul 20 18:23 jrules-res-mdb-WAS85.jar
-rw-r--r-- 1 wasadmin wasadmins  1559934 Jul 20 18:23 jrules-res-session-ejb3-WAS85.jar
-rw-r--r-- 1 wasadmin wasadmins 25865277 Oct 31 10:50 jrules-res-xu-WAS85.rar
-rw-r--r-- 1 wasadmin wasadmins 67130079 Jul 20 18:23 jrules-ssp-WAS85.ear

This time around, the augmentation succeeds: -

/opt/ibm/WebSphere/AppServer/bin/manageprofiles.sh -augment -profileName Dmgr01 -templatePath /opt/ibm/WebSphere/AppServer/profileTemplates/odm/decisionserver/management/ -odmHome /opt/ibm/ODM89/

INSTCONFSUCCESS: Profile augmentation succeeded.

Now I need to work out what happened during the binary installation …….