Thursday, 16 October 2014

Storing transaction and compensation logs in a relational database for high availability and disaster recovery in IBM Business Process Manager

This is especially relevant for me right now, as I'm involved in not one, not two, but three projects where HA *AND* DR are both in scope for IBM BPM etc.

Learn how to leverage the latest WebSphere® Application Server feature by configuring an IBM® Business Process Manager transaction and compensation logs into a relational database for high availability and disaster recovery purposes. This content is part of the IBM Business Process Management Journal.

at No comments:

IBM BPM 8.5.0.1 Advanced - More about BPM > PDW Communication

On the back of: -


for the record, this is what I changed: -

Add cell-level configuration file 101Custom.xml

cd /opt/ibm/WebSphereProfiles/Dmgr02/config/cells/PSCell1/nodes/AppSrv02Node/servers/AppClusterMember1/process-server/config

vi 101Custom.xml

Contains: -

<properties>
<!-- Added by Dave Hay, IBM, to disable Process Server to Performance Data Warehouse communication - 18/09/2014 -->
        <common>
                <performance-server-communication>
                        <enabled merge="replace">false</enabled>
                </performance-server-communication>
        </common>
</properties>

Synchronise Node from Cell-level configuration

/opt/ibm/WebSphereProfiles/Dmgr02/bin/wsadmin.sh -lang jython -user wasadmin -password <<removed>> -c "AdminNodeManagement.syncActiveNodes()"

Start Process Server Deployment Environment

/opt/ibm/WebSphere/AppServer/bin/BPMConfig.sh -start -profile Dmgr02 -de PSDeployEnv -username deAdmin -password <<removed>>

Validate new configuration for PDW

view /opt/ibm/WebSphereProfiles/AppSrv02/config/cells/PSCell1/nodes/AppSrv02Node/servers/AppClusterMember1/process-server/TeamWorksConfiguration.running.xml

Now contains: -

...
    <performance-server-communication>
      <enabled>false</enabled>
...

I also needed to update the cluster template version of this file, so that new cluster members would get the same configuration.

Therefore, I have placed a copy of 101Custom.xml here: -

/opt/ibm/WebSphereProfiles/Dmgr02/config/cells/PSCell1/nodes/AppSrv02Node/servers/AppClusterMember1/process-server/config/101Custom.xml

Just thought you ought to know ....
at No comments:

ACID - it's all about transactionality ...

Atomicity

Atomicity requires that each transaction be "all or nothing": if one part of the transaction fails, the entire transaction fails, and the database state is left unchanged. An atomic system must guarantee atomicity in each and every situation, including power failures, errors, and crashes. To the outside world, a committed transaction appears (by its effects on the database) to be indivisible ("atomic"), and an aborted transaction does not happen.

Consistency

The consistency property ensures that any transaction will bring the database from one valid state to another. Any data written to the database must be valid according to all defined rules, including constraints, cascades, triggers, and any combination thereof. This does not guarantee correctness of the transaction in all ways the application programmer might have wanted (that is the responsibility of application-level code) but merely that any programming errors cannot result in the violation of any defined rules.

Isolation

The isolation property ensures that the concurrent execution of transactions result in a system state that would be obtained if transactions were executed serially, i.e. one after the other. Providing isolation is the main goal of concurrency control. Depending on concurrency control method, the effects of an incomplete transaction might not even be visible to another transaction.

Durability

Durability means that once a transaction has been committed, it will remain so, even in the event of power loss, crashes, or errors. In a relational database, for instance, once a group of SQL statements execute, the results need to be stored permanently (even if the database crashes immediately thereafter). To defend against power loss, transactions (or their effects) must be recorded in a non-volatile memory.

at No comments:

It's been too long - IBM Tivoli Access Manager for e-business ( WebSEAL ) ... and openLDAP - what can possibly go wrong ?

So I haven't installed/used IBM Tivoli Access Manager for e-business ( WebSEAL ) properly since ~2008, although I've worked with a number of clients utilising this secure reverse proxy access management solution.

Last night, I decided to rectify that, and install/configure TAMeB on a VM on my Mac.

Me being me, I decided not to bother with documentation or too many pre-requisites, so I downloaded the software ( plus fixes, taking me to version 6.1.1.8 ), and try and configure it against ... openLDAP. Last time around, I did the right thing and used a "real" user identity management solution, IBM Tivoli Directory Server.

So, after much hacking around, I got to a bit of a blocker.

As part of the installation, one is required to run the following command: -

./install_ammgr -console

to install AND configure TAM.

This, in part, runs the following command: -

/opt/PolicyDirector/sbin/PDMgr_config

This runs for a while and then fails with: -

Unable to verify the management domain location DN in the
LDAP server: (secAuthority=Default).
If the location does not exist on the server, create it,
otherwise specify a different location that does exist.

...

2014-10-15-21:20:23.563+01:00I----- 0x16B480C9 IRAapi ERROR rgy ira ira_domain.c 1424 0xf77e46c0
HPDRG0201E   Error code 0x22 was received from the LDAP server. Error text: "Invalid DN syntax".

Whist this was obviously an LDAP issue, I wasn't too sure precisely what the root cause was.

Thankfully, some digging about in various internal/external fora gave me these two life-saver commands: -

export LDAP_DEBUG=65535 
export LDAP_DEBUG_FILE=/tmp/ldap_debug_file 

Having run these inside the shell from which I then ran PDMgr_config, I got a nice trace log - /tmp/ldap_debug_file - which contained, in part: -

T-143694144:    +---------------------------------------------------------+
T-143694144:    |0000|30840000 01370201 07688400 00012E04|0....7...h......|
T-143694144:    |0010|31736563 41757468 6F726974 793D4465|1secAuthority=De|
T-143694144:    |0020|6661756C 742C6F75 3D757365 72732C64|fault,ou=users,d|
T-143694144:    |0030|633D756B 2C64633D 69626D2C 64633D63|c=uk,dc=ibm,dc=c|
T-143694144:    |0040|6F6D3084 000000F5 30840000 0086040B|om0.....0.......|
T-143694144:    |0050|6F626A65 6374636C 61737331 84000000|objectclass1....|
T-143694144:    |0060|73041073 65634175 74686F72 69747949|s..secAuthorityI|
T-143694144:    |0070|6E666F04 12654170 706C6963 6174696F|nfo..eApplicatio|
T-143694144:    |0080|6E537973 74656D04 07655379 7374656D|nSystem..eSystem|
T-143694144:    |0090|04116369 6D4C6F67 6963616C 456C656D|..cimLogicalElem|
T-143694144:    |00A0|656E7404 1763696D 4D616E61 67656453|ent..cimManagedS|
T-143694144:    |00B0|79737465 6D456C65 6D656E74 04116369|ystemElement..ci|
T-143694144:    |00C0|6D4D616E 61676564 456C656D 656E7404|mManagedElement.|
T-143694144:    |00D0|03746F70 30840000 001D040C 73656341|.top0.......secA|
T-143694144:    |00E0|7574686F 72697479 31840000 00090407|uthority1.......|
T-143694144:    |00F0|44656661 756C7430 84000000 14040776|Default0.......v|
T-143694144:    |0100|65727369 6F6E3184 00000005 0403362E|ersion1.......6.|
T-143694144:    |0110|30308400 00002604 0B696E73 74616C6C|00....&..install|
T-143694144:    |0120|44617465 31840000 00130411 32303134|Date1.......2014|
T-143694144:    |0130|31303135 32303430 32312E30 5A      |1015204021.0Z   |
T-143694144:    +---------------------------------------------------------+

and: -

T-143694144:    +---------------------------------------------------------+
T-143694144:    |OSet| Address = 08656C00  Length = 0016 |     ASCII      |
T-143694144:    +---------------------------------------------------------+
T-143694144:    |0000|02010769 110A0122 0400040A 696E7661|...i..."....inva|
T-143694144:    |0010|6C696420 444E                      |lid DN          |
T-143694144:    +---------------------------------------------------------+

This made me think, and look for references to secAuthority=Default online, from whence I found: -



both of which made me realise that TAMeB is looking for a particular suffix/object class, which, of course, doesn't exist, by default, in openLDAP.

Now I'm not sure whether I'll continue to hack around to make TAMeB work with openLDAP, or follow the path of least resistance and install ITDS .........

However, it's good to know about that LDAP debug string, as I'm SURE to need it again in the future ....
at No comments:

Wednesday, 15 October 2014

IBM Integration Bus and WebSphere Service Registry and Repository

Looking at the integration between IIB and WSRR ( which runs on WAS 8.5.5 ), so have a nice new learning curve or two.

Here's a bit of ( for me ) pre-reading: -


Combining IBM Integration Bus (IIB) with WebSphere Service Registry and Repository (WSRR) lets you dynamically retrieve service metadata from WSRR and use it to modify message flow behavior at runtime. This article series shows you how to integrate the two products, and provides samples that address several significant business problems. Part 7 describes the configuration and behavior of the cache used by the Endpoint Lookup and Registry Lookup nodes, including how the cache supports efficient look-up of previously retrieved WSRR artifacts, and how to make this data available to production systems.

This is part of a series: -

Combining IBM Integration Bus (IIB) with WebSphere Service Registry and Repository (WSRR) lets you dynamically retrieve service metadata from WSRR and use it to modify message flow behavior at runtime. This article series shows you how to integrate the two products, and provides samples that address several significant business problems. Part 1 describes a number of IIB/WSRR integration scenarios, shows you how to configure IIB to communicate with WSRR using a number of different nodes, and describes resources that will be used in Parts 2 through 9 to demonstrate IIB/WSRR integration capabilities.

Combining IBM Integration Bus (IIB) with WebSphere Service Registry and Repository (WSRR) lets you dynamically retrieve service metadata from WSRR and use it to modify message flow behavior at runtime. This article series shows you how to integrate the two products, and provides samples that address several significant business problems. Part 2 describes the Enpoint Lookup and Registry Lookup nodes in detail.


Combining IBM Integration Bus (IIB) with WebSphere Service Registry and Repository (WSRR) lets you dynamically retrieve service metadata from WSRR and use it to modify message flow behavior at runtime. This article series shows you how to integrate the two products, and provides samples that address several significant business problems. Part 3 shows you how to use the Endpoint Lookup node to dynamically retrieve service metadata from WSRR and use this metadata to route service requests at run time, including the ability to route around problem endpoints when an error occurs.


Combining IBM Integration Bus (IIB) with WebSphere Service Registry and Repository (WSRR) lets you dynamically retrieve service metadata from WSRR and use it to modify message flow behavior at runtime. This article series shows you how to integrate the two products, and provides samples that address several significant business problems. Part 4 shows you how to use the Registry Lookup node to dynamically retrieve XSL transforms from WSRR and apply them to messages in a flow at runtime.

Integrating IBM Integration Bus with WebSphere Service Registry and Repository: Part 5: Performing SLA checks at runtime and registry lookup scenarios

Combining IBM Integration Bus (IIB) with WebSphere Service Registry and Repository (WSRR) lets you dynamically retrieve service metadata from WSRR and use it to modify message flow behavior at runtime. This article series shows you how to integrate the two products, and provides samples that address several significant business problems. Part 5 shows you how to use the Registry Lookup node to dynamically retrieve service metadata from WSRR, and use this metadata to check that a service consumer is authorized to invoke the target service.

Combining IBM Integration Bus (IIB) with WebSphere Service Registry and Repository (WSRR) lets you dynamically retrieve service metadata from WSRR and use it to modify message flow behavior at runtime. This article series shows you how to integrate the two products, and provides samples that address several significant business problems. Part 6 shows you how to use the Endpoint Lookup and Registry Lookup nodes to implement a message flow that can act as a service gateway in your SOA environment, analyzing the service request and then querying WSRR to determine how to route the request to the correct service provider.

Combining IBM Integration Bus (IIB) with WebSphere Service Registry and Repository (WSRR) lets you dynamically retrieve service metadata from WSRR and use it to modify message flow behavior at runtime. This article series shows you how to integrate the two products, and provides samples that address several significant business problems. Part 7 describes the configuration and behavior of the cache used by the Endpoint Lookup and Registry Lookup nodes, including how the cache supports efficient look-up of previously retrieved WSRR artifacts, and how to make this data available to production systems.


Combining IBM Integration Bus with WSRR lets you dynamically retrieve service metadata from WSRR and use it to modify message flow behavior at runtime. This article series shows you how to integrate the two products, and provides samples that address several significant business problems. Part 8 shows you how to use the HTTP Request node to dynamically retrieve service metadata from WSRR and use it to check that a service consumer is authorized to invoke the target service.

Combining IBM Integration Bus (IIB) with WebSphere Service Registry and Repository (WSRR) lets you dynamically retrieve service metadata from WSRR and use it to modify message flow behavior at runtime. This article series shows you how to integrate the two products, and provides samples that address several significant business problems. Part 9 shows you how to create and manage IIB Workload Management (WLM) policies in WSRR, and how to use the JMS Input node in a message flow to automatically synchronize policy changes with IIB.


at No comments:

Tuesday, 14 October 2014

Using Jython to list ports within a WebSphere Application Server cell

To be 100% clear, I did NOT NOT NOT invent this script.

The script came from a rather nice chap called Steve Robinson, who's posted it on his site here: -


I've just happily used it with WebSphere Application Server 8.5.5.2, as part of a post-installation documentation exercise ( ensuring that the deliverable ties up with the up-front design ).

Here we go: -

/opt/IBM/WebSphere/AppServer/profiles/Dmgr01/bin/wsadmin.sh -lang jython -conntype NONE -f getports.jy 

WASX7357I: By request, this scripting client is not connected to any server process. Certain configuration and application operations will be available in local mode.
AppSrv01Node ORB_LISTENER_ADDRESS 2809
AppSrv01Node CSIV2_SSL_MUTUALAUTH_LISTENER_ADDRESS 9900
AppSrv01Node CSIV2_SSL_SERVERAUTH_LISTENER_ADDRESS 9202
AppSrv01Node NODE_DISCOVERY_ADDRESS 9201
AppSrv01Node NODE_IPV6_MULTICAST_DISCOVERY_ADDRESS 9353
AppSrv01Node NODE_MULTICAST_DISCOVERY_ADDRESS 7272
AppSrv01Node SAS_SSL_SERVERAUTH_LISTENER_ADDRESS 5001
AppSrv01Node BOOTSTRAP_ADDRESS 5000
AppSrv01Node DCS_UNICAST_ADDRESS 9901
AppSrv01Node SOAP_CONNECTOR_ADDRESS 8878
AppSrv01Node IPC_CONNECTOR_ADDRESS 9629
AppSrv01Node XDAGENT_PORT 7062
AppSrv01Node OVERLAY_TCP_LISTENER_ADDRESS 11004
AppSrv01Node OVERLAY_UDP_LISTENER_ADDRESS 11003
AppSrv01Node BOOTSTRAP_ADDRESS 9810
AppSrv01Node SOAP_CONNECTOR_ADDRESS 8880
AppSrv01Node ORB_LISTENER_ADDRESS 9101
AppSrv01Node SAS_SSL_SERVERAUTH_LISTENER_ADDRESS 9404
AppSrv01Node CSIV2_SSL_SERVERAUTH_LISTENER_ADDRESS 9405
AppSrv01Node CSIV2_SSL_MUTUALAUTH_LISTENER_ADDRESS 9406
AppSrv01Node WC_adminhost 9061
AppSrv01Node WC_defaulthost 9080
AppSrv01Node DCS_UNICAST_ADDRESS 9354
AppSrv01Node WC_adminhost_secure 9044
AppSrv01Node WC_defaulthost_secure 9443
AppSrv01Node SIP_DEFAULTHOST 5060
AppSrv01Node SIP_DEFAULTHOST_SECURE 5061
AppSrv01Node OVERLAY_UDP_LISTENER_ADDRESS 11007
AppSrv01Node OVERLAY_TCP_LISTENER_ADDRESS 11008
AppSrv01Node IPC_CONNECTOR_ADDRESS 9633
AppSrv01Node SIB_ENDPOINT_ADDRESS 7276
AppSrv01Node SIB_ENDPOINT_SECURE_ADDRESS 7286
AppSrv01Node SIB_MQ_ENDPOINT_ADDRESS 5558
AppSrv01Node SIB_MQ_ENDPOINT_SECURE_ADDRESS 5578
AppSrv01Node BOOTSTRAP_ADDRESS 9811
AppSrv01Node SOAP_CONNECTOR_ADDRESS 8881
AppSrv01Node ORB_LISTENER_ADDRESS 9102
AppSrv01Node SAS_SSL_SERVERAUTH_LISTENER_ADDRESS 9407
AppSrv01Node CSIV2_SSL_SERVERAUTH_LISTENER_ADDRESS 9408
AppSrv01Node CSIV2_SSL_MUTUALAUTH_LISTENER_ADDRESS 9409
AppSrv01Node WC_adminhost 9062
AppSrv01Node WC_defaulthost 9081
AppSrv01Node DCS_UNICAST_ADDRESS 9355
AppSrv01Node WC_adminhost_secure 9045
AppSrv01Node WC_defaulthost_secure 9444
AppSrv01Node SIP_DEFAULTHOST 5062
AppSrv01Node SIP_DEFAULTHOST_SECURE 5063
AppSrv01Node OVERLAY_UDP_LISTENER_ADDRESS 11009
AppSrv01Node OVERLAY_TCP_LISTENER_ADDRESS 11010
AppSrv01Node IPC_CONNECTOR_ADDRESS 9634
AppSrv01Node SIB_ENDPOINT_ADDRESS 7278
AppSrv01Node SIB_ENDPOINT_SECURE_ADDRESS 7287
AppSrv01Node SIB_MQ_ENDPOINT_ADDRESS 5559
AppSrv01Node SIB_MQ_ENDPOINT_SECURE_ADDRESS 5579
AppSrv01Node WEBSERVER_ADDRESS 8443
AppSrv01Node WEBSERVER_ADMIN_ADDRESS 8008
Dmgr CELL_DISCOVERY_ADDRESS 7277
Dmgr WC_adminhost_secure 9043
Dmgr DCS_UNICAST_ADDRESS 9352
Dmgr XDAGENT_PORT 7060
Dmgr OVERLAY_UDP_LISTENER_ADDRESS 11005
Dmgr OVERLAY_TCP_LISTENER_ADDRESS 11006
Dmgr STATUS_LISTENER_ADDRESS 9420
Dmgr BOOTSTRAP_ADDRESS 9809
Dmgr IPC_CONNECTOR_ADDRESS 9632
Dmgr SOAP_CONNECTOR_ADDRESS 8879
Dmgr ORB_LISTENER_ADDRESS 9100
Dmgr SAS_SSL_SERVERAUTH_LISTENER_ADDRESS 9401
Dmgr CSIV2_SSL_MUTUALAUTH_LISTENER_ADDRESS 9402
Dmgr CSIV2_SSL_SERVERAUTH_LISTENER_ADDRESS 9403
Dmgr WC_adminhost 9060
Dmgr DataPowerMgr_inbound_secure 5555

If you want the actual script, please visit Steve's site: -

at No comments:

Red Hat Enterprise Linux 6.5 - Creating a local Yum Repository

More of the same: -


but using RHEL 6.5.

Here's the shortened version: -

mount /dev/cdrom /mnt/

mount: block device /dev/sr0 is write-protected, mounting read-only

cat /etc/redhat-release

Red Hat Enterprise Linux Server release 6.5 (Santiago)

rpm -ivh /mnt/Packages/deltarpm-3.5-0.5.20090913git.el6.x86_64.rpm

warning: /mnt/Packages/deltarpm-3.5-0.5.20090913git.el6.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY
Preparing...                ########################################### [100%]
   1:deltarpm               ########################################### [100%]

rpm -ivh /mnt/Packages/python-deltarpm-3.5-0.5.20090913git.el6.x86_64.rpm 

warning: /mnt/Packages/python-deltarpm-3.5-0.5.20090913git.el6.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY
Preparing...                ########################################### [100%]
   1:python-deltarpm        ########################################### [100%]

rpm -ivh /mnt/Packages/createrepo-0.9.9-18.el6.noarch.rpm 

warning: /mnt/Packages/createrepo-0.9.9-18.el6.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY
Preparing...                ########################################### [100%]
   1:createrepo             ########################################### [100%]

rpm --import /mnt/RPM-GPG-KEY-redhat-beta
rpm --import /mnt/RPM-GPG-KEY-redhat-release

mkdir /var/repo
mkdir /var/repo/rhel65
cd /mnt
cd /var/repo/rhel65/
cp -R /mnt/Packages/* .

cd /var/repo/
createrepo .

Spawning worker 0 with 3763 pkgs
Workers Finished
Gathering worker results

Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete

cd /etc/yum.repos.d/
vi server.repo 

[server]
gpgcheck=1
name=Red Hat Enterprise Linux 6.5
baseurl=file:///var/repo

 yum list

...
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Installed Packages
ConsoleKit.x86_64                      0.4.1-3.el6                       @anaconda-RedHatEnterpriseLinux-201311111358.x86_64/6.5
ConsoleKit-libs.x86_64                 0.4.1-3.el6                       @anaconda-RedHatEnterpriseLinux-201311111358.x86_64/6.5
MAKEDEV.x86_64                         3.24-6.el6                        @anaconda-RedHatEnterpriseLinux-201311111358.x86_64/6.5
Red_Hat_Enterprise_Linux-Release_Notes-6-en-US.noarch
                                       5-2.el6                           @anaconda-RedHatEnterpriseLinux-201311111358.x86_64/6.5
...
yum-presto.noarch                      0.6.2-1.el6                         server                                                 
zenity.x86_64                          2.28.0-1.el6                        server                                                 
zlib.i686                              1.2.3-29.el6                        server                                                 
zlib-devel.i686                        1.2.3-29.el6                        server                                                 
zlib-devel.x86_64                      1.2.3-29.el6                        server                                                 
zsh.x86_64                             4.3.10-7.el6                        server        
...

umount /mnt
yum install xauth

Loaded plugins: product-id, security, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package xorg-x11-xauth.x86_64 1:1.0.2-7.1.el6 will be installed
--> Processing Dependency: libXmuu.so.1()(64bit) for package: 1:xorg-x11-xauth-1.0.2-7.1.el6.x86_64
--> Running transaction check
---> Package libXmu.x86_64 0:1.1.1-2.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package               Arch          Version                Repository     Size
================================================================================
Installing:
 xorg-x11-xauth        x86_64        1:1.0.2-7.1.el6        server         35 k
Installing for dependencies:
 libXmu                x86_64        1.1.1-2.el6            server         66 k

Transaction Summary
================================================================================
Install       2 Package(s)

Total download size: 101 k
Installed size: 216 k
Is this ok [y/N]: y
Downloading Packages:
--------------------------------------------------------------------------------
Total                                            29 MB/s | 101 kB     00:00     
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Warning: RPMDB altered outside of yum.
  Installing : libXmu-1.1.1-2.el6.x86_64                                    1/2 
  Installing : 1:xorg-x11-xauth-1.0.2-7.1.el6.x86_64                        2/2 
  Verifying  : libXmu-1.1.1-2.el6.x86_64                                    1/2 
  Verifying  : 1:xorg-x11-xauth-1.0.2-7.1.el6.x86_64                        2/2 

Installed:
  xorg-x11-xauth.x86_64 1:1.0.2-7.1.el6                                         

Dependency Installed:
  libXmu.x86_64 0:1.1.1-2.el6                                                   

Complete!

END OF LINE
at No comments:
Subscribe to: Posts (Atom)

Note to self - Firefox and local connections

 Whilst trying to hit my NAS from Firefox on my Mac, I kept seeing errors such as:- Unable to connect Firefox can’t establish a connection t...