Saturday, 18 April 2015

The WebSphere Contrarian: Are you getting the most out of virtualization?

This popped up via IBM developerWorks on Twitter - @developerWorks: -

Much has happened in the area of virtualization since the topic was first discussed in this column, so this is a good time to revisit (or re-introduce) virtualization and all its flavors to help you determine which type of virtualization could benefit your organization the most, and how to make it the most effective. This content is part of the IBM WebSphere Developer Technical Journal.

Wednesday, 15 April 2015

How to manually generate a Heapdump in WebSphere on Windows


How to generate a heapdump without waiting for an OutOfMemoryError condition to occur.

Resolving the problem

Occasionally a heapdump needs to be generated without waiting for an OutOfMemoryError condition to occur. 

For example: an OutOfMemoryError will probably cause a system outage. On a production system this is not a desired event. If we manually create a heapdump before the OutOfMemoryError generates one, then the application server can be recycled and we will get a heapdump without an unscheduled outage. 

The manually generated heapdump will not be as useful as one which was generated by an OutOfMemoryError, but sometimes it can be of some use. The longer we can wait before generating the heapdump, the more likely the heapdump will contain the objects causing the OutOfMemory. If the heapdump is generated too soon, the largest memory user is not likely to be the same largest memory user when the heap is completely full.

IBM Business Monitor - Yet more on Cognos dependencies

This follows an earlier post: -

I saw a similar issue: -

Caused by: java.lang.UnsatisfiedLinkError: JCAM_Crypto_JNI ( cannot open shared object file: No such file or directory)

with Cognos 10.2.2 underlying IBM Business Monitor 8.5.6, during an installation on Red Hat Enterprise Linux 6.6 earlier today.

I went through the same problem determination regime as described in my earlier blog post, and found that the missing link was

Therefore, the solution was the same: -

yum install -y 

which returned: -

Running Transaction
Installing : libstdc++-4.4.7-11.el6.i686 1/1 
Verifying : libstdc++-4.4.7-11.el6.i686 1/1 

libstdc++.i686 0:4.4.7-11.el6 



and all was then well.


This is something that I've recently added to my reading list.

It's by an IBM colleague, Mr Kim Clark Esq., and dates from 2010.

However, it's directly relevant to my current project, in the context of Long-Running and Straight Through business processing, hosted on IBM Business Process Manager Advanced 8.5.5.

I really really need to learn this stuff: -

and also this: -


Tuesday, 14 April 2015

IBM Business Monitor 8.5.6 on Red Hat Enterprise Linux 6.6 - Forked

Again, THIS is why I blog.

One of my colleagues saw issues with IBM Business Monitor 8.5.6 on a Red Hat Enterprise Linux 6.6 VM, including core dumps and OutOfMemory exceptions.

Given that I'd created the base VM, and also installed all of the IBM middleware components ( this is for an enablement event that we're co-delivering next week ), I wanted to get to the bottom of it.

I took his build notes, and went through the same process.

And I saw much the same thing, specifically: -

[4/14/15 20:06:37:172 BST] 00000001 ContainerHelp E   WSVR0102E: An error occurred stopping,
java.lang.OutOfMemoryError: Failed to create a thread: retVal -1073741830, errno 11

[14/04/15 20:30:24:939 BST] 00000001 CommandMgr    E   ADMF0008E: Command Framework failed to initialize or cannot create CommandMgr in server mode. Root cause is java.lang.OutOfMemoryError: Failed to create a thread: retVal -1073741830, errno 11

in the AppClusterMember's SystemOut.log and: -

JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2015/04/14 20:29:55 - please wait.
JVMDUMP032I JVM requested System dump using '/opt/IBM/WebSphere/AppServer/profiles/BAMCell1AppSrv01/core.20150414.202955.3420.0001.dmp' in response to an event
JVMDUMP012E Error in System dump: insufficient system resources to generate dump, errno=11 "Resource temporarily unavailable"
JVMDUMP032I JVM requested Heap dump using '/opt/IBM/WebSphere/AppServer/profiles/BAMCell1AppSrv01/' in response to an event
JVMDUMP010I Heap dump written to /opt/IBM/WebSphere/AppServer/profiles/BAMCell1AppSrv01/
JVMDUMP032I JVM requested Java dump using '/opt/IBM/WebSphere/AppServer/profiles/BAMCell1AppSrv01/javacore.20150414.202955.3420.0003.txt' in response to an event
JVMDUMP010I Java dump written to /opt/IBM/WebSphere/AppServer/profiles/BAMCell1AppSrv01/javacore.20150414.202955.3420.0003.txt
JVMDUMP032I JVM requested Snap dump using '/opt/IBM/WebSphere/AppServer/profiles/BAMCell1AppSrv01/Snap.20150414.202955.3420.0004.trc' in response to an event
UTE001: Error starting trace thread for "Snap Dump Thread": -1
JVMDUMP010I Snap dump written to /opt/IBM/WebSphere/AppServer/profiles/BAMCell1AppSrv01/Snap.20150414.202955.3420.0004.trc

in the Node Agent's native_stderr.log.

Interestingly, I also saw this: -

tail -f SystemOut.log

-bash: fork: retry: Resource temporarily unavailable
-bash: fork: retry: Resource temporarily unavailable
-bash: fork: retry: Resource temporarily unavailable
bash: fork: retry: Resource temporarily unavailable
-bash: fork: Resource temporarily unavailable

which immediately reminded me of not one BUT two blog posts from last year: -

at which point I thought …. ah, yes, about those ulimits :-)

I referred back to an older set of notes, and found this: -

Need to ensure ulimits  are set: -

Increase: -

open files                      (-n) 10240
max user processes              (-u) 1024

to: -

open files                      (-n) 65536
max user processes              (-u) 16384

as follows: -

Add: -

# - nofile - max number of open files
wasadmin   soft    nofile  65536
wasadmin   hard    nofile  65536

# - nproc - max number of processes
wasadmin   soft    nproc   16384
wasadmin   hard    nproc   16384

to: -


Once I did this, and rebooted, everything came up ( not smelling of roses, but working A-OK ), which is nice.

Guess what I've added to my VM template ready for next week ??

Sunday, 12 April 2015

Learn the workings of Git, not just the commands

This: -

Learn the workings of Git, not just the commands

When you come from using a classical source code repository like Concurrent Versions System (CVS) or Subversion (SVN), it can be difficult to understand the flexibility and the possibilities of Git. This article tries to explain the workings of Git and how it all fits together, so you don't have to try to remember the commands, but instead just know what to do because you understand it well.

via IBM developerWorks on Twitter.

Thursday, 9 April 2015

IBM AIX commands you should not leave home without

Saw this via Twitter: -

Do you ever wish you could answer some of your own questions when you work with IBM AIX® and your IBM Power Systems™ server? Do you ever feel you could save time by not having to call on the support professionals all the time? Well, wish no more. Shiv Dutta discusses some of the AIX commands that answer those questions and tells you how to enlarge the list of such answers.