Friday, 14 November 2014

Hmmmm, HTTP404 and SRVE0190E seen with IBM HTTP Server and WebSphere Application Server

Hmm, so I am seeing this: -

Error 404: java.io.FileNotFoundException: SRVE0190E: File not found: /index.html

when I attempt to access a HTML page from IBM HTTP Server via HTTPS: -


even though I can get the page via HTTP: -


This is part of an IBM BPM Advanced 8.5.5 infrastructure, and the most recent change was to add IHS into the mix, federate it into the WAS cell, and add a Virtual Host entry for port 8443: -

cellID=AdminControl.getCell() 
AdminConfig.create('HostAlias', AdminConfig.getid('/Cell:'+cellID+'/VirtualHost:default_host/'), '[[hostname "*"] [port "8443"]]') 
AdminConfig.save() 
AdminNodeManagement.syncActiveNodes() 
quit 

I tried enabling further debugging in IHS by changing: -

LogLevel warn

to: -

LogLevel debug

in: -

/opt/IBM/HTTPServer/conf/httpd.conf

but the only thing that popped up in access_log was: -

192.168.1.70 - - [14/Nov/2014:15:08:26 +0000] "GET / HTTP/1.1" 404 42

and nothing useful in error_log.

I wondered whether the problem was related to the WAS Plugin, which was configured at the end of httpd.conf : -

LoadModule was_ap22_module "/opt/IBM/WebSphere/Plugins/bin/64bits/mod_was_ap22_http.so"
WebSpherePluginConfig /opt/IBM/WebSphere/Plugins/config/webserver1/plugin-cfg.xml

so I temporarily commented out both lines, and restarted IHS.

At that point, getting index.html via HTTP *and* HTTPS worked a treat.

So it's definitely somehow related to the Plugin and/or WAS.

I enabled debug in the plugin, by changing: -

   <Log LogLevel="Error" Name="/opt/IBM/WebSphere/Plugins/logs/webserver1/http_plugin.log"/>

to: -

   <Log LogLevel="Debug" Name="/opt/IBM/WebSphere/Plugins/logs/webserver1/http_plugin.log"/>

in: -

/opt/IBM/WebSphere/Plugins/config/webserver1/plugin-cfg.xml 

and can see: -

[14/Nov/2014:15:01:23.89900] 000045c2 057fb700 - DETAIL:    GET /index.html HTTP/1.1
[14/Nov/2014:15:01:23.89901] 000045c2 057fb700 - DETAIL:    Host: bpm855.uk.ibm.com:8443
[14/Nov/2014:15:01:23.89901] 000045c2 057fb700 - DETAIL:    Cache-Control: max-age=0
[14/Nov/2014:15:01:23.89902] 000045c2 057fb700 - DETAIL:    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
[14/Nov/2014:15:01:23.89902] 000045c2 057fb700 - DETAIL:    User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.122 Safari/537.36
[14/Nov/2014:15:01:23.89903] 000045c2 057fb700 - DETAIL:    DNT: 1
[14/Nov/2014:15:01:23.89903] 000045c2 057fb700 - DETAIL:    Accept-Encoding: gzip,deflate,sdch
[14/Nov/2014:15:01:23.89904] 000045c2 057fb700 - DETAIL:    Accept-Language: en-US,en;q=0.8
[14/Nov/2014:15:01:23.89905] 000045c2 057fb700 - DETAIL:    Cookie: JSESSIONID=00002GxGEWjz6XsUDTV69LDQYmi:196509vjn; CoreID6=40736703432114159613577&ci=50200000|ESTKCS; UnicaNIODID=JOlYGvtFslm-Y5kRjJ7; CoreM_State=72~-1~-1~-1~-1~3~3~5~3~3~7~7~|~~|~~|~~|~||||||~|~~|~~|~~|~~|~~|~~|~~|~; CoreM_State_Content=6~|~38D1983DAF0161AD~|~0; 50200000|ESTKCS_clogin=v=1&l=1415961357&e=1415963160646; LtpaToken2=ELcJ3s000hNQAmekT9IFK4NhSC419BFnN4WsQxjSBdXOIt78jL/UVc/tSQd4p+wBCpygJwkLG51qhZQcYOoZN0PNsYOwLyow8ERnvf4jeWyqw45ZetsBSErQjemaZHoHXMyO8c85FdH4edxT5M5AFdojKPpKKAabTKe94Jm2KAlsGb6Mw5BkQMkVIIocUMqTFqpoyBxmxGbADGAjJ86eiI+1MtYQXVFYggAIevtQllckeB77xEkionOnm1q0POYq+5CLdjVpGxXE0sizd2vua5iySBjWoIuIR6pfhfBw8DE8htetEXvHzJlDBErGXcUdo43zMeQQINzSxAYzWsEvlxjxCqYEuEpYSzplj7Kf8RmM4QSjtYWc+v9ZmHYgGBA9; sessionCode=514564614; TJE=; TE3=
[14/Nov/2014:15:01:23.89905] 000045c2 057fb700 - DETAIL:    $WSCS: TLS_RSA_WITH_AES_128_GCM_SHA256
[14/Nov/2014:15:01:23.89906] 000045c2 057fb700 - DETAIL:    $WSIS: true
[14/Nov/2014:15:01:23.89907] 000045c2 057fb700 - DETAIL:    $WSSC: https
[14/Nov/2014:15:01:23.89907] 000045c2 057fb700 - DETAIL:    $WSPR: HTTP/1.1
[14/Nov/2014:15:01:23.89908] 000045c2 057fb700 - DETAIL:    $WSRA: 192.168.1.70
[14/Nov/2014:15:01:23.89908] 000045c2 057fb700 - DETAIL:    $WSRH: 192.168.1.70
[14/Nov/2014:15:01:23.89909] 000045c2 057fb700 - DETAIL:    $WSSN: bpm855.uk.ibm.com
[14/Nov/2014:15:01:23.89909] 000045c2 057fb700 - DETAIL:    $WSSP: 8443
[14/Nov/2014:15:01:23.89910] 000045c2 057fb700 - DETAIL:    $WSSI: wkUAAOMXLq4ktsQ6fdbYIboGeENYWFhYQxlmVAAAAAI=
[14/Nov/2014:15:01:23.89910] 000045c2 057fb700 - DETAIL:    Surrogate-Capability: WS-ESI="ESI/1.0+"
[14/Nov/2014:15:01:23.89911] 000045c2 057fb700 - DETAIL:    _WS_HAPRT_WLMVERSION: -1
[14/Nov/2014:15:01:23.89916] 000045c2 057fb700 - DEBUG: ws_common: websphereExecute: Wrote the request; reading the response (timeout 900)
[14/Nov/2014:15:01:23.89917] 000045c2 057fb700 - DETAIL: lib_htresponse: htresponseRead: Reading the response: f002ab30
[14/Nov/2014:15:01:23.90631] 000045c2 057fb700 - DETAIL:    HTTP/1.1 404 Not Found
[14/Nov/2014:15:01:23.90637] 000045c2 057fb700 - DETAIL:    X-Powered-By: Servlet/3.0
[14/Nov/2014:15:01:23.90638] 000045c2 057fb700 - DETAIL:    Content-Type: text/html;charset=ISO-8859-1
[14/Nov/2014:15:01:23.90639] 000045c2 057fb700 - DETAIL:    $WSEP:
[14/Nov/2014:15:01:23.90639] 000045c2 057fb700 - DETAIL:    Content-Language: en-GB

in: -

/opt/IBM/WebSphere/Plugins/logs/webserver1/http_plugin.log

So this suggests that the request is being forwarded, by IHS, via the WAS Plugin, to WAS.

So I drilled into the plugin-cfg.xml file a bit more.

Logic dictated that the problem was somehow related to the context root of the URL, in that the page I'm requesting - /index.html - doesn't have the usual context root of, say, /ProcessCenter or /ProcessPortal.

Lo and behold, in plugin-cfg.xml, I can see: -

   <ServerCluster CloneSeparatorChange="false" GetDWLMTable="false" IgnoreAffinityRequests="false" LoadBalance="Round Robin" Name="dmgr_Dmgr_Cluster" PostBufferSize="0" PostSizeLimit="-1" RemoveSpecialHeaders="true" RetryInterval="60" ServerIOTimeoutRetry="-1">
      <Server ConnectTimeout="0" ExtendedHandshake="false" MaxConnections="-1" Name="Dmgr_dmgr" ServerIOTimeout="900" WaitForContinue="false"/>
   </ServerCluster>
   <UriGroup Name="default_host_AppCluster_URIs">
...
<Uri AffinityCookie="JSESSIONID" AffinityURLIdentifier="jsessionid" Name="/*"/>
...
   </UriGroup>
   <Route ServerCluster="AppCluster" UriGroup="default_host_AppCluster_URIs" VirtualHostGroup="default_host"/>

Now where did that little sucker come from ?

I checked the source, which is held at the WAS cell level: -

/opt/IBM/WebSphere/AppServer/profiles/Dmgr01/config/cells/PCCell1/nodes/Node1/servers/webserver1

which has the same thing.

I'm going to retrace my steps but .... at least I have a successful resolution / circumvention, which is nice.

at 2 comments:

New - IBM Business Process Management Developer Center

IBM Business Process Management Developer Center

For Developers of Reusable Smarter Process Applications and Toolkits

at No comments:

Think big! Scale your business rules solutions up to the world of big data

This is from my ISSW colleague, Nigel Crowther, and is definitely worth a read: -


<snip>
Traditional business rule applications process records of a few megabytes of data at a time. Records are usually processed as client server requests or in a batch, one record at a time. As solutions move to the cloud, and applications apply rules to terabytes of data, these traditional approaches cannot keep up. To scale business rules solutions up to the world of big data, consider using the Business Rules and the IBM® Analytics for Hadoop services in IBM Bluemix™.

This tutorial describes a generic application called RulesAdaptor that uses services in IBM Bluemix to integrate the business rules of IBM Operational Decision Manager and the big data capabilities of Apache™ Hadoop®. This application opens up the possibility for data scientists to analyze big data with business rules.
</snip>

at No comments:

Thursday, 13 November 2014

More on WebSphere Application Server Java Versions

Following on from a discussion with one of my colleagues - who hit the precise same problem that I outlined here: -


when creating some JEE code for IBM Operational Decision Manager (ODM), I thought I'd add a bit more detail about the Java version supplied with a popular IBM product, IBM Business Process Manager 8.5.0.1.

So we have BPM Advanced 8.5.0.1 deployed here, on top of WebSphere Application Server 8.5.5.2 as evidenced here: -

/opt/ibm/WebSphere/AppServer/bin/versionInfo.sh

…
Name                  IBM Business Process Manager Advanced
Version               8.5.0.1
ID                    BPMPC
Build Level           20131107-140634
Build Date            11/7/13
Package               com.ibm.bpm.ADV.v85_8.5.1.20131107_1830
Architecture          PPC64
Installed Features    Business Process Manager Advanced Process Server Non-production
Name                  IBM WebSphere Application Server Network Deployment
Version               8.5.5.2
ID                    ND
Build Level           cf021414.01
Build Date            4/8/14
Package               com.ibm.websphere.ND.v85_8.5.5002.20140408_1947
Architecture          PPC64
Installed Features    IBM 64-bit WebSphere SDK for Java
                      WebSphere Application Server Full Profile
                      EJBDeploy tool for pre-EJB 3.0 modules
                      Embeddable EJB container
                      Stand-alone thin clients and resource adapters
….


Therefore one might assume that we also have the latest available Java JDK/JRE, namely Java 7 aka Java 1.7.0 ??

Nope, as evidenced here: -

/opt/ibm/WebSphere/AppServer/java/jre/bin/java –version

java version "1.6.0"
Java(TM) SE Runtime Environment (build pap6460_26sr7fp1ifix-20140220_01(SR7 FP1+IX90136+IX90137))
IBM J9 VM (build 2.6, JRE 1.6.0 AIX ppc64-64 Compressed References 20131230_180580 (JIT enabled, AOT enabled)
J9VM - R26_Java626_SR7_20131230_1725_B180580
JIT  - r11.b05_20131003_47443.02
GC   - R26_Java626_SR7_20131230_1725_B180580_CMPRSS
J9CL - 20131230_180580)
JCL  - 20140107_01

/opt/ibm/WebSphere/AppServer/java/bin/java –version

java version "1.6.0"
Java(TM) SE Runtime Environment (build pap6460_26sr7fp1ifix-20140220_01(SR7 FP1+IX90136+IX90137))
IBM J9 VM (build 2.6, JRE 1.6.0 AIX ppc64-64 Compressed References 20131230_180580 (JIT enabled, AOT enabled)
J9VM - R26_Java626_SR7_20131230_1725_B180580
JIT  - r11.b05_20131003_47443.02
GC   - R26_Java626_SR7_20131230_1725_B180580_CMPRSS
J9CL - 20131230_180580)
JCL  - 20140107_01

Easy when you know :-)

Furthermore, this shows you what SDKs are available: -

/opt/ibm/WebSphere/AppServer/java/bin/managesdk.sh -listAvailable

CWSDK1003I: Available SDKs :
CWSDK1005I: SDK name: 1.6_64
CWSDK1001I: Successfully performed the requested managesdk task.

This is the GUI alternative: -


and this shows what's available to install: -

/opt/ibm/InstallationManager/eclipse/tools/imcl listAvailablePackages -repositories /mnt/SWGRepo/WAS85/Base/Product -features -long

/mnt/SWGRepo/WAS85/Base/Product : com.ibm.websphere.BASE.v85_8.5.0.20120501_1108 : IBM WebSphere Application Server : 8.5.0.0 : core.feature,ejbdeploy,thinclient,embeddablecontainer,samples,liberty,com.ibm.sdk.6_32bit,com.ibm.sdk.6_64bit

In other words, we only have the 32-bit and 64-bit Java 6 SDKs with which to play.

Java 7 can be downloaded as part of WAS 8.5.5.X, as per this: -



However, it's not clear to me ( and I've not investigated ) whether IBM BPM, ODM etc. actually support Java 7.

In other words, Your Mileage May Vary (YMMC)





Want to know more about Java and WAS ? Read here ....




at 2 comments:

IBM Business Process Manager Advanced - Fun with Integration and Monitoring

So we had two interesting issues yesterday, in the context of a Business Process Execution Language (BPEL) application, using the Service Component Architecture (SCA) Java APIs.

Summary

- Ensure that the WAS configuration matches the application e.g. if the application uses Connection Factory, don't configure Queue Connection Factory
- Ensure that JMS Connection Factory / Queue Connection Factory objects are correctly configured for WebSphere MQ Queue Manager *IF* using clustered Queues
- Ensure that JMS Queue objects are correctly configured for WebSphere MQ Queue *IF* using clustered Queues
- Ensure that the Common Event Infrastructure (CEI) framework is correctly configured if running BPEL applications in one cluster ( AppCluster ), with CEI running on another cluster ( SupCluster )

Detail

The application writes messages out to an integration solution ( IBM Integration Bus via a WebSphere MQ cluster ), and also emits events to IBM Business Monitor ( BAM ), via the Service Integration Bus (SIBus) infrastructure.

The application has been written using IBM Integration Designer, and leverages the underlying WebSphere Application Server (WAS) platform, upon which BPM Advanced is deployed.

So the first issue that we saw was that, despite evidence to the contrary, the application was coded to utilise the Connection Factory interface, which is OK. However, the WAS configuration was built on the assumption that the code was actually using the Queue Connection Factory subclass of Connection Factory.

Therefore, the code was looking for CF whereas WAS was configured for QCF.

This was evidenced by the Resource References configuration in WAS, seen via Applications > Application Types > WebSphere Enterprise Applications, which did NOT allow us to browse/select the Queue Connection Factory, even though the QCF could be seen within the Resources > JMS > Queue Connection Factories view.

Initially, we thought this was a WAS scope ( cell/cluster/node/server ) issue, but both the application AND the QCF were scoped at the same level - AppCluster.

However, once we created a Connection Factory in place of the QCF, with the same settings, it showed up in the Resource References view.

We then saw WMQ-related issues in the logs, including: -

JMSWMQ2008: Failed to open MQ queue 'AUDITQUEUE'.

Having confirmed that this queue DID exist in the WebSphere MQ, we scratched our heads for a while ....

However, the solution was forthcoming :-)

In essence, the Queue AUDITQUEUE is hosted by a WMQ Cluster, which is "owned" by a remote ( to BPM ) Queue Manager, running on our ESB server ( IBM Integration Bus ).

Therefore, this Queue is, effectively, unknown to the local ( to BPM ) Queue Manager.

This means that, whilst it is possible to write messages to this Queue from the BPM server using, for example, the amqsput command: -

amqsput auditqueue localQM

the Queue is not "owned" by the local Queue Manager.

Bottom line, this means that the Queue Connection Factory or, in our case, the Connection Factory does NOT need to be configured to "know" about the Queue, even though there is an option to enter BOTH the Queue Manager AND Queue names.

Therefore, we changed the newly created CF to point at the local Queue Manager but NOT the clustered Queue.

However, we also have a JMS Queue, which IS used by the BPEL application, seen via the Applications > SCA Modules view in the WAS console. The JMS Queue only needs to "know" about the WMQ Queue, NOT the Queue Manager, which is "handled" by the Connection Factory.

Summary

JMS Connection Factory / Queue Connection Factory > only requires name of local Queue Manager
JMS Queue > only requires name of clustered Queue

This overcame the JMSWMQ2008 exception.

We then moved the problem on - the application starts OK, but then throws a bunch of JNDI-related exceptions during use.

This proved to be because the application emits events intended to be consumed by IBM Business Monitor.

Examples include: -

CEMEM0003E: The specified emitter factory was not found in JNDI. javax.naming.NameNotFoundException: Context: PSCell1/clusters/AppCluster, name: com/ibm/events/configuration/emitter/Default: First component in name events/configuration/emitter/Default not found. 

Caused by: javax.xml.ws.WebServiceException: javax.xml.ws.WebServiceException: com.ibm.websphere.sca.ServiceRuntimeException: com.ibm.bpe.api.EngineStateObserverEventEr

ror: CWWBE0013E: An error occurred during the event handling of 'activityFailed' in the observer plug-in type 'com.ibm.bpe.engine.observer.CEMStateObserverPlugin'.: cau

sed by: com.ibm.bpe.api.EngineStateObserverEventError: CWWBE0013E: An error occurred during the event handling of 'activityFailed' in the observer plug-in type 'com.ibm

.bpe.engine.observer.CEMStateObserverPlugin'.

Caused by: java.lang.IllegalStateException: Cannot access CEI

We spent a long time trying to resolve this.

Having confirmed that BPM was already happily emitting events to BAM, via the Service Integration Bus ( in essence, one configures a remote Messaging Engine, hosted by BAM, within the BPM SIBus configuration ). This means that the BPM applications emit, running on the AppCluster, emit events, which are passed via the Common Event Infrastructure (CEI) framework, running on the SupCluster, to the SIBus, hosted by the MECluster.

We had previously verified that BPM > BAM connectivity was working, using a Business Process Modelling Notation (BPMN) application.

However, our BPEL application was failing with the JNDI messages referenced previously :-(

After discussions with others in the wider WebSphere team ( specifically two of the world's best BAM SMEs ), we determined that the process that BPEL uses to emit events from BPM to BAM is completely different to that used by BPMN, which is good to know :-)

Eventually, we stumbled on the solution, partly through dialogue and partly through online searching.

In essence, it is necessary to "tell" the AppCluster, hosting the BPEL applications, that CEI is hosted by the SupCluster.

Ironically, this is documented here: -


<snip>

Optional: If you are migrating to IBM BPM V8.5 from an earlier version, or if CEI and Business Process Choreographer are deployed on different clusters, configure the CEI emitter factory:
DMGR_PROFILE/bin/wsadmin -lang jython -f WAS_home\util\migration\scripts\setCEIDestination.py -a applicationClusterName -s supportClusterName -no-sync

Where:

-a applicationClusterName
is the name of the application cluster
-s supportClusterName
is the name of the support cluster
–no-sync

is an optional parameter that specifies whether to synchronize the changes to all nodes. By default, no-sync is false, which means that the node synchronize command is called immediately after the change. If no-sync is true, the node synchronize command is not called automatically and you must synchronize the nodes later so that the changes take effect.

An example of configuring the CEI emitter factory follows:

DMGR_PROFILE/bin/wsadmin -lang jython -f P:\bpm8500\util\migration\scripts\setCEIDestination.py -a AppCluster -s SupportCluster 

</snip>

Note the word Optional - that does NOT help :-) Also, note the phrase If you are migrating to IBM BPM V8.5 from an earlier version. Again, that doesn't help :-)

The important phrase is: -

if CEI and Business Process Choreographer are deployed on different clusters, configure the CEI emitter factory:

That IS the important part.

We're using the so-called Gold Standard BPM topology, aka Application, Remote Messaging, and Remote Support.

Therefore, we DO have CEI and BPC on different clusters, respectively SupCluster and AppCluster.

Following the so-called optional step, we "told" AppTarget where SupCluster is: -

opt/IBM/WebSphere/AppServer/profiles/Dmgr01/bin/wsadmin.sh -lang jython -host `hostname` -port 8879 -username wasadmin -password passw0rd -f /home/wasadmin/setCEIDestination.py -a AppCluster -s SupCluster

WASX7209I: Connected to process "dmgr" on node dmgr using SOAP connector;  The type of process is: DeploymentManager
WASX7303I: The following options are passed to the scripting environment and are available as arguments that are stored in the argv variable: "[-a, AppCluster, -s, SupCluster]"
XXXX0000I: Updating configuration on server AppClusterMember1 on node AppSrv01Node.
XXXX0000I: Current value: com/ibm/events/configuration/emitter/Default
XXXX0000I: New value    : cell/clusters/SupCluster/com/ibm/events/configuration/emitter/Default
XXXX0000I: Configuration is being saved.
XXXX0000I: Synchronizing the configuration across all nodes.

---------------------------------------------------------------
AdminNodeManagement:        Synchronize the active nodes
Usage: AdminNodeManagement.syncActiveNodes()
Return: If the command is successfully invoked, a value of 1 is returned.
---------------------------------------------------------------

AppSrv01Node
XXXX0000I: Done.

Once we did this, and restarted the entire BPM Deployment Environment, we no longer saw the JNDI / CEI exceptions, and events emitted by our BPEL application started showing up in BAM.

at No comments:

Tuesday, 11 November 2014

How do I enable XSS and CSRF protection in IBM Cognos provided with IBM Business Monitor

Question

I am using IBM Cognos provided with IBM Business Monitor, and want to ensure it is protected against XSS and CSRF attacks. How do I do this?

at No comments:

Sunday, 9 November 2014

More on NFS and Mac OS X and Linux

Following on from my earlier post: -


I discovered that, if I want to mount a NFS share for read/write operations from Mac OS X, I needed to ensure that the NFS server is set up to "enforce" the UID and GID of the owner of the target directory.

In my case, I was trying to write from Eclipse on the Mac to WebSphere Application Server on RHEL, where the WAS file-system ( /opt/IBM/WebSphere etc. ) is "owned" by the user wasadmin.

Thus, I needed to enforce the UID/GID by changing /etc/exports as follows: -

/opt/IBM/WebSphere/AppServer *(rw,all_squash,insecure,anonuid=500,anongid=500)

and then re-export the file-system: -

exportfs -a

and mount the directory from the Mac: -

mount bpm855.uk.ibm.com:/opt/IBM/WebSphere/AppServer ~/nfs/

Without this, I was seeing the following: -

rm: /Users/hayd/nfs/foobar: Permission denied

whenever I tried to write from OS X to RHEL.

In short, it's all because I am writing from the Mac as user hayd to RHEL where that user does NOT exist and, to make it worse, I'm trying to write to a file-system owned by a user, wasadmin, of which the Mac knows nothing.

Adding the anonuid and anongid parameters to the /etc/exports does the trick.

Final point - I also needed to include all_squash as per the above; the combination of that, with the anonuid and anongid were what did the trick ( as I'd already established that I needed insecure as per the previous NFS-related post )

at No comments:
Subscribe to: Posts (Atom)

Note to self - Firefox and local connections

 Whilst trying to hit my NAS from Firefox on my Mac, I kept seeing errors such as:- Unable to connect Firefox can’t establish a connection t...