Wednesday, 17 December 2008

WebSphere Portal 6.1 and Site Analytics

Just working to show how WebSphere Portal's Site Analytics service works, in terms of generating logs in the NCSA Combined Log format that can then be analysed by an appropriate tool, such as the free-as-in-beer GPL tool AWStats, Nihuo  or a chargeable product/service such as WebTrends.

I'd previously set this up in an earlier release ( I even found a developerWorks forum post from myself back in 2006 ), but had forgotten precisely how to do it.

I confused myself by reading the formal documentation so ended up hacking around and stop/starting the portal several times.

In the end, the trick was to: -

a) Amend C:\IBM\WebSphere\wp_profile\PortalServer\config\SiteAnalyzerLogService.properties and add/change: -

SiteAnalyzerFileHandler.fileName=logs/$APPSERVER_NAME/sa.log
SiteAnalyzerPageLogger.isLogging=true
SiteAnalyzerPortletLogger.isLogging=true
SiteAnalyzerSessionLogger.isLogging=true

( there are a number of other isLogging parameters to choose from )

b) Push the updated properties into the portal configuration, using the command: -

ConfigEngine.bat update-properties

from the directory C:\IBM\WebSphere\wp_profile\ConfigEngine.

c) Restart the portal

Once this has been done, you can see/change the properties by logging into the WebSphere Application Server administration console ( https://localhost:10041/ibm/console ) and then navigating to Resource > Resource environment providers > WP SiteAnalyzerLogService > Custom properties as per the screenshot.

Having restarted the portal, and logged in, I now see data such as: -

192.168.44.1 - - [17/Dec/2008:11:33:14 +0000] "GET /Page/6_CGAH47L0082M302B566VO730M6/null HTTP/1.1" 200 -1 "" "Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.0.4) Gecko/2008111318 Ubuntu/8.10 (intrepid) Firefox/3.0.4" "pSite=http%3A//www-304.ibm.com/jct03001c/services/learning/ites.wss/us/en%3FpageType%3Dcourse_description%26courseCode%3DPW510;sasidp=S420311311229505852077;90130510_clogin=l=1229505853&v=1&e=1229514686475;CoreID6=62742661401312295058533;ibmSurvey=1229505848905;sauidp=U420311311229505852077;satst=test"
192.168.44.1 - - [17/Dec/2008:11:33:14 +0000] "GET /Portlet/5_CGAH47L0082M302B566VO730E4/Login_Portlet?PortletPID=5_CGAH47L0082M302B566VO730E4&PortletMode=View&PortletState=Normal HTTP/1.1" 200 -1 "http://stargate.uk.ibm.com/Page/6_CGAH47L0082M302B566VO730M6/null" "Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.0.4) Gecko/2008111318 Ubuntu/8.10 (intrepid) Firefox/3.0.4" "pSite=http%3A//www-304.ibm.com/jct03001c/services/learning/ites.wss/us/en%3FpageType%3Dcourse_description%26courseCode%3DPW510;sasidp=S420311311229505852077;90130510_clogin=l=1229505853&v=1&e=1229514686475;CoreID6=62742661401312295058533;ibmSurvey=1229505848905;sauidp=U420311311229505852077;satst=test"

being logged to C:\IBM\WebSphere\wp_profile\logs\WebSphere_Portal\sa.log.

I can now use my log analyser tool to see what's going on within my portal.

I'm actually using AWStats on my Ubuntu host whilst the portal is running within a VMware image.

I'll use the AWStats documentation as well as this excellent article from Stefan Liesche on developerWorks.

7 comments:

Andrew Frayling said...

Do you have to parse the log files before running through AWStat?

We've tried something similar on Portal 6.0.x, but because portal logs the encoded URLs and object IDs to the log files we found it hard (read impossible) to get anything sensible out of AWStat when we used the raw sa.log files.

Would be interested if you've got a solution for this.

Dave Hay said...

Hi Andrew, nope, I didn't have to do anything to the SA.LOG file in order to parse it with AWStats or the other tool that I briefly played with, Nihuo. I've posted an update to my earlier article with some more information -> http://tinyurl.com/9xwgsb

HtH

Dave

Andrew Frayling said...

Hi Dave, thanks for the update.

What I meant was have you looked at converting portal URLs to something readable? On page 7 of your work in progress (http://tinyurl.com/9xwgsb) you've got URLs like "Page/6_CGAH47L0009JD02B..." and wondered if you've ever looked at converting these to "Page/Home", "Page/Email", etc.?

Thanks,

Andrew.

Dave Hay said...

Hi Andy, ah, OK, sorry I misunderstood your question. It's my assumption that I can override page URls with so-called friendly URLs which will then be more meaningful in SA.LOG, regards, Dave

Andrew Frayling said...

Thanks again Dave. Will be watching with interest as I never got friendly URLs to log properly on 6.0.x, it kept logging the encoded page URIs

Probably a config step I missed somewhere :/

Chris Sparshott said...

Hello there, I am trying to find some examples of Web Site analytic tools that have been used with WebSphere Portal - any ideas?

Dave Hay said...

@Chris - check this out: -

http://www.ibm.com/developerworks/websphere/zones/portal/proddoc/dw-w-siteanalytics/index.html

Note to self - use kubectl to query images in a pod or deployment

In both cases, we use JSON ... For a deployment, we can do this: - kubectl get deployment foobar --namespace snafu --output jsonpath="{...