Using unzip on Mac OS X with L-A-R-G-E files

Whilst trying to unpack a large >4 GB .ZIP file: -

unzip /Volumes/DaveHay/Software/ODM87/Fixes/Windows/8.7.1-WS-ODM_DC-WIN-FP001.zip 

I saw this: -

Archive:  /Volumes/DaveHay/Software/ODM87/Fixes/Windows/8.7.1-WS-ODM_DC-WIN-FP001.zip

warning [/Volumes/DaveHay/Software/ODM87/Fixes/Windows/8.7.1-WS-ODM_DC-WIN-FP001.zip]:  4294967296 extra bytes at beginning or within zipfile
  (attempting to process anyway)
file #1:  bad zipfile offset (local header sig):  4294967296
  (attempting to re-compensate)
   creating: 8700/
…

I'm using the stock unzip command that comes with Mac OS X 10.11.2: -

unzip -v

UnZip 5.52 of 28 February 2005, by Info-ZIP.  Maintained by C. Spieler.  Send
bug reports using http://www.info-zip.org/zip-bug.html; see README for details.

Latest sources and executables are at ftp://ftp.info-zip.org/pub/infozip/ ;
see ftp://ftp.info-zip.org/pub/infozip/UnZip.html for other sites.

Compiled with gcc 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.1) for Unix on Jul 31 2015.
...

I've seen this before, albeit on AIX - I solved it then using the jar command, as shipped with IBM Java7.

As ever, the answer was online: -

How to deal with unzip error on a large file in OSX?

…

I managed to unzip it with tar.

tar xzvf 2015-09-24-raspbian-jessie.zip

Requires no installation of additional software

…

I tried the same thing: -

tar xvzf /Volumes/DaveHay/Software/ODM87/Fixes/Windows/8.7.1-WS-ODM_DC-WIN-FP001.zip 

which worked a treat.

Thanks Internet :-)

IBM Installation Manager - 32 vs 64 bits

I see this from time to time, this time whilst scripting up an IBM ODM Rules installation on Windows.

During the installation process: -

"c:\IBM\Installation Manager\eclipse\tools\imcl.exe" -input z:\DaveHay\Software\Repo\Win\ResponseFiles\installWAS8555_WIN.rsp -acceptLicense

I see this: -

ERROR: The 32-bit IBM WebSphere Application Server Network Deployment package cannot be installed in a 64-bit package group.

even though I know that I've got the 64-bit version of WAS ready to install on a 64-bit OS.

As ever, the solution is simple :-)

Read my own blog, read my own notes, look at an existing response file, record a NEW response file.

This time around, I did the latter: -

"c:\IBM\Installation Manager\eclipse\IBMIM.exe" -record foobar.rsp -skipInstall c:\temp\foobar

which resulted in: -

<?xml version='1.0' encoding='UTF-8'?>
<agent-input>
  <variables>
    <variable name='sharedLocation' value='C:\IBM\IMShared'/>
  </variables>
  <server>
    <repository location='z:\DaveHay\Software\Repo\WAS\Product'/>
    <repository location='z:\DaveHay\Software\Repo\WAS\Fixes'/>
  </server>
  <profile id='IBM WebSphere Application Server V8.5' installLocation='C:\IBM\WebSphere\AppServer'>
    <data key='cic.selector.arch' value='x86'/>
  </profile>
  <install>
    <!-- IBM WebSphere Application Server Network Deployment 8.5.5.5 -->
    <offering profile='IBM WebSphere Application Server V8.5' id='com.ibm.websphere.ND.v85' version='8.5.5005.20150220_0158' features='core.feature,ejbdeploy,thinclient,embeddablecontainer,com.ibm.sdk.6_64bit'/>
  </install>
  <preference name='com.ibm.cic.common.core.preferences.eclipseCache' value='${sharedLocation}'/>
  <preference name='offering.service.repositories.areUsed' value='false'/>
</agent-input>

I compared/contrasted the highlighted line: -

    <data key='cic.selector.arch' value='x86'/>

with the response file that I was trying to use: -

    <data key='cic.selector.arch' value='x86_64'/>

and changed it accordingly.

It's annoying when things are so simple, especially when it seems counter-intuitive i.e. if one is installing a 64-bit product onto a 64-bit OS, why would one NOT specify x86_64 :-)

Still, it all works now: -

"c:\IBM\Installation Manager\eclipse\tools\imcl.exe" -input z:\DaveHay\Software\Repo\Win\ResponseFiles\installWAS8555_WIN.rsp -acceptLicense

Installed com.ibm.websphere.ND.v85_8.5.5005.20150220_0158 to the C:\IBM\WebSphere\AppServer directory.

which is nice :-)

Integrate Docker in your IBM Operational Decision Manager environment

This is DEFINITELY something on my short-term reading list, as it's something with which I want / need to get to grips in the New Year: -

IBM® Operational Decision Manager (ODM) application developers who are new to Docker might consider adding Docker to their development process. This tutorial shows how to build a Docker image for an IBM ODM Rule Execution Server and Decision Center that run on a WebSphere Liberty server. You also learn how to run the Rule Execution Server and Decision Center in a container from the Docker image. This content is part of the IBM Business Process Management Journal.

Integrate Docker in your IBM Operational Decision Manager environment

It covers Docker, Liberty and ODM Rules in a single article. What's not to like ?

Book Review - 57 Challenges to Develop Your Coding Skills by Brian P. Hogan

It's been a while since last I reviewed a book, so I wanted to end the year with … a review.

Our friends at The British Computer Society (BCS) kindly sent me a copy of Exercises for Programmers : 57 Challenges to Develop Your Coding Skills by Brian P. Hogan a few weeks back.

This is available on Amazon here but my ebook came via a different route, thanks to BCS.

The purpose of the book is to, via a series of exercises, teach the reader the fundamentals of computer programming. The book is language-neutral, so it makes no difference whether you choose to use C, Java, Python, Swift etc. as the focus is on teaching one how to solve a problem in code, rather than how to use a particular language.

Through a series of increasingly complex problems, the author coaches the reader in the art of problem-solving, demonstrating how one can output data, perform calculations, compare strings, develop re-usable functions, sort data, use public APIs etc.

This took me back to my student days, where the instructor would set an exercise e.g. write a programme to output the Fibonacci sequence, print Pi to 32 decimal places etc.

Whilst this is a useful way to teach one HOW to programme, I found the exercises to be a little repetitive, and overly frustrating, perhaps because I'm not the ideal target audience i.e. I first learned to write computer programs more than 30 years ago, and I don't program for a living.

Additionally, whilst the book does set out each problem clearly, it does not provide any example solutions, even for the most simple of problems. This means that there's no obvious feedback loop between teacher and student. Again, it's perhaps my bias, but I'd have preferred to see a few example solutions, showing how the problem might be solved.

Don't get me wrong - this is a very useful book, and one that I'd definitely recommend to students of computing, but I'm not wholly convinced that it'd be the right tool for every job.

Perhaps it's down to the different ways that we all choose to learn, but I'd have personally preferred fewer, longer exercises, focused upon teaching a few fundamental concepts in one fell swoop.

As an ebook, this comes in around 263 pages in length, comprising, quelle surprise, 57 discrete exercises ( challenges ), each increasing in complexity, building upon the preceding chapters.

So, in conclusion, whilst this is an excellent book, I'd definitely recommend that the reader either be completely new to computer programming, or, perhaps better still, read this in conjunction with other, more detailed, tutorials.

Out of 10, I'd give this 7, taking into account the reservations expressed previously.

Before I finish, I want to thank Becky Youe at BCS for kindly providing me with this book to review.

Things that make you go "Hmmmm" - Part 321

A few days ago, having watched an excellent programme about Aardman Animation, Wallace and Gromit, Nick Park etc. I decided to rent one of Aardman's most recent films, The Pirates! In An Adventure with Scientists from iTunes.

So the film is sitting in my iTunes library, waiting to be watched …..

Last evening, I decide to watch it …..

On my Apple TV ……

At that point, the wheels come off my wagon …

Alas I can't find the movie, or even the Rental tab, on Apple TV, although I could choose to re-rent the film from iTunes :-)

Long story short, I Google and find this: -

About renting movies from the iTunes Store

…

If you rented the movie on your iOS device or Apple TV, you can't move it to any other device.

…

which, as far as I can establish, means that I cannot simply rent a movie on iTunes on my iPhone and then expect to watch it on my Apple TV.

The same article does helpfully say: -

…

You can also stream a rented movie from your computer or your iOS device to your Apple TV using AirPlay after it's fully downloaded.

…

which is precisely what I did.

In case it was me mis-reading the article, I asked one of the iMore gurus, Serenity Caldwell, via Twitter, who responded: -

settern @david_hay Yep. It's one of the most mind-blowingly stupid policies on the store. 29/12/2015, 20:02

which helps, although it's an annoyance.

I have raised this with Apple Feedback - we'll see what happens ;-)

Fix list for the IBM Business Process Manager Version 8.5 products

Thanks Twitter :-)

IBM_BPM IBM Fix list for the IBM #BPM Version 8.5 products (Dec update) https://t.co/BtyuN4h6f8 29/12/2015, 14:25

Fix list for the IBM Business Process Manager Version 8.5 products

Version 8.5 Refresh Pack 6 Cumulative Fix 2

Version 8.5 Refresh Pack 6 Cumulative Fix 1

IBM Fix Level Recommendation Tool (FLRT)

I love the acronym, I love (even more) the functionality: -

Fix Level Recommendation Tool

Windows Update - Refuses to Update

I've had this problem a few times in the past.

I find that freshly-squeezed VMs running Windows Server 2008 R2 ( built on VMware Fusion using an ISO downloaded from Microsoft Developer Network - MSDN ) simply refuse to update.

I run the Windows Update tool, click the "Check for Updates" button and am immediately told that no updates are available i.e. that I'm up to date, even though I know that I'm using a piece of software that was released in 2009.

The solution ?

Google :-)

The solution ! 

Install an update to … Windows Update :-)

Found here: -

How to update the Windows Update Agent to the latest version

which is nice :-)

WebSphere - Support Technical Exchanges

Twitter reminded me to x-post this: -

Support technical exchanges for WebSphere brand support (general)

…

…

Learn Linux, 101: Customize and use the shell environment

Learn Linux, 101: Customize and use the shell environment

Learn how to customize your Linux shell environment and write simple bash functions. You can use the material in this tutorial to study for the LPI 102 exam for Linux system administrator certification, or to learn for fun.

Overview

In this tutorial, learn to customize your Linux bash shell environment to meet user needs. Learn to:

• Modify global and user profiles

• Set environment variables when you log in or spawn a new shell

• Create bash functions for frequently used sequences of commands

• Maintain skeleton directories for new user accounts

• Set your command search path

Using watch to loop in Bash

I wanted to monitor a particular set of TCP/IP listeners, on an ongoing basis, and thought "I wonder how I do this?".

A quick Google led me here: -

https://www.linuxquestions.org/questions/linux-networking-3/netstat-grep-tail-f-output-log-749396/

specifically to the /usr/bin/watch command: -

…

       watch - execute a program periodically, showing output fullscreen
…

Here's my command: -

watch -n5 'netstat -a | grep db2'

which watches the output from the netstat command, and updates the display every 5 seconds, resulting in this: -

Every 5.0s: netstat -a | grep db2                                                                                                                                                   Thu Dec 17 13:29:07 2015

tcp        0   0 *:ibm-db2                   *:*                         LISTEN
tcp        0   0 bpm856.uk.ibm.com:39889     db2two.uk.ibm.com:60007     ESTABLISHED
tcp        0   0 bpm856.uk.ibm.com:39888     db2two.uk.ibm.com:60007     ESTABLISHED
tcp        0   0 bpm856.uk.ibm.com:39890     db2two.uk.ibm.com:60007     ESTABLISHED
udp        0   0 *:ibm-db2                   *:*

which is nice :-)

Reminder - DB2 HADR and SQL1768N rc7

Reminder to self: -

If I see SQL1768N rc7 when attempting to use DB2 HADR, there's a good chance that I've forgotten to update /etc/hosts meaning that one/both hosts is returning the loopback / localhost address of 127.0.0.1 when attempting to connect to itself via hostname.

For the record, /etc/hosts should look similar to this: -

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.153.100 bpm856.uk.ibm.com bpm856
192.168.153.110 db2one.uk.ibm.com db2one
192.168.153.120 db2two.uk.ibm.com db2two

In other words, there must be no other hostname alongside the localhost entries.

This is also useful: -

SQL1768N Unable to start HADR, Reason Code 7 occurs while initializing high availability disaster recovery (HADR).

How to tune a guitar and a Business Process Manager (BPM) topology at the same time

I found this whilst searching for something pertaining to Java2 Security: -

How to tune a guitar and a Business Process Manager (BPM) topology at the same time

<snip>

As such, here's a quick checklist of tuning tips that cover all areas of your IBM Business Process Manager (BPM) environment:

• Use a 64-bit JVM for all servers.
• Disable tracing, logging, and monitoring when possible.
• Ensure all of your databases are well-tuned.
• If security is required, use Application security, not Java2 security.
• Use an appropriate hardware configuration for performance measurement (not notebooks or desktops).
• If hardware virtualization is used, ensure that adequate processor, memory, and I/O resources are allocated to each virtual machine.
• Minimize network latency and ensure sufficient network bandwidth between all systems in the configuration.
• Do not run production servers in development mode or with a development profile.
• Tune external service providers and external interfaces to ensure that they do not cause a system bottleneck.
• Configure:
  • Message-driven bean (MDB) activation specifications.
  • Thread pool sizes.
  • Settings of data sources for connection pool size and prepared statement cache size.
• Increase the maximum number of connections in the data pool to greater than or equal to the sum of all maximum thread pool sizes.

This list represents just the fundamental requirements for optimal performance. See IBM Business Process Manager V8.0 Performance Tuning and Best Practices, REDP-4935-00 for a complete list of tuning recommendations for all areas of your IBM Business Process Manager topology. 

</snip>

More textile manipulation with sed and bash

One of my colleagues was looking for a way to manipulate an XML file, using sed on Linux.

In essence, he wanted to remove an XML tag at the current end of the file, and insert a new line BELOW the current end of the file, making the new line the new end of the file :-)

Here's a mockup of my solution.

I used a file called foobar.bak as my input file, mainly so that I could quickly see/compare the changes between input and output.

Input file - foobar.bak

The quick brown fox
Jumped over the lazy dog
In the pouring rainfall

( Note the tab characters; they're there because my friends' XML file had tabs for pagination )

Script - replace.sh

#!/bin/bash
cp foobar.bak foobar.txt

file="foobar.txt"
replace="fall"
insert="That is all"
sed -i'' "$ s/$replace//g" $file
sed -i'' "$ a\ \t \t$insert" $file

In essence, I'm copying the foobar.bak file to foobar.txt each time I run the script, meaning that I preserve the original file :-)

I'm then setting some variables, one for the filename, one for the character to be replaced, and one for the new line.

I then use sed to replace ( remove ) the to-be-replaced string.

Finally, I again use sed to insert a new line - note that I'm using sed twice, mainly because I couldn't work out how to replace and then insert a blank line in sed I even messed about with ^M characters, but to no avail.

Output file - foobar.txt

The quick brown fox
Jumped over the lazy dog
In the pouring rain
    That is all

Again, note the tabs for pagination ( they're handled with \t above ).

*UPDATE*

I then thought about the two occurrences of sed again, and came up with this improvement: -

#!/bin/bash
cp foobar.bak foobar.txt

file="foobar.txt"
replace="fall"
insert="That is all"
sed -i'' "$ s/$replace/\n\t\t$insert/g" $file

which has the same effect, but does everything in a single sed statement :-)

IBM Business Process Manager Advanced - Developer Enablement

IBM - Business Process Manager - Library

Tutorials

Hiring Tutorial

Hello World Part 1: Getting started

Hello World Part 2 : Service Components and Web Interfaces

Hello World for Datapower

Stock Quote

Flat Files

Business State Machine: Vending Machine