Monday, 22 July 2019

IBM WebSphere Liberty Profile and IBM DB2 and Docker - An approach

One of my IBM colleagues asked me about an approach for "versioning" DB2 JDBC drivers within a Docker image that's using those drivers to connect to a DB2 database.

He was looking at options for ensure that the drivers can be updated to match the target DB2 runtime, without needing to rebuild the entire WebSphere Liberty Profile image each and every time.

There are a number of approaches, including storing the drivers on an external ( to the image/container ) file-system, using a Docker volume.

As an example, something like this: -

docker run -v /host/directory:/container/directory image

In the absence of volumes, we could configure the container to reach out to an external file/web-server at boot-time ( akin to a bootstrap service )

As an alternative, we considered an approach where the image hosting WebSphere Liberty Profile references the JDBC drivers from a DIFFERENT image when it's built, meaning that we CAN rebuild the Liberty image without a huge amount of impact upon the size/layers of the Liberty image itself.

This approach seemed to work ....

I started by pulling the requisite images: -

docker pull websphere-liberty

docker pull store/ibmcorp/db2_developer_c:11.1.4.4-s390x

and created an environment file to start the DB2 instance within its own container: -

vi ~/.envlist

LICENSE=accept
DB2INSTANCE=db2inst1
DB2INST1_PASSWORD=passw0rd
TO_CREATE_SAMPLEDB=true

and started the DB2 database container: -

db2=`docker run -h db2server --name db2server --restart=always --detach  --privileged=true -p 50000:50000 -p 55000:55000 --env-file ~/.envlist -v /db2data:/database store/ibmcorp/db2_developer_c:11.1.4.4-s390x`

and checked the logs until DB2 came up: -

docker logs $db2 -f

...
07/19/2019 19:10:30     0   0   SQL1063N  DB2START processing was successful.
SQL1063N  DB2START processing was successful.
(*) Starting TEXT SEARCH service ...
CIE00001 Operation completed successfully. 
...

and then logged into the running container: -

docker exec -it $db2 /bin/bash

and switched to the db2inst1 user: -

su - db2inst1

Last login: Mon Jul 22 13:44:17 UTC 2019

and listed the DB directory: -

db2 list db directory

 System Database Directory

 Number of entries in the directory = 1

Database 1 entry:

 Database alias                       = SAMPLE
 Database name                        = SAMPLE
 Local database directory             = /database/config/db2inst1
 Database release level               = 14.00
 Comment                              =
 Directory entry type                 = Indirect
 Catalog database partition number    = 0
 Alternate server hostname            =
 Alternate server port number         =

and connected to the SAMPLE DB: -

db2 connect to sample

   Database Connection Information

 Database server        = DB2/LINUXZ64 11.1.4.4
 SQL authorization ID   = DB2INST1
 Local database alias   = SAMPLE

and query the EMPLOYEE table: -

db2 'select * from employee'

EMPNO  FIRSTNME     MIDINIT LASTNAME        WORKDEPT PHONENO HIREDATE   JOB      EDLEVEL SEX BIRTHDATE  SALARY      BONUS       COMM       
------ ------------ ------- --------------- -------- ------- ---------- -------- ------- --- ---------- ----------- ----------- -----------
000010 CHRISTINE    I       HAAS            A00      3978    01/01/1995 PRES          18 F   08/24/1963   152750.00     1000.00     4220.00
000020 MICHAEL      L       THOMPSON        B01      3476    10/10/2003 MANAGER       18 M   02/02/1978    94250.00      800.00     3300.00
000030 SALLY        A       KWAN            C01      4738    04/05/2005 MANAGER       20 F   05/11/1971    98250.00      800.00     3060.00
000050 JOHN         B       GEYER           E01      6789    08/17/1979 MANAGER       16 M   09/15/1955    80175.00      800.00     3214.00
000060 IRVING       F       STERN           D11      6423    09/14/2003 MANAGER       16 M   07/07/1975    72250.00      500.00     2580.00
000070 EVA          D       PULASKI         D21      7831    09/30/2005 MANAGER       16 F   05/26/2003    96170.00      700.00     2893.00
000090 EILEEN       W       HENDERSON       E11      5498    08/15/2000 MANAGER       16 F   05/15/1971    89750.00      600.00     2380.00
000100 THEODORE     Q       SPENSER         E21      0972    06/19/2000 MANAGER       14 M   12/18/1980    86150.00      500.00     2092.00
000110 VINCENZO     G       LUCCHESSI       A00      3490    05/16/1988 SALESREP      19 M   11/05/1959    66500.00      900.00     3720.00
000120 SEAN                 O'CONNELL       A00      2167    12/05/1993 CLERK         14 M   10/18/1972    49250.00      600.00     2340.00
000130 DELORES      M       QUINTANA        C01      4578    07/28/2001 ANALYST       16 F   09/15/1955    73800.00      500.00     1904.00
000140 HEATHER      A       NICHOLLS        C01      1793    12/15/2006 ANALYST       18 F   01/19/1976    68420.00      600.00     2274.00
000150 BRUCE                ADAMSON         D11      4510    02/12/2002 DESIGNER      16 M   05/17/1977    55280.00      500.00     2022.00
000160 ELIZABETH    R       PIANKA          D11      3782    10/11/2006 DESIGNER      17 F   04/12/1980    62250.00      400.00     1780.00
000170 MASATOSHI    J       YOSHIMURA       D11      2890    09/15/1999 DESIGNER      16 M   01/05/1981    44680.00      500.00     1974.00
000180 MARILYN      S       SCOUTTEN        D11      1682    07/07/2003 DESIGNER      17 F   02/21/1979    51340.00      500.00     1707.00
000190 JAMES        H       WALKER          D11      2986    07/26/2004 DESIGNER      16 M   06/25/1982    50450.00      400.00     1636.00
000200 DAVID                BROWN           D11      4501    03/03/2002 DESIGNER      16 M   05/29/1971    57740.00      600.00     2217.00
000210 WILLIAM      T       JONES           D11      0942    04/11/1998 DESIGNER      17 M   02/23/2003    68270.00      400.00     1462.00
000220 JENNIFER     K       LUTZ            D11      0672    08/29/1998 DESIGNER      18 F   03/19/1978    49840.00      600.00     2387.00
000230 JAMES        J       JEFFERSON       D21      2094    11/21/1996 CLERK         14 M   05/30/1980    42180.00      400.00     1774.00
000240 SALVATORE    M       MARINO          D21      3780    12/05/2004 CLERK         17 M   03/31/2002    48760.00      600.00     2301.00
000250 DANIEL       S       SMITH           D21      0961    10/30/1999 CLERK         15 M   11/12/1969    49180.00      400.00     1534.00
000260 SYBIL        P       JOHNSON         D21      8953    09/11/2005 CLERK         16 F   10/05/1976    47250.00      300.00     1380.00
000270 MARIA        L       PEREZ           D21      9001    09/30/2006 CLERK         15 F   05/26/2003    37380.00      500.00     2190.00
000280 ETHEL        R       SCHNEIDER       E11      8997    03/24/1997 OPERATOR      17 F   03/28/1976    36250.00      500.00     2100.00
000290 JOHN         R       PARKER          E11      4502    05/30/2006 OPERATOR      12 M   07/09/1985    35340.00      300.00     1227.00
000300 PHILIP       X       SMITH           E11      2095    06/19/2002 OPERATOR      14 M   10/27/1976    37750.00      400.00     1420.00
000310 MAUDE        F       SETRIGHT        E11      3332    09/12/1994 OPERATOR      12 F   04/21/1961    35900.00      300.00     1272.00
000320 RAMLAL       V       MEHTA           E21      9990    07/07/1995 FIELDREP      16 M   08/11/1962    39950.00      400.00     1596.00
000330 WING                 LEE             E21      2103    02/23/2006 FIELDREP      14 M   07/18/1971    45370.00      500.00     2030.00
000340 JASON        R       GOUNOT          E21      5698    05/05/1977 FIELDREP      16 M   05/17/1956    43840.00      500.00     1907.00
200010 DIAN         J       HEMMINGER       A00      3978    01/01/1995 SALESREP      18 F   08/14/1973    46500.00     1000.00     4220.00
200120 GREG                 ORLANDO         A00      2167    05/05/2002 CLERK         14 M   10/18/1972    39250.00      600.00     2340.00
200140 KIM          N       NATZ            C01      1793    12/15/2006 ANALYST       18 F   01/19/1976    68420.00      600.00     2274.00
200170 KIYOSHI              YAMAMOTO        D11      2890    09/15/2005 DESIGNER      16 M   01/05/1981    64680.00      500.00     1974.00
200220 REBA         K       JOHN            D11      0672    08/29/2005 DESIGNER      18 F   03/19/1978    69840.00      600.00     2387.00
200240 ROBERT       M       MONTEVERDE      D21      3780    12/05/2004 CLERK         17 M   03/31/1984    37760.00      600.00     2301.00
200280 EILEEN       R       SCHWARTZ        E11      8997    03/24/1997 OPERATOR      17 F   03/28/1966    46250.00      500.00     2100.00
200310 MICHELLE     F       SPRINGER        E11      3332    09/12/1994 OPERATOR      12 F   04/21/1961    35900.00      300.00     1272.00
200330 HELENA               WONG            E21      2103    02/23/2006 FIELDREP      14 F   07/18/1971    35370.00      500.00     2030.00
200340 ROY          R       ALONZO          E21      5698    07/05/1997 FIELDREP      16 M   05/17/1956    31840.00      500.00     1907.00

  42 record(s) selected.


Having validated that the DB2 container was clean-and-green, I then proceeded to download a recent set of DB2 JDBC drivers from here: -


and extracted the relevant JARs to a newly created subdirectory: -

mkdir /db2jars
tar xzvf /tmp/v11.1.4fp4_jdbc_sqlj.tar.gz -C /tmp
unzip /tmp/jdbc_sqlj/db2_db2driver_for_jdbc_sqlj.zip -d /tmp
cp /tmp/db2jcc.jar /db2jars
cp /tmp/db2jcc4.jar /db2jars

and then created a TAR file containing those JARs: -

tar cvf dependency.tar /db2jars

I then created a Dockerfile: -

vi Dockerfile.DB2

FROM scratch
ADD dependency.tar /

and built an image: -

docker build -t db2jars:latest -f ~/Dockerfile.DB2 .

Sending build context to Docker daemon  1.459GB
Step 1/2 : FROM scratch
 ---> 
Step 2/2 : ADD dependency.tar /
 ---> d7cfd446014e
Successfully built d7cfd446014e
Successfully tagged db2jars:latest

which gave me a Docker image: -

docker images

REPOSITORY                                            TAG                 IMAGE ID            CREATED             SIZE
db2jars                                               latest              69deb549f3f0        5 days ago          8.05MB

I then created a second Dockerfile, for WebSphere Liberty Profile: -

vi ~/Dockerfile.WLP

FROM websphere-liberty:latest
ENV LICENSE accept

COPY --from=db2jars:latest /db2jars /db2jars
COPY JdbcTestDB2.class /
CMD ["java","-cp","/:/db2jars/db2jcc.jar","JdbcTestDB2","172.18.10.190","50000","sample","db2inst1 ","passw0rd"]

and built the image: -

docker build -t wlp:latest -f ~/Dockerfile.WLP .

Sending build context to Docker daemon  1.459GB
Step 1/5 : FROM websphere-liberty:latest
 ---> 5005e127f3b4
Step 2/5 : ENV LICENSE accept
 ---> Using cache
 ---> 4fb20054c1b4
Step 3/5 : COPY --from=db2jars:latest /db2jars /db2jars
 ---> Using cache
 ---> 7e7ba23d46d4
Step 4/5 : COPY JdbcTestDB2.class /
 ---> Using cache
 ---> ffe1ef5dca2c
Step 5/5 : CMD ["java","-cp","/:/db2jars/db2jcc.jar","JdbcTestDB2","172.18.10.190","50000","sample","db2inst1 ","Qp455w0rd"]
 ---> Using cache
 ---> 490fac5dabe4
Successfully built 490fac5dabe4
Successfully tagged wlp:latest

I then instantiated the container: -

wlp=`docker run -d -t -p 80:9080 -p 443:9443 wlp:latest`

and checked the logs: -

docker logs $wlp -f

000010 CHRISTINE HAAS
000020 MICHAEL THOMPSON
000030 SALLY KWAN
000050 JOHN GEYER
000060 IRVING STERN
000070 EVA PULASKI
000090 EILEEN HENDERSON
000100 THEODORE SPENSER
000110 VINCENZO LUCCHESSI
000120 SEAN O'CONNELL
000130 DELORES QUINTANA
000140 HEATHER NICHOLLS
000150 BRUCE ADAMSON
000160 ELIZABETH PIANKA
000170 MASATOSHI YOSHIMURA
000180 MARILYN SCOUTTEN
000190 JAMES WALKER
000200 DAVID BROWN
000210 WILLIAM JONES
000220 JENNIFER LUTZ
000230 JAMES JEFFERSON
000240 SALVATORE MARINO
000250 DANIEL SMITH
000260 SYBIL JOHNSON
000270 MARIA PEREZ
000280 ETHEL SCHNEIDER
000290 JOHN PARKER
000300 PHILIP SMITH
000310 MAUDE SETRIGHT
000320 RAMLAL MEHTA
000330 WING LEE
000340 JASON GOUNOT
200010 DIAN HEMMINGER
200120 GREG ORLANDO
200140 KIM NATZ
200170 KIYOSHI YAMAMOTO
200220 REBA JOHN
200240 ROBERT MONTEVERDE
200280 EILEEN SCHWARTZ
200310 MICHELLE SPRINGER
200330 HELENA WONG
200340 ROY ALONZO

Magic has occurred ....

For the record, the reason that this happens is in the last line of Dockerfile.WLP: -

...
CMD ["java","-cp","/:/db2jars/db2jcc.jar","JdbcTestDB2","172.18.10.190","50000","sample","db2inst1 ","passw0rd"]
...

which uses this Java class: -

JdbcTestDB2.java 

import java.sql.Connection ;
import java.sql.DriverManager ;
import java.sql.ResultSet ;
import java.sql.Statement ;
import java.sql.SQLException;

class JdbcTestDB2
{
public static void main (String args[])
{
try
{
Class.forName("com.ibm.db2.jcc.DB2Driver");
}
catch (ClassNotFoundException e)
{
System.err.println (e) ;
System.exit (-1) ;
  }
  String hostname      = args[0];
  String port          = args[1];
  String dbName        = args[2];
  String userName      = args[3];
  String password      = args[4];
  String sslConnection = "false";

  java.util.Properties properties = new java.util.Properties();
  properties.put("user",userName);
  properties.put("password", password);

String url = "jdbc:db2://" + hostname + ":" + port + "/" + dbName;
try
{
      Connection connection = DriverManager.getConnection(url,properties);

      String query = "select EMPNO,FIRSTNME,LASTNAME from DB2INST1.EMPLOYEE" ;

      Statement statement = connection.createStatement () ;
ResultSet rs = statement.executeQuery (query) ;

while ( rs.next () )
System.out.println (rs.getString (1) + " " + rs.getString(2) + " " + rs.getString(3)) ;
connection.close () ;
}
  catch (java.sql.SQLException e)
{
System.err.println (e) ;
System.exit (-1) ;
}
}
}

to connect to the DB2 container, using the IP address of the box hosting the container, on port 50000 ( which is mapped from container to host when we start the DB2 container, via -p 50000:50000 

So, when I'm ready to rev the DB2 drivers, I merely need to repeat the above steps to download them, TAR them up, create a new Docker image, and rebuild the WLP image, without making a huge impact upon the size / layers of the image itself.

As I said at the beginning, this is ONE way of solving the problem BUT FUN!!!!

No comments:

Note to self - use kubectl to query images in a pod or deployment

In both cases, we use JSON ... For a deployment, we can do this: - kubectl get deployment foobar --namespace snafu --output jsonpath="{...