IBM Connections 4 - Manually populating the Profiles database - and learning lessons on the way

In this post, I describe how I resolved an issue using IBM Tivoli Directory Integrator 7.1 to populate the Profiles database for IBM Connections 4, using IBM Lotus Domino 8.5.3 FP2 as the source. There's a large amount of trial n' error involved, but I wanted to show my working ( and the tips and tools that I applied ), rather than simply jumping to the solution.

Following on from an earlier post - IBM Connections 3 - Manually populating the Profiles database - who needs wizards anyway ? - I wanted to repeat my previous experience, and set things up manually, rather than relying upon the GUI ( launched by populationWizard.sh ), mainly because I often find myself working on client servers where there's NO GUI.

Having installed TDI 7.1 ( see this post ) and unpacked the Connections 4 Wizards: -

$ cd /tmp

$ tar xvf /mnt/hgfs/Software/IC4/IBM_Connection40_Wzd_LNXAIX_CIA3HML.tar

I then created a working directory for my Connections-related assembly lines: -

$ mkdir /opt/IBM/TDI/V7.1/tdisol_DOMINO

and then copied the assembly lines, property files etc. from the /tmp directory: -

$ cd /opt/IBM/TDI/V7.1/tdisol_DOMINO

$ cp -R /tmp/Wizards/TDIPopulation/linux/TDI/* .

I then manually edited profiles_tdi.properties and set the following properties: -

dbrepos_jdbc_driver=com.ibm.db2.jcc.DB2Driver
dbrepos_jdbc_url=jdbc:db2://ic4.uk.ibm.com:50000/PEOPLEDB

dbrepos_username=db2inst1
dbrepos_password=passw0rd

source_ldap_url=ldap://ic4.uk.ibm.com:389
source_ldap_user_login=CN=wasadmin,O=ibm
source_ldap_user_password=passw0rd
source_ldap_search_base=O=ibm

source_ldap_search_filter=(&(cn=*)(objectClass=dominoPerson))

When I ran the script populate_from_dn_file.sh using the following entry in collect.dns: -

CN=wasadmin,O=ibm

I received: -

Platform: 'Generic'
CLFRN0027I: After operation, success records is 0, duplicate records 0, failure records is 1, and last successful entry is null.
Populate of Database Repository failed

Digging into the logs ( specifically /opt/IBM/TDI/V7.1/tdisol_DOMINO/logs/PopulateDBFromDNFile.log ) I see: -

2012-09-13 15:54:13,207 INFO  [com.ibm.di.log.FileRollerAppender.a84dda54-ee01-4fee-a46a-049ed3ef1676] - CTGDIS087I Iterating.
2012-09-13 15:54:13,397 WARN  [com.ibm.di.log.FileRollerAppender.a84dda54-ee01-4fee-a46a-049ed3ef1676] - CLFRN0209E: Validation failed for field guid. Value is .
2012-09-13 15:54:13,402 ERROR [com.ibm.di.log.FileRollerAppender.a84dda54-ee01-4fee-a46a-049ed3ef1676] - CLFRN1183E: Validation failed for entry CN=wasadmin,O=ibm.
2012-09-13 15:54:13,404 ERROR [com.ibm.di.log.FileRollerAppender.a84dda54-ee01-4fee-a46a-049ed3ef1676] - !com.ibm.di.exceptions.SkipEntryException: CTGDIS393I Throwing this exception to tell the AssemblyLine to skip the current Entry. If used in an EventHandler, this exception tells the EventHandler to skip the remaining actions.!
2012-09-13 15:54:13,413 INFO  [com.ibm.di.log.FileRollerAppender.a84dda54-ee01-4fee-a46a-049ed3ef1676] - CTGDIS088I Finished iterating.

I changed collect.dns to: -

CN=wasadmin

but to no avail, so changed it back to: -

CN=wasadmin,O=ibm

I then validated the LDAP search filter: -

$ su notes -c "cd /local/notesdata; /opt/ibm/lotus/bin/ldapsearch -h ic4.uk.ibm.com -p 389 -b o=ibm -D CN=wasadmin,O=ibm -p passw0rd '(&(cn=wasadmin)(objectClass=dominoPerson))'"

which returned: -

CN=wasadmin,O=ibm
cn=wasadmin
mail=wasadmin/ibm%ibm@uk.ibm.com
displayname=wasadmin/ibm
objectclass=dominoPerson
objectclass=inetOrgPerson
objectclass=organizationalPerson
objectclass=person
objectclass=top
givenname=was
sn=admin
uid=wasadmin
maildomain=ibm

as I'd expect.

Next, I enabled LDAP debugging on the Domino server, using the Domino console command: -

> set config ldapdebug=7

and restarted Domino.

I then monitored the Domino console when I ran the populate_from_dn_file.sh process, and saw: -

<snip>

[07113:00006-3891055472] 14/09/2012 12:01:05.42 LDAP> *** No matching entries & no errors, attempting to find base or closest ancestor of wasadmin ...
[07113:00006-3891055472] 14/09/2012 12:01:05.42 LDAP> *** Base does NOT exist, no ancestor found
[07113:00006-3891055472] 14/09/2012 12:01:05.42 LDAP> ***** Count of search entries returned (total): 0 *****
</snip>

which wasn't of much use, given that I knew that the search base and search filter worked ( using LDAPSearch ).

However, turning my attention to the profiles_tdi.properties file, I then experimented with the two LDAP-related attributes: -

source_ldap_search_base=o=ibm
source_ldap_search_filter=(&(cn=*)(objectclass=dominoPerson))

but to no avail.

I also checked / validated the database related attributes: -

dbrepos_jdbc_driver=com.ibm.db2.jcc.DB2Driver
dbrepos_jdbc_url=jdbc:db2://ic4.uk.ibm.com:50000/peopledb
dbrepos_username=db2inst1
dbrepos_password=passw0rd

but again without any joy.

Finally (!), I looked back at the original errors that I saw: -

/opt/IBM/TDI/V7.1/tdisol_DOMINO/logs/PopulateDBFromDNFile.log )

2012-09-13 15:54:13,397 WARN  [com.ibm.di.log.FileRollerAppender.a84dda54-ee01-4fee-a46a-049ed3ef1676] - CLFRN0209E: Validation failed for field guid. Value is .
2012-09-13 15:54:13,402 ERROR [com.ibm.di.log.FileRollerAppender.a84dda54-ee01-4fee-a46a-049ed3ef1676] - CLFRN1183E: Validation failed for entry CN=wasadmin,O=ibm.

and did a quick Google search, finding this: -

Since upgrade to 2.5, sync_all_dns no longer runs

( I've linked to the post where the OP reports Fixed! )

which made reference to the: -

guid={function_map_from_dominoUNID}

element in map_dbrepos_from_source.properties.

When I checked the map_dbrepos_from_source.properties file in , I saw: -

guid=ibm-entryUuid

which goes a LONG way to explain it - this is the property that I'd be using if I was keying against IBM Tivoli Directory Server, rather than Lotus Domino.

TDI has a rather nice JavaScript function to infer the GUID from the dominoUNID field within the Domino Directory.

Therefore, I changed it to: -

guid={function_map_from_dominoUNID}

and, lo and behold, my assembly line burst into life - I was able to run populate_from_dn_file.sh and, c'est voila, my Profiles DB is now populated, as validated below: -

$ su - db2inst1

$ db2 connect to peopledb

$ db2 "select prof_given_name,prof_surname,prof_display_name from empinst.employee"

PROF_GIVEN_NAME PROF_SURNAME PROF_DISPLAY_NAME                                                                                                                                                                                                                                               
was admin wasadmin

$ db2 terminate

$ exit

This has been a frustrating journey but, as always, I learned a LOAD on the way, and hope that others may benefit from the solution - especially as the CLFRN0209E and CLFRN1183E messages don't currently point to any hits on Google - until this blog post goes out :-)