Loading unique passwords with OIM bulk load

Using Oracle Identity Manager's bulk load tool is a great way to load large numbers of user records into OIM in an efficient and performant way. The standard and documented usage of the tool does not make provision for the loading of unique user passwords, though. This article describes an approach that can be used to achieve that requirement. This post is also part of the OIM 11g Academy Series.

How does OIM bulk load work?

The tool uses the SQL Loader functionality of the underlying Oracle Database to "pump" records into OIM's tables, rather than working through the Java API layer. Those familiar with OIM's architecture and functionality will know that there is typically a large amount of processing associated with adding a user record via the user interface (or the API layer underneath) - processing necessary for data integrity and validity checking, username and password generation, role assignment, policy-based provisioning and so on. While the reconciliation engine provides a more efficient way to create users, by allowing costly operations to be run in batch via the API layer, no approach to user loading that works through the API layer could ever perform as well as one that goes directly to the database. That's probably just an interpretation of Occam's razor, if you think about it.

The direct database approach used by bulk load does, of course, impose some restrictions, one of these being the inability to perform encryption of user attributes during the load process. Since the user password is always stored in an encrypted form (along with other attributes if required), this limitation makes it impossible to load a unique password for each user, since the SQL Loader tool doesn't have access to the encryption routines (and keys) necessary to encrypt these (plain-text) password values at the time of loading.

What is the use case?

There are two major scenarios in which we might want to load a unique password value for users (and, I'm sure, hundreds of minor ones).

The first is where we are migrating users in bulk from some other repository and wish to migrate their existing passwords with their accounts. The assumption here is that the password values are available in plain-text at the time of loading, which admittedly is not often the case.

The second is where we would like to do a bulk take-on of new users and would like to generate and load a random initial password for each user, which would then be shared with the users out of band.We could, of course, perform this activity post load using the API layer although this approach would be significantly less performant.

The approach

The approach used to enable direct database loading of unique passwords is relatively straightforward; it involves running the plain-text passwords via a pre-encryption utility in order to generate the appropriate encrypted values (or cipher-text values, to be correct) using OIM's encryption key. These cipher-text values can then be loaded directly into the database using the existing OIM bulk load tool. The specifics of the pre-encryption utility, as well as the process to be followed, will be discussed below, but we do need to mention a caveat at this point:

 After running bulk load, there is a "bulk load post process" scheduled task which needs to be run. By default, this task will attempt to generate a random password for each user. Be sure to disable this option via the scheduled task options, if you are loading unique passwords via bulk load.

Pre-encryption

The pre-encryption utility can be a simple piece of Java code that encrypts plain-text values using the OIM encryption key. The sample below should demonstrate the concept.  

import com.thortech.xl.crypto.tcCryptoException;
import com.thortech.xl.crypto.tcCryptoUtil;

public class EncryptionUtil {

 public static void main(String[] args) {
  if (args.length != 1 ) {
   System.out.println("Usage: PINEncryptionUtil  ");
   System.exit(1);
  }
  if (System.getProperty("DOMAIN_HOME") == null) {
   System.out.println("Please set DOMAIN_HOME system property");
   System.exit(1);
  }
  if (System.getProperty("OIM_HOME") == null) {
   System.out.println("Please set OIM_HOME system property");
   System.exit(1);
  }
  if (System.getProperty("oracle.security.jps.config") == null) {
   System.out.println("Please set oracle.security.jps.config system property");
   System.exit(1);
  }
  if (System.getProperty("XL.HomeDir") == null) {
   System.out.println("Please set XL.HomeDir system property");
   System.exit(1);
  }
  try {
   String plainText = args[0];
   String cipherText = tcCryptoUtil.encrypt(plainText, "DBSecretKey");
   System.out.println(cipherText); 
  } catch (Exception e) {
   System.out.println("Error encrypting value");
  }
 }
}

Ultimately, the code above is not very exciting in itself. All that it does is take a plain-text value as input and print the cipher-text value as output. Line 29 is where the magic happens, but as you can probably tell from all the environment checking that we do beforehand, we need to do quite a bit of set-up in order to make this code work. Once we have our encryption utility class complied, we can use a run script similar to the below to ensure that we are setting the correct classpath, as well as the correct JVM options, to run the utility.

#Environment variable setup.... modify this section for your environment.

export JAVA_HOME=/opt/oracle/java
export PATH=$JAVA_HOME/bin:$PATH
export MW_HOME=/opt/oracle/Middleware
export WL_HOME=$MW_HOME/wlserver_10.3
export ORACLE_HOME=$MW_HOME/Oracle_IDM1
export DOMAIN_HOME=/opt/oracle/Middleware/user_projects/domains/oim_domain

#Do not modify below

. $ORACLE_HOME/server/bin/setEnv.sh
export CLASSPATH=./classes:$ORACLE_HOME/server/platform/iam-platform-utils.jar:$ORACLE_HOME/server/platform/iam-platform-auth-server.jar:$ORACLE_HOME/server/apps/oim.ear/APP-INF/lib/csv.jar:$MW_HOME/oracle_common/modules/oracle.jrf_11.1.1/jrf-api.jar:$MW_HOME/oracle_common/modules/oracle.jmx_11.1.1/jmxframework.jar:$MW_HOME/oracle_common/modules/oracle.jmx_11.1.1/jmxspi.jar:$MW_HOME/Oracle_IDM/modules/oracle.oes_11.1.1/thirdparty/identitystore.jar:$MW_HOME/oracle_common/modules/oracle.pki_11.1.1/oraclepki.jar:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-manifest.jar:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jacc-spi.jar:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-common.jar:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-internal.jar:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-wls-trustprovider.jar:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-api.jar:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-platform.jar:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-wls.jar:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-unsupported-api.jar:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-az-common.jar:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-az-api.jar:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-az-sspi.jar:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-az-management.jar:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-ee.jar:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-mbeans.jar:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-az-rt.jar:$MW_HOME/modules/javax.management.j2ee_1.0.jar:$MW_HOME/modules/javax.servlet_1.0.0.0_2-5.jar:$CLASSPATH

java -DDOMAIN_HOME=$DOMAIN_HOME -DOIM_HOME=$ORACLE_HOME/server/ -Doracle.security.jps.config=$DOMAIN_HOME/config/fmwconfig/jps-config-jse.xml -DXL.HomeDir=/$DOMAIN_HOME/config/fmwconfig/ -DJAVA_HOME=$JAVA_HOME -DkeyStoreFileName=.xldatabasekey com.oracle.ateam.custom.EncryptionUtil  $1

All the usual warnings about plain-text passwords obviously still apply here. Please be extra careful not to leave a text file full of plain-text passwords lying around on the file system, or anything like that.

The bulk load process

Now that we have a utility that we can use to pre-encrypt a plain text value, we can generate a bulk load CSV file containing encrypted password values, to load as normal. Below is a very stripped down sample of what such a file could look like.

USR_LOGIN,USR_FIRST_NAME,USR_LAST_NAME,USR_PASSWORD
testuser1,Test,User1,"6726:A2ZLndw90HzYrEtOhkl6HQ=="
testuser2,Test,User2,"3090:tEZQiear2ilc+HSZeq7Bcw=="

Be sure to enclose your encrypted values in quotes, or else they won't load correctly.

During the execution of the bulk load process, OIM's tool will ask you to specify the name of an existing user, whose password will be copied to all the user records being loaded. If you are providing a value for USR_PASSWORD in the file, your unique value will override the default, so you can enter any valid user name for this step.

You can use the same approach to pre-encrypt values for other database columns as well, not just the user's password

A final (but important) word

Please be sure to read the official product documentation related to the OIM bulk load utility and ensure that you are comfortable with its usage before attempting this approach. Bear in mind, also, that this is far from a silver bullet, or "one-size-fits-all" solution; rather, it is a technique that has been found useful in certain scenarios and may prove effective in your own environment too.