The “reassociation” business

Since Fusion Middleware 11.1.1.4, OPSS (Oracle Platform Security Services) support 3 types of security stores: file, OID (Oracle Internet Directory) and Oracle database. When a Weblogic server domain is first created, OPSS is “associated” to a file-based security store by default, which is ok for development purposes. But for production, that is not recommended (Please check Multiple Nodes Servers Environments section in OPSS docs). That would be ok if your whole environment is a single Weblogic domain with only one server in a single machine. But 99,99% of the cases are not like that. Usually, an SOA or WebCenter environment is composed of multiple servers in clusters spread across different machines. A file-based security store is not a scalable option. In these cases, you should look at OID or the database. Fusion Applications, a gigantic set of apps, adopt OID as the security store.

The OPSS security store is a composite of policies, credentials, keys and audit services. Notice that I am leaving the identity store service out. OPSS delegates the identity store service to the identity providers configured in WebLogic server.

As a side note, OPSS is not a product, but a set of security services used by Fusion Middleware. If you’re a Fusion Middleware user, trying to understand OPSS is a great idea.

This post is about the nitty-gritty details of configuring (or reassociating) a Weblogic server domain (or multiple domains) to a different type of security store. That’s where the term “reassociation” comes from.

The information presented here is a small subset, but complements and sometimes overlaps “Configuring OPSS Security Store” documentation (reading is strongly recommended).
Before going any further on reassociation, let me talk a bit about an important character: jps-config.xml.

jps-config.xml

This is the OPSS file that describes all its services. It is located through the –Doracle.security.jps.config system property, which is set in setDomainEnv.sh script in a standard JRF (Java Required Files) domain. By the default, the property points to ${DOMAIN_HOME}/config/fmwconfig/jps-config.xml and it is defined in the variable EXTRA_JAVA_PROPERTIES. It is NOT a good idea to change it, since jps-config.xml holds several relative references to other files.



Whenever you create a BPM, SOA or WebCenter domain via the config.sh script, JRF template is automatically selected as a dependency.

That put, jps-config.xml is a domain-wide artifact. There’s no such thing as a server-level or application-level jps-config.xml. However, jps-config.xml provides the concept of contexts. They can be explored in case you want to hookup different applications to different services, but this would be a topic for another post.

When a reassociation operation is performed, configuration changes are made to jps-config.xml. In many cases, a corrupted jps-config.xml can bring your domain to a non startable state. Therefore, be very diligent and careful when making changes to it. Do NOT perform manual changes. Instead, use either Enterprise Manager or wlst.

The Policy Store

The policy store holds all security policies used by applications deployed on a Fusion Middleware instance. These include grants given to principals (users, groups, application roles) as well as to code.

For instance, if you look at the OOTB policy store of a BPM domain, you would see policies scoped into 4 applications (OracleBPMProcessRolesApp, OracleBPMComposerRolesApp, b2bui and soainfra) as well as a bunch of code-source policies (which are applicable to code in any application deployed in the domain).

In jps-config.xml, the “default” context defines the services that are by default :-) picked up by Fusion Middleware applications.

<jpsContext name="default">

 <serviceInstanceRef ref="credstore"/>

 <serviceInstanceRef ref="keystore"/>

 <serviceInstanceRef ref="policystore.xml"/>

 <serviceInstanceRef ref="audit"/>

 <serviceInstanceRef ref="idstore.ldap"/>

 <serviceInstanceRef ref="trust"/>

 <serviceInstanceRef ref="pdp.service"/>

</jpsContext>

If you follow policystore.xml up in the file, you should see a serviceInstance with that name:

<serviceInstance name="policystore.xml" 

                 provider="policystore.xml.provider" 

                 location="./system-jazn-data.xml">

   <description>File Based Policy Store Service Instance</description>

</serviceInstance>

which brings us to the OOTB file-based policy store, system-jazn-data.xml. Down in this post, we’re going to change this.

The Credential Store

The credential store securely holds credentials to be used by Fusion Middleware applications when connecting to other systems. OWSM agents, for instance, use the credential store service when a WSS username token needs to be attached to an outgoing SOAP message. Another heavy user is ADF (Application Development Framework), who uses it to store credentials required to connect to external systems. OOTB, the credential store is materialized as the cwallet.sso file pointed by the credstore serviceInstance (notice the file name itself is not specified):

<serviceInstance location="./" provider="credstoressp" name="credstore">

   <description>File Based Credential Store Service Instance</description>

</serviceInstance>

Notice this credential store is not the same as the bootstrap credential store, described in "Bootstrap cwallet.sso" section down below.

Reassociating to OID

There are two options to do reassociation: Enterprise Manager and wlst. They are very well covered in the OPSS documentation, but let me still explore them here a little bit more. I hope to add some details that are not very clear in the docs.

First and foremost, make sure to satisfy a requirement: create a root node in the LDAP server that is going to hold our security store tree. Forgetting this is a very common mistake.

Enterprise Manager

Navigate to the drop-down menu for the Weblogic Domain and choose Security –> Security Provider Configuration



Notice that both the policy store and the credential store are of the same store type: file. That tells us something: the OPSS Security store can only be persisted in one physical store type for production. Persisting credentials to a different store from keys or policies is not supported or recommended.

By clicking “Change Store Type” button, you get:



The LDAP Server Details properties are straightforward. Those under Root Node Details deserves some comments.

Root DN: the node you’ve manually created in OID before.

Create New Domain: no, this is not going to create a new Weblogic domain. It determines whether or not the new security store (OID) is going to be bootstrapped with data from the source security store (in this case, the file-based policy, credential and keystore). Unchecking the box is relevant when you want more than one Weblogic domain sharing the security store. This flag corresponds to the join parameter in the wlst command, shown below.

Domain Name: Enterprise Manager uses the Weblogic domain name as a convenience, but it can actually be any arbitrary string. This name is going to manifest itself as a container node, under which OPSS security data are migrated. Several Weblogic domains can bind to the same container, but a single Weblogic domain cannot bind to different containers. I am talking about this kind of deployment:



Within OID, cn=SecurityStore is the Root DN. cn=JPSContext is implicitly created upon the first reassociation.

Notice there’s no reference to a particular Weblogic server. That essentially means that you bind one domain as a whole to a container node. Within a container, specific applications bind to application  stripes (nodes cn=AppsA, cn=AppsB, cn=AppsC, cn=AppsD).  Although not shown in the picture, any number of applications can bind to a given stripe, but the same application can bind to one and only one stripe.

wlst

Now that we understand the behavior of those reassociation properties, the wlst command is straightforward. It is an online command, which means you must connect to the Admin Server to execute it.

> reassociateSecurityStore(domain="farm1", admin="cn=orcladmin", 
password="welcome1", ldapurl=ldap://localhost:3060, servertype="OID", 
jpsroot="cn=SecurityStore",[join=”trueOrfalse”]) 

where:

domain corresponds to the policy container that the WebLogic domain will bind to and it does NOT need to be named as the Weblogic domain. Notice the format: you do NOT prepend “cn=” to the value.

servertype is the security store type. Supported values are “OID” and “DB_Oracle”.

jpsroot corresponds to the Root DN node in OID, that, again, has to be manually created upfront.

join is optional, but of uttermost importance. It corresponds to the “Create New Domain” checkbox in Enterprise Manager. The default value (if unspecified) is “false”, which means a container node specified by the domain parameter is going to be created in OID and the security artifacts (policies, credentials, keys) migrated. As a pre-requisite, OPSS first automatically seeds an LDAP schema in OID. If there’s a container with the same name already created, you’re likely going to be presented with an error. If join is set to “true”, any security artifacts in the Weblogic domain are left alone and not migrated to the security store. 

Details on admin and password parameters in the “Bootstrap cwallet.sso” section below.

Bootstrap cwallet.sso

When “reassociated” to OID, Weblogic needs to know which credentials to use when connecting to the server. Such credentials are, by default, kept in the location that is pointed by the following jpsContext in jps-config.xml:

<jpsContext name="bootstrap_credstore_context">
 <serviceInstanceRef ref="bootstrap.credstore"/>
</jpsContext>

Looking up the file, you find:

<serviceInstance location="./bootstrap" 
                 provider="credstoressp" 
                 name="bootstrap.credstore">
 <property value="./bootstrap" name="location"/>
</serviceInstance>

That implicitly means cwallet.sso file in ./bootstrap folder. This is an encrypted file which only authorized code is allowed access.

Important note: make sure the used credential has permissions to write to OID if you expect to allow changes to your policies through the Weblogic domain. That’s true, I’d say, always, or in 99.99% of the cases.

What if you need to change these credentials later? wlst command to the rescue (in wlst, type help(‘opss’) for a list of OPSS-related commands. You will see modifyBootStrapCredential):

> modifyBootStrapCredential(jpsConfigFile='<filepath>',username='<username>', password='<password>')

where

jpsConfigFile = path of the valid jps config file from which the context is read
username = distinguished name of the user.
password = the password to be reset.

Example:

> modifyBootStrapCredential(jpsConfigFile='/opt/wls/oracle/middleware/user_projects/domains/soa_domain/config/fmwconfig/jps-config.xml',username='cn=orcladmin', password='welcome1')

 Note: modifyBootStrapCredential is an offline command.

Reassociating to Oracle database

My colleague Kavitha Srinivasan already describes the process along with some benign error messages when reassociating to an Oracle database. I will add a link to her post once she makes it public. Here I simply want to mention that the same principles discussed in the OID section apply.

For the reader convenience, I repeat here the 2 pre-requisites for Oracle database reassociation:

1) An OPSS schema needs to be created in the database. This is done using the RCU (Repository Creation Utility) tool. Here’s the RCU screenshot (notice the Metadata Services scheme is automatically selected as a dependency once you select Oracle Platform Security Services):



2) A data source needs to be created in Weblogic. It must be non-XA with support for global transactions disabled. Thanks Kavitha for such an important detail.

Enterprise Manager

Follow the same path as in the OID reassociation up to this screen:



Notice the Root DN parameter value. Differently than the OID case, it does NOT need to be previously created and it also does NOT need to follow an LDAP DN format (starting with “cn=”). This is just a convention.

Here’s the typical output of a successful reassociation:

wlst

Please check reassociateSecurityStore command.

For your convenience, here’s an example:

> reassociateSecurityStore(domain="farm2", servertype="DB_ORACLE", datasourcename="jndi/OPSS_DS", jpsroot="cn=SecurityStore",[admin="adminAccnt"], [password="passWord"],[join="trueOrfalse"])

Inform admin and password parameters only if your data source itself is protected. Their values are going to be stored in the bootstrap credential store. Notice they do NOT correspond to the database credentials used by the data source (these are actually defined in the data source itself in Weblogic). I guess the join parameter is understood by now. If not, take a look at the “Reassociating to OID” section above. 

After Reassociation

This is how jps-config.xml looks like after reassociation to Oracle DB:

<jpsContext name="default">
<serviceInstanceRef ref="credstore.db"/>
<serviceInstanceRef ref="keystore.db"/>
<serviceInstanceRef ref="policystore.db"/>
<serviceInstanceRef ref="audit"/>
<serviceInstanceRef ref="idstore.ldap"/>
<serviceInstanceRef ref="trust"/>
<serviceInstanceRef ref="pdp.service"/>
</jpsContext>

Look up in jps-config.xml for credstore.db, policystore.db and keystore.db serviceInstances. They all refer to the database via the props.db.1 property. For example, policystore.db is like:

<serviceInstance provider="policystore.provider" name="policystore.db">
 <property value="DB_ORACLE" name="policystore.type"/>
 <propertySetRef ref="props.db.1"/>
</serviceInstance>

 And props.db.1:

<propertySet name="props.db.1">
 <property value="cn=soa_domain" name="oracle.security.jps.farm.name"/>
 <property value="DB_ORACLE" name="server.type"/>
 <property value="cn=policystore" name="oracle.security.jps.ldap.root.name"/>
 <property value="jndi/OPSS_DS" name="datasource.jndi.name"/>
</propertySet>

 Does the value of datasource.jndi.name (jndi/OPSS_DS) look familiar?

Important Note: You may be tempted to get rid of some files after reassociation, like system-jazn-data.xml, for example. Not a good idea. It’s true that it is completely out of the runtime picture, but there are some implications to the JMX framework used by Enteprise Manager and you may want it later in case you need to revert your security store to file again.

Reassociating to file

As you may have noticed, “file” is not a supported value for the servertype parameter in the reassociateSecurityStore command. Indeed, the only way to switch back to file-based security store is changing jps-config.xml manually, which is NOT a good idea. Therefore, the best thing you can do is backing up jps-config.xml before running any reassociation, so you can revert it back later on if needed.

What if you made changes to the security store (creating new policies, for example) in OID or DB mode and want those policies in the file-based policy store? Check wlst’s migrateSecurityStore command (help(‘migrateSecurityStore’)), which is actually a good topic for a future post.

Enjoy your reassociations! :-)