OAG/OES Integration for Web API Security: skin and guts

Introduction

When it comes to defining a strategy for web API security, OAG (Oracle API Gateway) and OES (Oracle Entitlements Server) together present a very interesting choice and are a very powerful combination indeed.

In this post we're going to take a look at what each component brings in (the skin) and then get our hands on actually describing the integration in detail (the guts).

OAG is designed to inspect and act on various types of messages that are delivered to it or just pass through it. It's usually positioned to be deployed on the DMZ (the De-Militarized Zone) within corporate networks. As such, it can block malicious traffic, authenticate users with a variety of protocols, integrate with anti-virus products, perform message throttling, thus delivering only the good stuff to your intranet servers and also off-loading them, decisively contributing to achieve some IT operational SLAs. More than that, OAG can switch protocols and transform messages. For instance, an organization may have SOAP-based web services and want to expose them as REST without any re-writing. Or implement SAML federation without touching origin systems. Or talk Kerberos or OAuth with clients and speak SAML with back-end servers. Or use it as an FTP server so that incoming files are immediately sent to a processing pipeline. The possibilities are numerous. Having mentioned these few features and examples, it's not unreasonable to think deploying OAG inside intranets. And that's not unusual, actually. It is a nice bridge with obvious benefits.

OES is designed to provide fine-grained authorization with externalized policies to client applications. It takes the coding of access decisions away from developers. Besides the obvious security pro, it shortens the change cycle, when a new security policy needs to be deployed. You simply avoid going through all the phases required for re-deploying your application just because of that change. It's true the new policy needs testing, but that's nowhere near when compared to what it takes to re-deploy a new application version. The time to market is drastically reduced. Now to the fine-grained part. OES can take a bunch of aspects in consideration when authorizing: the user identity, user roles, user attributes, context information about the request being made (like originating IP address), factors external to the request (like time of day, day of week, etc) and, of course, request data. Those combined makes it a very powerful authorization engine. It's not coincidence that OES is the component behind OAM's (Oracle Access Manager) authorization engine.

While OAG itself brings in authorization capabilities, in this field OES offers a much richer model. And if the organization already employs OES elsewhere, integrating it with OAG makes a lot of sense, because we end up with a single and consistent approach for authorization across applications.

The Integration

OES basic architecture comprises a server and different client modules, called SMs (Security Modules). The server connects to a repository where policies are physically kept. The SMs are attached to client applications and connect either to OES server or to the repository directly, depending on their configured mode (I will touch up on this later). There are SMs available for Java, RMI, web services, Weblogic server, Websphere, JBoss, MS Sharepoint. When integrating with OAG, a Java SM is used. Despite its core being a C process, OAG forks up a JVM for some of its functions.

The integration hook between OAG and OES is the "OES 11g Authorization" filter, as seen below:

This is how OAG delegates authorization decisions to OES. Under the covers, an OpenAZ API authorization call is made to the SM. In the filter, we observe the following:

• Resource: the OES resource for which authorization is being requested. There's an implicit formation rule here: <OES_APPLICATION_NAME>/<OES_RESOURCE_TYPE>/<OES_RESOURCE_NAME>. Later on I will show how these placeholders map to the OES policy.

• Action: the action supported by the policy. More later on the OES policy.

• Environmental/Context attributes: any extra information that you want to pass in to OES. These map to attributes in the OES policy. In this example, INVOKING_APPLICATION is an attribute used in a policy condition.

Besides, OAG always passes what's in the message authentication.subject.id attribute, who basically defines the authenticated user principal name within the executing OAG circuit instance.

Ok, with these in mind, let's look at what the OES policy looks like:

Read back again the OAG filter description and realize the mappings. What isn't shown in the OES policy is <OES_RESORCE_TYPE> and the authenticated user.

As for <OES_RESOURCE_TYPE>, it suffices saying that the target name /resources/empSalaries is of type restapi, as shown:

The authenticated user is implicit in the policy. And in this case, it is going to be authorized only if it has Managers as one of its application roles.

So far, so good, but OAG can only talk to OES if it has the SM properly installed and configured.

Installing and Configuring OES SM for OAG

1 - Download and install OES Client in the same machine as OAG. It's an Oracle installer and simply copies the binaries to a given location. Let's call it OES_CLIENT_HOME.

2 - Create the SM in OES Console. Bind the applications to it. In the screen shot below, oag_sm Security Module is bound to MyServices application.

3 - Edit $OES_CLIENT_HOME/oessm/SMConfigTool/srmconfig.java.controlled.prp, pointing it to OES admin server. The following properties are to be edited:

• oracle.security.jps.runtime.pd.client.policyDistributionMode=controlled-push <change this only if you want to have another distribution mode>
• oracle.security.jps.runtime.pd.client.RegistrationServerHost=<OES Admin Server>
• oracle.security.jps.runtime.pd.client.RegistrationServerPort=<OES Admin Server SSL port>

OES supports 3 distribution modes: controlled-push, controlled-pull and uncontrolled.

• controlled-push means that policies are distributed from OES server to the SM. The SM provides a listener that the OES server connects to. In OES Console, once the Distribute button is clicked, policies are *immediately* distributed to the configured SMs. The Distribute button doesn't actually distribute policies. It simply marks policies as "Ready for Distribution". In the case of push, policies are distributed right after. From this point on, SM needs no connection at all to the OES server (it can even be shut down), since policies are all local to the SM.

Note: I am only showing the image to illustrate. Trying to distribute policies at this point is going to generate an error, because the SM isn't yet registered with OES server.

• controlled-pull means that the SM pulls policies on a defined frequency (default 10 min) directly from the *OES repository* into its local cache. It doesn't get policies through OES server. But still, in other to be pulled, policies do need to be marked "Ready for Distribution" in OES Console.

• uncontrolled means that SMs read policies directly from OES repository on demand and updates the local cache if the requested policy is not available locally. And the SM still pulls new policies and changes from OES repository periodically. As a result, OES repository is supposed to be up and reachable by SMs at all times. Policies don't need to be marked "Ready for Distribution", i.e., there's no further control once policies are created/changed in OES.

Note: controlled distribution modes are only supported if the repository is database-based.

4 - Run $OES_CLIENT_HOME/oessm/bin/config.sh -smConfigId <sm_name> -prpFileName $OES_CLIENT_HOME/oessm/SMConfigTool/smconfig.java.controlled.prp

<sm_name> MUST match the SM you've created in step 2 before. In this case, oag_sm.

In this step you're basically enrolling the SM, but no configuration gets written to OES server. A couple of files are generated at this point under $OES_CLIENT_HOME/oes_sm_instances/<sm_name>. All of them are important, but the most relevant one is ./config/jps-config.xml, where you can find the following configuration data about the SM. The DistributionServicePort is a random port number picked at enrollment time used by the SM to listen for policy distribution events.

<serviceInstance name="pdp.service" provider="pdp.service.provider">

            <description>Runtime PDP service instance</description>

            <property name="oracle.security.jps.runtime.pd.client.policyDistributionMode" value="controlled-push"/>

            <property name="oracle.security.jps.runtime.pd.client.sm_name" value="oag_sm"/>

            <property name="oracle.security.jps.runtime.pd.client.SMinstanceType" value="java"/>

            <property name="oracle.security.jps.runtime.pd.client.RegistrationServerURL" value="https://slc05ylp.us.oracle.com:3002/pd-server"/>

            <property name="oracle.security.jps.runtime.pd.client.DistributionServicePort" value="16933"/>

            <property name="oracle.security.jps.pd.client.sslMode" value="two-way"/>

            <property name="oracle.security.jps.pd.client.ssl.identityKeyStoreFileName" value="/scratch/fmwapps/oes11gps2_client/oes_sm_instances/oag_sm/security/identity.jks"/>

            <property name="oracle.security.jps.pd.client.ssl.trustKeyStoreFileName" value="/scratch/fmwapps/oes11gps2_client/oes_sm_instances/oag_sm/security/trust.jks"/>

</serviceInstance>

5 - Create a file named jvm.xml under $OAG_INSTALL_HOME/apigateway/conf with the following contents. It tells OAG about OES jar files and the Security Module name. It also defines system properties used by OAG, like log4j properties file (used for dumping out SM's authorization decisions)

<ConfigurationFragment>

<Environment name="JRE_HOME" value="/scratch/fmwapps/oag_11.1.2.2.0/apigateway/Linux.x86_64/jre" />

<!-- OES Settings -->

<Environment name="OES_CLIENT_HOME" value="/scratch/fmwapps/oes11gps2_client" />

<Environment name="SM_NAME" value="oag_sm" />

<Environment name="INSTANCE_HOME" value="$OES_CLIENT_HOME/oes_sm_instances/$SM_NAME" />

<!-- Add OES Client to classpath -->

<ClassPath name="$OES_CLIENT_HOME/modules/oracle.oes.sm_11.1.1/oes-client.jar" />

<VMArg name="-Doracle.security.jps.config=$INSTANCE_HOME/config/jps-config.xml"/>

<VMArg name="-Djava.util.logging.config.file=$JRE_HOME/lib/logging.properties"/>

</ConfigurationFragment>

6 - Restart the API Gateway process. No need to restart the OAG Node Manager.

At this point, OAG's "OES 11g Authorization" filter can be safely invoked. The filter is natively aware of jvm.xml settings, so that no external resource is required to be configured in OAG Policy Studio.

If you've really followed through, at this point you might be wondering how come the filter is going to work if policies were not distributed (supposing we're in some controlled mode). The answer is that, exceptionally, policies get distributed to the SM upon the very first usage of OAG filter. We can clearly see this if we analyze the following log snippets.

OAG log snippet upon first usage of "OES 11g Authorization" filter:

DEBUG 21/May/2014:11:28:55.369 [b9da8700] run filter [Call 'Authorize Access'] {

DEBUG 21/May/2014:11:28:55.369 [b9da8700] run circuit "Authorize Access"...

DEBUG 21/May/2014:11:28:55.369 [b9da8700] run filter [Authorize Access (OES 11g Authorization)] {

DEBUG 21/May/2014:11:28:55.369 [b9da8700] creating subject from 'jane'

DEBUG 21/May/2014:11:28:55.373 [b9da8700] checking 'GET' to resource: MyServices/restapi//resources/empSalaries

DEBUG 21/May/2014:11:28:55.373 [b9da8700] env attribute name: 'INVOKING_APPLICATION' env attribute value: 'OAG'

DEBUG 21/May/2014:11:28:55.634 [b9da8700] parsing (options value 2052) XML body from input stream of type java.io.ByteArrayInputStream. ContentSource is of type java InputStream

DATA 21/May/2014:11:28:55.637 [b9da8700] getting class com.vordel.jaxprovider.libxml.XPathExpressionImpl with classLoader.loadClass()

DATA 21/May/2014:11:28:55.638 [b9da8700] loaded class com.vordel.jaxprovider.libxml.XPathExpressionImpl

DATA 21/May/2014:11:28:55.638 [b9da8700] getting class javax.xml.xpath.XPath with classLoader.loadClass()

DATA 21/May/2014:11:28:55.638 [b9da8700] loaded class javax.xml.xpath.XPath

DATA 21/May/2014:11:28:55.638 [b9da8700] getting class javax.xml.xpath.XPathConstants with classLoader.loadClass()

DATA 21/May/2014:11:28:55.638 [b9da8700] loaded class javax.xml.xpath.XPathConstants

DATA 21/May/2014:11:28:55.638 [b9da8700] getting class javax.xml.namespace.QName with classLoader.loadClass()

DATA 21/May/2014:11:28:55.638 [b9da8700] loaded class javax.xml.namespace.QName

DATA 21/May/2014:11:28:55.639 [b9da8700] getting class javax.xml.namespace.NamespaceContext with classLoader.loadClass()

DATA 21/May/2014:11:28:55.639 [b9da8700] loaded class javax.xml.namespace.NamespaceContext

DEBUG 21/May/2014:11:28:55.656 [b9da8700] Loaded XML file /scratch/fmwapps/oes11gps2_client/oes_sm_instances/oag_sm/config/jps-config.xml

DEBUG 21/May/2014:11:28:55.656 [b9da8700] parsing (options value 2052) XML body from input stream of type java.io.FileInputStream. ContentSource is of type java InputStream

DEBUG 21/May/2014:11:28:55.656 [b9da8700] release system resources for the loaded XML

DEBUG 21/May/2014:11:28:56.216 [b9da8700] parsing (options value 2052) XML body from input stream of type sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream. ContentSource is of type java InputStream

DEBUG 21/May/2014:11:28:56.253 [b9da8700] parsing (options value 2052) XML body from input stream of type sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream. ContentSource is of type java InputStream

DEBUG 21/May/2014:11:28:58.742 [31663700] parsing (options value 2052) XML body from input stream of type java.io.ByteArrayInputStream. ContentSource is of type java InputStream

DEBUG 21/May/2014:11:28:58.771 [31663700] parsing (options value 2052) XML body from input stream of type java.io.ByteArrayInputStream. ContentSource is of type java InputStream

DEBUG 21/May/2014:11:28:58.775 [31663700] parsing (options value 2052) XML body from input stream of type java.io.ByteArrayInputStream. ContentSource is of type java InputStream

DEBUG 21/May/2014:11:28:58.778 [31663700] parsing (options value 2052) XML body from input stream of type java.io.ByteArrayInputStream. ContentSource is of type java InputStream

DEBUG 21/May/2014:11:28:59.071 [31663700] parsing (options value 2052) XML body from input stream of type java.io.ByteArrayInputStream. ContentSource is of type java InputStream

DEBUG 21/May/2014:11:28:59.077 [31663700] parsing (options value 2052) XML body from input stream of type java.io.ByteArrayInputStream. ContentSource is of type java InputStream

DEBUG 21/May/2014:11:28:59.988 [b9da8700] Request: {jane, GET, MyServices/restapi//resources/empSalaries}

Result: true

DEBUG 21/May/2014:11:28:59.988 [b9da8700] result from OES: true

DEBUG 21/May/2014:11:28:59.989 [b9da8700] } = 1, filter [Authorize Access (OES 11g Authorization)]

DEBUG 21/May/2014:11:28:59.989 [b9da8700] Filter [Authorize Access (OES 11g Authorization)] completes in 4620 milliseconds.

DEBUG 21/May/2014:11:28:59.989 [b9da8700] ..."Authorize Access" complete.

DEBUG 21/May/2014:11:28:59.989 [b9da8700] } = 1, filter [Call 'Authorize Access']

DEBUG 21/May/2014:11:28:59.989 [b9da8700] Filter [Call 'Authorize Access'] completes in 2620 milliseconds.

See how long it took. This is because the SM had to be registered into OES and policies distributed. Look at what happens in OES:

[2014-05-21T11:28:57.664-07:00] [AdminServer] [TRACE] [] [oracle.jps.policymgmt] [tid: [ACTIVE].ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: <anonymous>] [ecid: 6af0f0aa917fd32e:-4e07427f:1461efea049:-8000-00000000000004ac,0] [APP: oracle.oes.admin.pd.ssl#11.1.1.3.0] [SRC_CLASS: oracle.security.jps.internal.policystore.entitymanager.impl.PDPRegistrationManagerImpl] [SRC_METHOD: registerPDP] registerPDP: PDPInfoEntry: {address=https://slc05ylp.us.oracle.com:16933/pd/PDClient, configurationID=oag_sm, instanceName=oagB_sm_slc03rfc.us.oracle.com__scratch_fmwapps_oes11gps2_client_oes_sm_instances_oagB_sm_config_jps-config_xml, isFusionApp=false, isTransactionalMode=false, appVersions={}}

[2014-05-21T11:28:57.665-07:00] [AdminServer] [TRACE] [] [oracle.jps.policymgmt] [tid: [ACTIVE].ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: <anonymous>] [ecid: 6af0f0aa917fd32e:-4e07427f:1461efea049:-8000-00000000000004ac,0] [APP: oracle.oes.admin.pd.ssl#11.1.1.3.0] [SRC_CLASS: oracle.security.jps.internal.policystore.rdbms.DBStoreManager] [SRC_METHOD: getDataManagerInternal] JpsDataManager ThreadLocal: current='null', new='oracle.security.jps.internal.policystore.rdbms.JpsDBDataManager@49fb3c8'

[2014-05-21T11:28:57.666-07:00] [AdminServer] [TRACE] [] [oracle.jps.policymgmt] [tid: [ACTIVE].ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: <anonymous>] [ecid: 6af0f0aa917fd32e:-4e07427f:1461efea049:-8000-00000000000004ac,0] [APP: oracle.oes.admin.pd.ssl#11.1.1.3.0] [SRC_CLASS: oracle.security.jps.internal.policystore.entitymanager.impl.PDPRegistrationManagerImpl] [SRC_METHOD: registerPDP] PDP registration: create a new PDPInfo.

[2014-05-21T11:28:57.666-07:00] [AdminServer] [TRACE] [] [oracle.jps.policymgmt] [tid: [ACTIVE].ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: <anonymous>] [ecid: 6af0f0aa917fd32e:-4e07427f:1461efea049:-8000-00000000000004ac,0] [APP: oracle.oes.admin.pd.ssl#11.1.1.3.0] [SRC_CLASS: oracle.security.jps.internal.policystore.ldap.JpsLdapAttributeMapper] [SRC_METHOD: prepEntryForPersist] entryType PDP_INFO, contextID cn=oes_domain,cn=JPSContext,cn=jpsroot, Updated Attributes: {orcloespdpstarttime=orclOESPDPStartTime: Wed May 21 11:28:57.666 PDT 2014, orcloespdpaddress=orclOESPDPAddress: https://slc05ylp.us.oracle.com:16933/pd/PDClient, orcloespdpheartbeattime=orclOESPDPHeartBeatTime: Wed May 21 11:28:57.666 PDT 2014, orcloespdpconfigurationid=orclOESPDPConfigurationID: oagB_sm, objectclass=objectclass: top, orclOESPDPInfo, orcloespdpstatus=orclOESPDPStatus: registered, orcloespdpinstancename=orclOESPDPInstanceName: oagB_sm_slc03rfc.us.oracle.com__scratch_fmwapps_oes11gps2_client_oes_sm_instances_oagB_sm_config_jps-config_xml, cn=cn: https://slc05ylp.us.oracle.com:16933/pd/PDClient}

Note: Once policies are distributed, a folder named "work" appears under $OES_CLIENT_HOME/oes_sm_instances/$SM_NAME/config. That's where the SM keeps local policies. That folder is updated every time a successful distribution occurs.

 "OES 11g Authorization" filter will finish way much faster in subsequent executions, like:

DEBUG 21/May/2014:11:47:56.791 [ba3ae700] run filter [Call 'Authorize Access'] {

DEBUG 21/May/2014:11:47:56.791 [ba3ae700] run circuit "Authorize Access"...

DEBUG 21/May/2014:11:47:56.791 [ba3ae700] run filter [Authorize Access (OES 11g Authorization)] {

DEBUG 21/May/2014:11:47:56.791 [ba3ae700] creating subject from 'jane'

DEBUG 21/May/2014:11:47:56.791 [ba3ae700] checking 'GET' to resource: MyServices/restapi//resources/empSalaries

DEBUG 21/May/2014:11:47:56.792 [ba3ae700] env attribute name: 'INVOKING_APPLICATION' env attribute value: 'OAG'

DEBUG 21/May/2014:11:47:56.811 [ba3ae700] Request: {jane, GET, MyServices/restapi//resources/empSalaries}

Result: true

DEBUG 21/May/2014:11:47:56.811 [ba3ae700] result from OES: true

DEBUG 21/May/2014:11:47:56.811 [ba3ae700] } = 1, filter [Authorize Access (OES 11g Authorization)]

DEBUG 21/May/2014:11:47:56.811 [ba3ae700] Filter [Authorize Access (OES 11g Authorization)] completes in 20 milliseconds.

DEBUG 21/May/2014:11:47:56.811 [ba3ae700] ..."Authorize Access" complete.

DEBUG 21/May/2014:11:47:56.811 [ba3ae700] } = 1, filter [Call 'Authorize Access']

DEBUG 21/May/2014:11:47:56.811 [ba3ae700] Filter [Call 'Authorize Access'] completes in 11 milliseconds.

Debugging OES SM in OAG

If you ever need to debug OES authorization decisions in OAG, refer back to jvm.xml.  There you find the entry

<VMArg name="-Djava.util.logging.config.file=$JRE_HOME/lib/logging.properties"/>

Specify the following properties in logging.properties. You're basically configuring a FileHandler to log FINEST level messages about OES authorization decisions to a file named java?.log in the user's home directory.

handlers=java.util.logging.FileHandler

.level=INFO

# default file output is in user's home directory.

java.util.logging.FileHandler.pattern = %h/java%u.log

java.util.logging.FileHandler.limit = 50000

java.util.logging.FileHandler.count = 1

java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter

# OES logging

oracle.jps.authorization.level=FINEST

oracle.jps.openaz.level=FINEST

As a result, a typical debug output message is like:

FINE: ========== Start Of Policy Evaluation Info ==========

Application: MyServices

Requested Resource Type: restapi

Requested Resource: resources/empSalaries

Requested Resource Present: false

Requested Action: GET

Request Subject Principals:

class weblogic.security.principal.WLSUserImpl:jane

Effective Roles Granted: [authenticated-role, Managers]

Role-Mapping Policies: NONE

Static Role Grants: NONE

Denied Static Role Grants: NONE

Authorization Policies:

1.Policy Name: permit_EmpSalaries

Matched Policy Principals:

class oracle.security.jps.service.policystore.ApplicationRole:Managers

Policy Principals Semantics: OR

Matched Policy Resource-Actions:

Resource = /resources/empSalaries, Action = GET

Policy Obligations: NONE

Policy Evaluation Result: GRANT

Policy Rules:

Rule Name: GetCarModelsPOLICY_RULE

Rule Effect: GRANT

Rule Condition: STRING_IS(INVOKING_APPLICATION,OAG)

Evaluated Rule Attributes and Functions:

INVOKING_APPLICATION(Dynamic, String) = OAG

Rule Evaluation Result: GRANT

========== End Of Policy Evaluation Info ==========

Wrapping Up…

As we can see, OAG and OES are a powerful combination when protecting web APIs. In this post we showed what to expect from this integration, how to configure policies on both sides, how to install OES SM in OAG and how to debug authorization decisions. Hopefully this is useful for some of you loyal readers out there. 

See you next time!