Service Throttling

We had to throttle the end service, and following are the two approaches which came quite handy:  

1. Out of the Box OSB
For the Business Service in OSB, we can configure the throttling as below. It allows us to configure concurrent threads, thread queue size, message expiration

  
2. HTTP Proxy Service
There could be a scenario where we don't have OSB or we don't want to use OSB to avoid piling up the request affect other important integration. The other standard approach to achieve similar thing via HTTP Proxy servlet and throttle down the number of threads to desired value and queue rest of the requests. This is involves some coding but it can be deployed on any weblogic managed server.

Create a Proxy Servlet
Instead of writing code from scratch, used com.jsos.httpproxy.HttpProxyServlet as below:

<?xml version = '1.0' encoding = 'windows-1252'?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
         version="2.5" xmlns="http://java.sun.com/xml/ns/javaee">
  <servlet>
    <servlet-name>TestServlet</servlet-name>
    <servlet-class>com.spring.service.proxy.TestServlet</servlet-class>
  </servlet>
  
  <servlet>
    <servlet-name>ProxyServlet</servlet-name>
    <servlet-class>com.jsos.httpproxy.HttpProxyServlet</servlet-class>
    <init-param>
      <param-name>host</param-name>
      <param-value>http://localhost:7001/NotificationService/NotificationPort</param-value>
    </init-param>
  </servlet>
  
  <servlet-mapping>
    <servlet-name>TestServlet</servlet-name>
    <url-pattern>/testservlet</url-pattern>
  </servlet-mapping>
  
  <servlet-mapping>
    <servlet-name>ProxyServlet</servlet-name>
    <url-pattern>/proxyservlet</url-pattern>
  </servlet-mapping>
  
</web-app>

Configure thread throttling
We can do that using following two different ways:

a) Local Work Manager in weblogic.xml
Configure Work Manager and Servlet dispatch policy in weblogic.xml

weblogic.xml

<?xml version = '1.0' encoding = 'windows-1252'?>
<weblogic-web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.bea.com/ns/weblogic/weblogic-web-app http://www.bea.com/ns/weblogic/weblogic-web-app/1.0/weblogic-web-app.xsd" xmlns="http://www.bea.com/ns/weblogic/weblogic-web-app">
  
  <wl-dispatch-policy>LocalProxyWorkManager</wl-dispatch-policy>                  
  
  <servlet-descriptor>
    <servlet-name>ProxyServlet</servlet-name>
    <dispatch-policy>LocalProxyWorkManager</dispatch-policy>
  </servlet-descriptor>
  
  <work-manager>
    <name>LocalProxyWorkManager</name>
    <max-threads-constraint>
      <name>LocalProxyWorkManagerMaximumThreadConstraint</name>
      <count>1</count>
    </max-threads-constraint>
    <capacity>
      <name>LocalProxyWorkManagerCapacityConstraint</name>
      <count>1000</count>
    </capacity>
    <ignore-stuck-threads>true</ignore-stuck-threads>
  </work-manager>
  
</weblogic-web-app>

b) Global Work Manager in weblogic.xml and Weblogic Console 
Configure Work Manager in Weblogic Console and Servlet dispatch policy in weblogic.xml

weblogic.xml

<?xml version = '1.0' encoding = 'windows-1252'?>
<weblogic-web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.bea.com/ns/weblogic/weblogic-web-app http://www.bea.com/ns/weblogic/weblogic-web-app/1.0/weblogic-web-app.xsd" xmlns="http://www.bea.com/ns/weblogic/weblogic-web-app">
  
  <wl-dispatch-policy>GlobalWorkManager</wl-dispatch-policy>                  
  
  <servlet-descriptor>
    <servlet-name>ProxyServlet</servlet-name>
    <dispatch-policy>GlobalWorkManager</dispatch-policy>
  </servlet-descriptor>
  
</weblogic-web-app>

Weblogic Console:

Source code can be downloaded from here.

AIA 11g PIP security policy with SOAP UI

With AIA PIP installation, you see basically three policy installed out of the box and there are lot of global policy set configured using these policies and attached to Provider, Requester or Adapter services.

Server side policies

1. oracle/aia_wss_saml_or_username_or_http_token_service_policy_OPT_ON 

If service is configured with this policy, then client needs to provide one of three security measures:

• SAML
• WSSE Username Token
• HTTP basic authentication

2. oracle/aia_wss_saml_or_username_token_service_policy_OPT_ON

If service is configured with this policy, then client needs to provide one of the two security measures:

• SAML
• WSSE Username Token

Client Side Policies

oracle/aia_wss10_saml_token_client_policy_OPT_ON

This is client side policy and it can be configured for any web service or composite which is protected via AIA server side policies.



Testing Service Side Policies using SOAP UI (or any other WS testing client)

1. oracle/aia_wss_saml_or_username_or_http_token_service_policy_OPT_ON 

• WSSE Username Token

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" 
               xmlns:sam="http://xmlns.oracle.com/SAMLProject/SAMLProcess2/SAMLBPELProcess2">
   <soapenv:Header>
      <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
         <wsse:UsernameToken>
            <wsse:Username>weblogic</wsse:Username>
            <wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">*******</wsse:Password>
         </wsse:UsernameToken>
      </wsse:Security>
   </soapenv:Header>
   <soapenv:Body>
      <sam:process>
         <sam:input>asdf</sam:input>
      </sam:process>
   </soapenv:Body>
</soapenv:Envelope>

• HTTP basic authentication

2. oracle/aia_wss_saml_or_username_token_service_policy_OPT_ON

• WSSE Username Token

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" 
               xmlns:sam="http://xmlns.oracle.com/SAMLProject/SAMLProcess2/SAMLBPELProcess2">
   <soapenv:Header>
      <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
         <wsse:UsernameToken>
            <wsse:Username>weblogic</wsse:Username>
            <wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">*******</wsse:Password>
         </wsse:UsernameToken>
      </wsse:Security>
   </soapenv:Header>
   <soapenv:Body>
      <sam:process>
         <sam:input>asdf</sam:input>
      </sam:process>
   </soapenv:Body>
</soapenv:Envelope>

Weblogic Custom Authentication Provider #2

I wrote in previous blog entry about how to configure custom authentication provider with Weblogic server. However, there are quite a few concerns associated with this approach, so I had to write generic custom authentication provider, and then I can plugin any module I like.

Some of the concerns with out of the box approches

• Need to implement both WLS and JPS identity 
•  WLS security is used for all basic Weblogic Modules (e.g. Console, EM, etc.)
• JPS security provider is used for SOA modules (e.g. Worklist application)
• WLS custom security provider is relatively easy to write - details
• JPS custom security provider is really lot of work as it requires you to implement multiple interfaces (similar to 10g custom identity service) and multiple methods details
• Need to register different security provider at different places

Implemented Solution

I believe at the end if would be much easier to if all the complex details can be hidden regarding WebLogic and SOA security provider and if client has to just implement a simple interface and provide that in the class path that would be ideal way to go. So here it goes:

Install Java Custom Security Provider

1. download CustomSecurityProvider.jar
2. Copy this jar file to $wls_server_home/server/lib/mbeantypes and $domain_home/lib directories
3. Modify file : $domain_home/config/fmwconfig/jps-config.xml
• Add Following  

<serviceProvider type="IDENTITY_STORE" name="custom.provider" class="oracle.security.jps.internal.idstore.generic.GenericIdentityStoreProvider">
    <description>Custom IdStore Provider</description>
</serviceProvider>

<serviceInstance name="idstore.custom" provider="custom.provider"  location="./">
    <description>Custom Identity Store Service Instance</description>
    <property name="idstore.type" value="CUSTOM"/>
    <property name="ADF_IM_FACTORY_CLASS" value="com.spring.security.jps.identity.CustomIdentityStoreFactory"/>
    <property name="CustomSecurityProviderPlugIn" value="com.spring.security.plugin.CustomSecurityProviderPlugin"/>
</serviceInstance>

• Replace Following

<jpsContext name="default">
    <serviceInstanceRef ref="credstore"/>
    <serviceInstanceRef ref="keystore"/> 
    <serviceInstanceRef ref="policystore.xml"/>
    <serviceInstanceRef ref="audit"/>
    <!--
        <serviceInstanceRef ref="idstore.ldap"/>
        <serviceInstanceRef ref="trust"/>
        <serviceInstanceRef ref="pdp.service"/>
        <serviceInstanceRef ref="attribute"/>
    -->
    <serviceInstanceRef ref="idstore.custom"/>
</jpsContext>

4. Restart Admin and Managed servers

Configure Java Custom Security Provider

• Implement the custom java security provider interface: com.spring.security.plugin.ICustomSecurityProviderPlugIn 
• Note that for given custom repository we only need to implement following methods

package com.spring.security.plugin;

import java.util.List;
import java.util.Map;
import java.util.Properties;

public interface ICustomSecurityProviderPlugIn {
    
     /* WLS */ 
    void initialize(Properties properties);
    boolean login(String userName, java.lang.String password);
    List<String> getUserRoles(java.lang.String userName);

     /* JPS */ 
    List<Map> searchUsers(String userNamePattern);
    List<Map> searchRoles(String roleNamePattern);
    Map getUserDetail(String userName);
    Map getRoleDetail(String roleName);
}

• If you opt to implement WLS, you can ignore to implement JPS related methods
• A sample implementation is provided with jar file (com.spring.security.plugin.CustomSecurityProviderPlugIn)

Make your implemented java or jar class available to weblogic classpath ($domain_home/lib)

Custom Security Provider should be available in drop down as below

Modify file : $domain_home/config/fmwconfig/jps-config.xml with your implementation

<serviceInstance name="idstore.custom" provider="custom.provider"  location="./">
    <description>Custom Identity Store Service Instance</description>
    <property name="idstore.type" value="CUSTOM"/>
    <property name="ADF_IM_FACTORY_CLASS" value="com.spring.security.jps.identity.CustomIdentityStoreFactory"/>
    <property name="CustomSecurityProviderPlugIn" value="com.spring.security.plugin.CustomSecurityProviderPlugin"/>
</serviceInstance>

Restart the server

Weblogic Custom Authentication #1

We can configure multiple WLS authentication provider (e.g. ActiveDirectory, Sun LDAP) as shown below.

If authentication and authorization information is stored custom repository not supported by above list, we can use following option.

CustomDBMSAuthenticator : Once it is configured as below, you can plugin in any Java class as long as it implements weblogic.security.providers.authentication.CustomDBMSAuthenticatorPlugin interface.

We can see the method "lookupPassword" which is called during authentication.

package weblogic.security.providers.authentication;
public interface CustomDBMSAuthenticatorPlugin {
    void initialize(weblogic.management.security.ProviderMBean providerMBean) { }
    void shutdown() { }
    java.lang.String lookupPassword(java.sql.Connection connection, java.lang.String userName) { }
    boolean userExists(java.sql.Connection connection, java.lang.String userName) { }
    java.lang.String[] lookupUserGroups(java.sql.Connection connection, java.lang.String userName) { }
}

We can completely ignore connection information and write custom java code to reach out to any custom repository and return the password. In that way it can be used for any custom repository instead of just custom database repository.

The major concern with this interface is that it requires you to return the password in lookupPassword method. Majority of the time enterprise level identity repository is not going to give you the password. Enterprise custom repository usually have their own authenticate method but above interface doesn't provide the password.

Another concern is that it only support WLS authentication and authorization. It doesn't provide JPS authentication and authorization.

• WLS authentication is used for all basic WLS modules (e.g. Console, EM, etc.)
• JPS authentication is used for SOA specific component, especially Worklist Application. 

If we implement custom authentication provider using above approach, it only covers WLS authentication and authorization, it would not be called during SOA module login (e.g. Worklist App).

AIA 11g Infinite loop - Running instances

We saw very strange behavior in Composite generated using AIA constructor where instance were having very similar behavior as they were in Infinite loop and using lot of server resources. This problem can be very well replicated in non AIA environment but with AIA it becomes highly visible due to AIA service constructor.

Scenario 1) Polling BPEL process



With AIA service constructor, following composite XML structure is already created



If process is polling process and it has following transaction properties:

    <property name="bpel.config.transaction">required</property>
    <property name="bpel.config.oneWayDeliveryPolicy">sync</property>

It puts process in transaction and if there is any error during processing, transaction will roll back. But with polling process transaction roll back will cause message to to go back in the source (file/jms, etc), and therefore a new instance of BPEL process will get created. This behavior will cause infinite loop scenario.

The simple resolution is to just remove these properties but in case we need transaction to work, we need to make sure that appropriate error handling is done.

Scenario 2)

Regardless of process is synchronous or asynchronous, AIA error handling fault policy is always integrated for AIA process. We had configured AIA fault policy to go human intervention after CompositeJavaAction was executed. Synchronous process needs to be completed transaction timeout but if there is any error, due to fault policy it will end up in Running state.

We saw that every time we restart the server, it somehow wakes up the running instance, and invokes AIA error handling again.

I guess for this particular scenario, it was the Fault Policy in Sync process causing the issue, but we have seen many other scenarios were instance can get stuck in Running state.

OSB Reporting with Co-releation

OSB reporting seems to have performance impact as mentioned by Ahmed. However, there are not not many options when it comes to report/tracing in OSB. Alerts and logs both are file based, a bit hard to search relevant info. We wanted to report start/exit and exception activity in OSB, and to check any way to co-relate all reporting actions which are related to same instance flow.

If we have co-relation id in all request messages payload, we can probably log that, but even body variable gets overwritten with fault or response, and it becomes very hard to co-relate different reporting entry which belongs to same instance.

Here the approach which worked out for us:
Basically we used OSB's messageId as correlation id for all reporting activity.

fn-bea:inlinedXML(fn:concat('<corelationid>',$messageID,'</corelationid>'))

Now we can use corelationid in any activity we want.

At the run time, we can see correlation, and from any event we can find corelationid and then search the entire trace with any corelationid.

Composite Fault vs Soap Fault

Nothing new, but just something for my own reference as it is bit confusing how composite faults gets mapped to actual soap fault. For the majority of the Runtime Faults, SOA 11g provides following structure:



With three main attributes, summary, detail and code. If you throw fault and test it from Soap UI or any WS client, you get following result. Between composites or through fault policy, you get full access to composite fault, but it looks quite different from WS client.



Below shows how SOA engine is converting Composite Fault to SOAP fault. Basically composite.summary goes to faultstring. detail goes to exception with sub elements. Fault code in composite is simply ignored.




Fault code is in soap fault is set as env:Server. Probably http://www.w3schools.com/soap/soap_fault.asp explains this.

To understand on how it works on custom fault, I created custom fault in XSD, and exposed in WSDL in synchronous process.  Upon testing from soap ui, we can see entire Fault gets mapped to child of Soap Fault [detail]. Fault code gets populated as name of the process. Faultstring and Faultfactor shows empty.



I guess it would be better to have full control over the entire fault message.