Chintan Shah's Blog

A cool hack

2012-02-13T17:24:00.000-08:00

Quite a long ago, I did post about decrypting jazn passwords, which came pretty handy for most of time including when I, myself forgot the passwords.

Here is some advance version, do email me if you figure it out:

Let's say, for sake of simplicity if you have 10g or 11g SOA installed on Unix box with unix oracle user.

a) If you happen to have only weblogic (or oc4jadmin) password to login to Web console. Is it possible to gain unix shell access (with oracle user) and also all database schema (and jms) passwords configured in app server?

b) If you happen to have unix shell access (with oracle user), can you gain access to weblogic or oc4j console and all db/jms password?

Answer to all questions is yes, the real question is how..

OSB Custom Transport Deployment Tips

2012-02-13T17:03:00.000-08:00

I was developing custom transport for OSB, development of custom transport was relatively easy but deployment and troubleshooting was a bit of nightmare. Just sharing some tips as it might be useful for someone facing same issues:

Deploy to both nodes (Adminserver and osb_server1)
It seems like UI stuff is used from AdminServer, and it run time it uses code from osb_server1. (of course, if you have developer version of OSB install, that would be separate story).

Redeploy of the transport always result in this error:

 ####<Feb 13, 2012 6:48:52 PM CST> <Error> <OSB Kernel> <springsoa> <osb_server1> <[STANDBY] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <> <dba6ff6b23159a46:64053457:135794b8417:-8000-0000000000000025> <1329180532849> <BEA-380003> <Exception on TransportManagerImpl.registerProvider, com.bea.wli.sb.transports.TransportException: Transport provider for ID **** already registered  
 com.bea.wli.sb.transports.TransportException: Transport provider for ID **** already registered  
      at com.bea.wli.sb.transports.TransportManagerImpl.registerProvider(TransportManagerImpl.java:157)  
      at com.bea.alsb.transports.sock.ApplicationListener$1.run(ApplicationListener.java:43)  
      at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)  
      at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:146)  
      at weblogic.security.Security.runAs(Security.java:61)  
      at com.bea.alsb.transports.sock.ApplicationListener.preStart(ApplicationListener.java:38)  
      at weblogic.application.internal.flow.BaseLifecycleFlow$PreStartAction.run(BaseLifecycleFlow.java:282)  
      at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)  
      at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)  
      at weblogic.application.internal.flow.BaseLifecycleFlow$LifecycleListenerAction.invoke(BaseLifecycleFlow.java:199)  
      at weblogic.application.internal.flow.BaseLifecycleFlow.preStart(BaseLifecycleFlow.java:62)  
      at weblogic.application.internal.flow.HeadLifecycleFlow.prepare(HeadLifecycleFlow.java:283)  
      at weblogic.application.internal.BaseDeployment$1.next(BaseDeployment.java:613)  
      at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52)  
      at weblogic.application.internal.BaseDeployment.prepare(BaseDeployment.java:184)  
      at weblogic.application.internal.EarDeployment.prepare(EarDeployment.java:58)  
      at weblogic.application.internal.DeploymentStateChecker.prepare(DeploymentStateChecker.java:154)  
      at weblogic.deploy.internal.targetserver.AppContainerInvoker.prepare(AppContainerInvoker.java:60)  
      at weblogic.deploy.internal.targetserver.operations.RedeployOperation.createAndPrepareContainer(RedeployOperation.java:104)  
      at weblogic.deploy.internal.targetserver.operations.RedeployOperation.doPrepare(RedeployOperation.java:128)  
      at weblogic.deploy.internal.targetserver.operations.AbstractOperation.prepare(AbstractOperation.java:217)  
      at weblogic.deploy.internal.targetserver.DeploymentManager.handleDeploymentPrepare(DeploymentManager.java:747)  
      at weblogic.deploy.internal.targetserver.DeploymentManager.prepareDeploymentList(DeploymentManager.java:1216)  
      at weblogic.deploy.internal.targetserver.DeploymentManager.handlePrepare(DeploymentManager.java:250)  
      at weblogic.deploy.internal.targetserver.DeploymentServiceDispatcher.prepare(DeploymentServiceDispatcher.java:159)  
      at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.doPrepareCallback(DeploymentReceiverCallbackDeliverer.java:171)  
      at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.access$000(DeploymentReceiverCallbackDeliverer.java:13)  
      at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer$1.run(DeploymentReceiverCallbackDeliverer.java:46)  
      at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)  
      at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)  
      at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)  
 >

Correct Re-deployment sequence

Undeploy from EM
Restart Admin and osb_server1
Note: This is to completely get rid of transport registration. It will get rid of : Transport provider for ID *** already registered
Redeploy to both Adminserver and osb_server1
Note: execution of business service at this time will result in "null" error.
Restart AdminServer and osb_server1

May be there is way to configure System.out seperately for custom transport but OOTB System.out didn't work well, so better to use weblogic logger. (e.g. I used : weblogic.logging.NonCatalogLogger )

Error: SCAC-50012 (Jdeveloper 11g Composite Compilation)

2012-02-06T13:51:00.000-08:00

While compiling SOA composite in JDeveloper, I saw this error many times:
Error: SCAC-50012

It happens mainly because of the Java code in the composite BPEL. Even after increasing the logging level of scac compilation, it didn't really show enough information to fix the problem. Here are some initial steps you can do to troubleshoot the issue (may be I will modify the scac task so we can show more information like 10g jdev used to show).

-> Check all imported libraries for composite project used by Java embed. If libraries are not found, jdev just shows SCAC-50012.

-> Make sure the java embed compiles fine by creating a sample Java project. If method signature changes or so, Jdev doesn't show any error, it just shows SCAC-50012.

OEG - how history repeats

2011-08-08T21:58:00.000-07:00

As we know, Oracle recently released product called OEG (Oracle Enterprise Gateway) which is meant to replace OWSM gatweay from 10g stack. I was just comparing how it with my 10g eyes.

1. So entire 11g stack is deployed on weblogic and installation time you choose user, which is pretty much weblogic and your preferred password. OEG is separate installation, it doesn't ask for username and password during install. (It is actually admin/changeme, which you can find buried in some documents).

Ah, remind me about the same in 10g, whole stack will use oc4jadmin, but OWSM would have separate user "admin" and oracle chosen password "oracle".

2. OEG password is changeme, but when I login to OEG using http://localhost:8090/runtime/management/ManagementAgent, can not find a way change it. It was not very hard and later figured out it is in the Policy studio, as below:

I don't have to tell, how hard is to change "admin/oracle" password in 10g, or almost impossible to add new user in OWSM.

3. BPEL compatibility :
I remember (~2005ish during first OWSM release), when you register BPEL service to OWSM it won't work. You have to copy schema manually. Well, I could deal with it for a while and later patches fixed those problems.
With latest release of OEG (11.1.1.5.0), Java webservice registration went fine, but when I try to register BPEL composite (deployed on SOA 11.1.1.5.0), it didn't register at all. Of course no logs in any of the log file. Later realized that it only like WS which are compatible with WS-I standard (http://www.ws-i.org/). Upon downloading their tools, I figured out that 11g composites (at least exposed BPEL WS) are not WS-I standard composite.
I probably have to make it work using policy studio somehow, but it would be nice if gateway can accept my BPEL wsdl URL.

Oracle CAMM with 10.1.3.5 SOA

2011-05-23T16:41:00.000-07:00

We recently upgraded to 10.1.3.5 and saw that CAMM was not able to discover any of the BPEL targets. We had to get one off patch to fix this issue:

Upgrading CAMM for 10.1.3.5 SOA:
- apply patch 9534485 (this is the latest 10.2.0.5.1 release of CAMM)
- apply patch 11939423 (change in one of the jar to read bpel descriptor)

Once it is done, just use 10.1.3.4 resource configuration to configure your 10.1.3.5 server.

- Restart SOA
- Restart CAMM

You can see BPEL process discoveries as below:

XRefUtility 10g - Round 2

2011-05-23T16:30:00.001-07:00

We were using XrefUtility and realized that Xref utility allows us to run select clause on source and target database, so that we can manipulate any data before comparing it with Xref table values. In our case, we had to manipulate the XRef values as well before comparing it with source and target database.

For example, it is pretty common to add revision number in BRM values in Xref table. Those revision number changes in BRM without getting updated in AIA database. To solve that problem, revision number should be truncated from AIA XREF and target query. I had to modify XRefUtility to achieve this functionality. I enhanced UI as below:

Another thing we found, is that going through 8 or so report is so cumbersome. If you look at my blog, you can see each report is combination of different cases. It would be nice to have all discrepancy in one or two reports, and have more control over queries. I ended up just writing my own utility in SQL as below:

create source view
drop materialized view sv;
create materialized view sv as
select a.row_id as sv_id, a.name as sv_name from SIEBEL.S_ASSET@sbltst a, SIEBEL.S_PROD_INT@sbltst b, SIEBEL.S_ORG_EXT@sbltst c, SIEBEL.S_ASSET@sbltst a2
where a.PROD_ID = b.ROW_ID
AND a.BILL_ACCNT_ID = c.row_id
AND a.root_asset_id = a2.row_id
AND a2.status_cd = 'Active'
create target view
drop materialized view tv;
create materialized view tv as
select * from (
select '0.0.0.1 '||PP.poid_type||' '||PP.poid_id0 AS tv_id, pp.descr as tv_name
from PIN.group_t@BRMMIG g, .....
create xref view (make it tabluer from row based view so that we can do joins and easier query).
drop materialized view xv;
create materialized view xv as
select row_number, XREF_TABLE_NAME as xref_table_name, max(decode(XREF_COLUMN_NAME, 'SEBL_01', VALUE,null)) sv_id ,max(decode(XREF_COLUMN_NAME, 'COMMON', VALUE,null)) common_id
,max(decode(XREF_COLUMN_NAME, 'BRM_01', rtrim(substr(value,0,length(VALUE)-2)),null))
tv_id from XREF_DATA
WHERE XREF_TABLE_NAME like 'INSTALLEDPRODUCT_ID' group by row_number, XREF_TABLE_NAME

Note we can create materialized view so that it doesn't affect the performance and refresh the view only when required. Now it is pretty simple to generate all reports and report numbers and data in one report:

count for sv_id null in xv - shows all descrepancy where source id is missing in AIA Xref Database
select count(*) from xv where sv_id is null
select * from xv where sv_id is null
count for tv_id null in xv - shows all descrepancy where target id is missing in AIA Xref Database
select count(*) from xv where tv_id is null
select * from xv where tv_id is null
count for rows in sv not in xv - shows all descrepancy where data is in source database but missing AIA xref
select count(*) from sv where not exists ( select * from xv where xv.sv_id = sv.sv_id )
select * from sv where not exists ( select * from xv where xv.sv_id = sv.sv_id )
count for rows in tv not in xv - shows all descrepancy where data is in target database but missing AIA xref
select count(*) from tv where not exists ( select * from xv where xv.tv_id = tv.tv_id )
select * from tv where not exists ( select * from xv where xv.tv_id = tv.tv_id )

OC4J JConsole 10g

2011-05-23T16:30:00.000-07:00

For JDK 1.5

Changes in opmn.xml
Add JVM parameter: -Dcom.sun.management.jmxremote=true
Change tmp directory to c:\temp
(For Windows) Change the owner user for service same as logged in user
Note make sure user name doesn't have (_) or (-) in it. bug: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6332311
Start the SOASuite in following sequence
set TMP=c:\temp
set TEMP=c:\temp
opmnctl shutdown
opmnctl startall
jps (this should show all the pid which are used by Oracle App server
Start jconsole and use pid for oc4j_soa

For JDK 1.6

Changes in opmn.xml:
Add following for oc4j_soa
-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=9999 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false

Change tmp directory to c:\temp
Start jconsole and use remote pid, with value localhost:9999
Please note that JPS will still show "process information unavailable", but jconsole would work.

XRefUtility 10g

2011-05-03T11:14:00.000-07:00

Recently had chance to work on XrefUtility. It is just designed for COMM PIPS to check on xref data anomaly, but it can be used for any XREF (as long as common column is named as COMMON). It generates multiple reports to show the different anomaly in Xref database. So basically it provides two distinct and valuable features:
- Generates multiple reports for Xref anomaly.
- Provides an interface to fix them

Installation: You can download the XRefUtility from Oracle Support 9326510. The installation note is included but I faced multiple jar files issues during installation. Nothing major, after adding those missing jars or changing path installation goes smooth.

It basically installs FixXref Bpel process and web application called XrefUtility. The FixXref should be installed in default domain. I tried to install in different domain which worked fine during install, but during run time it expected to have it in default domain.

Usage: usage is well documented in the guide but for my own purpose, I created my own XREF table and added my own data to create all possible scenario where Xref can have issues.

I created source table, target table and Xref called Source_Target_Xref. I also created meta data Xref which is not required for the XrefUtility but definitely required if you planning to use FixXRef.

integration/esb/bin/xreftool.sh -shell
deleteTable SOURCE_TARGET_XREF
createTable SOURCE_TARGET_XREF
addColumns SOURCE_TARGET_XREF SOURCE
addColumns SOURCE_TARGET_XREF TARGET
addColumns SOURCE_TARGET_XREF COMMON
listColumns SOURCE_TARGET_XREF

User screen to configure Xref is quite simple as show below:

Configure AIA Database and Xref

Configure Source

Configure Target

It can generate multiple reports. As mentioned earlier, I created all possible combination for Xref anomaly, and as mentioned below, reports were able to catch pretty much all of them. Following diagram shows which report causes which type of anomaly.

Reports

Along with report it also generates text file which can be used for input to FixXref interface. I did found some issue with Fix Xref but mostly tried to avoid as it is better to fix the code or process which causes such anomaly. Probably, I will blog about FixXref in future.

AQ - queue is stopped

2011-03-25T14:42:00.000-07:00

We started getting this error message in the log file all of sudden, and not sure how the dequeue for the queue got disabled.

[Linked-exception]
java.sql.SQLException: ORA-25226: dequeue failed, queue JMSUSER.AIA_CUSTOMERJMSQUEUE is not enabled for dequeue
ORA-06512: at "SYS.DBMS_AQIN", line 571
ORA-06512: at line 1

        at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:138)
        at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:316)
        at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:282)
        at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:639)
        at oracle.jdbc.driver.T4CCallableStatement.doOall8(T4CCallableStatement.java:184)
        at oracle.jdbc.driver.T4CCallableStatement.execute_for_rows(T4CCallableStatement.java:873)
        at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1161)
        at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3001)
        at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3043)
        at oracle.jms.AQjmsConsumer.dequeue(AQjmsConsumer.java:1601)
        at oracle.jms.AQjmsConsumer.receiveFromAQ(AQjmsConsumer.java:916)
        at oracle.jms.AQjmsConsumer.receiveFromAQ(AQjmsConsumer.java:835)
        at oracle.jms.AQjmsConsumer.receive(AQjmsConsumer.java:776)
        at oracle.tip.adapter.jms.JMS.JMSMessageConsumer.consumeBlockingWithTimeout(JMSMessageConsumer.java:405)
        at oracle.tip.adapter.jms.inbound.JmsConsumer.run(JmsConsumer.java:330)
        at oracle.j2ee.connector.work.WorkWrapper.runTargetWork(WorkWrapper.java:242)
        at oracle.j2ee.connector.work.WorkWrapper.doWork(WorkWrapper.java:215)
        at oracle.j2ee.connector.work.WorkWrapper.run(WorkWrapper.java:190)
        at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:830)
        at java.lang.Thread.run(Thread.java:595)

It was quite easy to fix it. Following query shows the queues which has dequeue or enqueue disabled:

select * from all_queues where trim(enqueue_enabled) = 'NO'

Once idetified which queue has issue, we can run following query to enabled the dequeue and enqueue for particular queue:

execute DBMS_AQADM.START_QUEUE(queue_name=> 'JMSUSER.AIA_CUSTOMERJMSQUEUE');

ESB 10g version agnostic routing rules - [default version]

2011-03-23T22:42:00.000-07:00

In BPEL, it is quite easy that if we don't specify the version during partnerlink invocation, it will use the default one. But in ESB, especially when ESB is calling BPEL via native binding the version of the BPEL process is hard coded inside the routing rule. This was quite annoying because this pattern is heavily used in AIA and redeploying newer version of BPEL requires change in routing rule in ESB.

In 10.1.3.5, Oracle released a new feature which solves this problem. After 10.1.3.5 upgrade, if you deploy any BPEL process, it creates a separate version in BPELSystem called "default" which is visible in ESB Console as well as JDeveloper.

If we select default during routing rule, then the default version of the BPEL process would be called from ESB during native invocation. Therefore if we change the BPEL default version or redeploy BPEL process with higher version, it won't require change in ESB routing rules.

BPELConsole promiscuous mode

2011-03-23T21:51:00.000-07:00

Nothing really new, but been having my eyes on this promiscuous mode, but never had time to sit down and put it under the test. It is at the server level accessible via collax-config.xml or BPELAdmin console.

So in 10.1.3.3, we had simple feature called:
productionServer = true/false (false by default)
true: same version of BPEL process is not allowed to be deployed. You get following error message:

is being re-deployed to a Production Server with same revision number.Please modify the revision for the process.

In 10.1.3.5, productionServer is deprecated and it is replaced with serverMode property. As per documentation:

serverMode = production/developement/promiscuous  
Identifies the server mode. Currently supported server modes are:
    * production - re-deployment of process with same revision is not allowed.
    * development - re-deployment of process with same revision marks the existing instance as stale.
    * promiscuous - re-deployment of process with same revision will not stale instances, work items will be migrated. 
The default value is "development".

I thought promiscuous is something everybody would want, so I thought experimenting it with. First changed the serverMode to promiscuous, and restarted.

Test case 1 (Synchronous) :

I deployed synchronous process A with couple of activities. Fired some instances of process A. Now I completely changed all the activity names in process A, added and deleted some more activities and redeployed process A with same version. Fired some more instances of process A.

Results: It was pretty good and as expected. None of the instances went stale. Old instances were shown based on old code and new instances were shown based on new BPEL code.

Now, as synchronous processes held the ground firm, it was time to experiment asynchronous flow.

Test case 2 (asynchronous ) :

I deployed asynchronous process B with a few activities. I had process B calling another asynchronous process C and then do receive activity. This was to make sure B goes to dehydration and I can get some inflight instances. I ran some transactions so that process B has a few instances inflight mode.

Now, I drastically change the process B, added/deleted new activities (but did keep the call to C and receive from C). I deployed the new code of process B with same version. I created some new transaction and tried to complete some old transactions.

Results: Quite fantastic. Nothing went stale. Process B in-flight instances with older code got migrated to newer code process B after dehydration point, and completed successfully with newer version of code. This is seems to be very powerful but delicate feature and person doing deployment should be very cautious about using promiscuous mode with in flight workflows.

Test case 3 (asynchronous human workflow) :

Created a simple process D, with Human Task, and created some inflight workflows for process D. I made some minor changes in process D, and redeployed the new code with same version.

Result: All bpel instances were inflight, but all human tasks went stale, at least didn't show up in WLA. This definitely didn't pass the expected results.

Conclusion
It is definitely very strong feature and avoid some hassles with stale activities. But in general it would be great to have this feature at the process level instead of server level. That way, we can do promiscuous only for sync and not just version the async one. I could not find such feature in 11g, guess I need to look around if composite have similar feature.

ESB Connection Leak in ESBPool

2011-03-02T12:39:00.000-08:00

If I have to come up the list for top 5 things I hate most, Oracle ESB (only 10g) would definitely make it to that list. I have been working on it for almost 5 years and pretty much every single project, including project involving complex business processing in ESB to millions of transaction per hour just using ESB. Each time ESB never fails to disappoint me further. Every new release since 10.1.3.1 fixes hundreds of bugs and introduced another new hundreds. Just an instance "Unable to build the instance relationship", one should get nobel price for solving it as it has been unsolved mystery for 5 years and seems to be more complex than E=MC². I personally have heard statement - "ESB is unreliable product" from internal Oracle SOA Gurus, and I completely disagree with it; I think that calling ESB a "product" itself is fundamentally breaking the laws of software ethics.

Anyways, recently did 10.1.3.5.2 upgrade and we saw that ESB connection pool (ESBPool) was growing 100 connections per hour. After deprived sleep multiple nights, we found that it was another ESB bug. There were constant error messages like "Unclosed connection detected : 'oracle.oc4j.sql.spi.ConnectionFinalizer@" in log.xml. Upon creating multiple different test cases, I found that it was with a very specific pattern:

AQ -> ESB Consumer -> ESB Async Routing Rule -> Asyn BPEL

The Async Routing Rule was the culprit and such flows were found in OOTB AIA code. Once we fixed it from Async to Sync, connections never went up more than 2, and life was back to normal again.

Undeploying BPEL process

2011-02-23T10:48:00.000-08:00

Well, not any cutting edge stuff, but did cause us lot of problem for us during 10.1.3.5 upgrade. It seems like 10.1.3.5 and especially 10.1.3.5.2 has lot of XSD validations and it can cause BPEL process not to load if some of the validation fails.

It doesn't allow user to click on this process in BPELConsole. Appropriate action would be to undeploy or redeploy those processes but without clicking on that process you can not undeploy it.

1) One way to undeploy such process would be to clean up database. We tried following:
a) clean up the database tables
PROCESS
PROCESS_DEFAULT
PROCESS_DESCRIPTOR
PROCESS_LOG
SUITCASE_BIN
b) clean up the BPEL process under tmp directory (bpel/domains/domain-name/tmp)
c) clean up ESB in BPELSystem

This was quite cumbersome process.

2) The other way we found was to use following URL:

http://host:port/BPELConsole/<<domain>>/undeployProcess.jsp?processId=<<process-id>>&revisionTag=<<revision>>

It was much cleaner to undeploy this way.

handleTopLevelFault - SOA 10.1.3.5

2011-02-10T17:15:00.000-08:00

I think all of us know that transaction=participate in 10.1.3.3 was replaced by handleTopLevelFault=true in 10.1.3.5. I never realized the impact it had in our processes and how Oracle keep changing their mind on what this value is and how the server behaves.

First of all, in 10.1.3.3, all BPEL processes DON'T participate in transaction by default. Which means if you have, process A -> Process B -> Process C, and

[C] is disabled

try {
    insert table 1
    invoke C
} catch {
    insert table 2
    throw
}

In this case, upon the failure of invoke C, entry in table 1 or 2 will never roll back, even if data sources are configured for XA transaction. If you do put transaction=partipate in your process at configuration level, then this process and their operation would be in Global transaction and throw activity would roll back insert in table 1 and 2.

[Note if you don't have throw, it would act like there is no error and nothing would ever roll back in any scenario, which is quite expected.]

Now, in 10.1.3.4, they replaced transaction=participate with handleTopLevelFault=false. The default value in the code [com.collaxa.cube.engine.core.map.BPELProcessBlock] was true. That's why behavior would be exactly same as 10.1.3.3. Nothing would participate in transaction unless you explicitly specify it to be. You can not define this property at domain level, it is only supported at process level.

In 10.1.3.5, the default value in [com.collaxa.cube.engine.core.map.BPELProcessBlock] got changed to "false". It caused numerous problems to us. Based on AIA best practice, we always have our own try/catch block, we do some processing in catch block (e.g. update db, queue, etc) and then we rethrow the error so the caller can get details of the error and instance can show as faulted. We had the pattern mentioned earlier all over the places. In 10.1.3.3 we could see the data which were inserted from catch block, but in 10.1.3.5 everything just rolled back because it started participating in transaction by default.

I agree, not a very good coding practice but it worked in 10.1.3.3 and nobody realized it can cause transaction issues during upgrade. The ideal solution should be to be catch block in local transaction so during roll back it doesn't rollback your work in catch activity.

[C] is disabled
try {
    insert table A
    invoke C
} catch {
    insert table B [make sure DB adapter is using tx-level='local' in data source]
    throw
}

Anyways, we had more than 200+ processes and we couldn't afford changing the code for each one of them, so I came up with neat solution which worked out without any issues. I wrote BPELClient API to insert "handleTopLevelFault=true" in majority of the BPEL processes, and that's it. No restart and no code change and life was back to normal. The only flow would be that if somebody redeploys the code and overwrites descriptor, but we can just run the util again in that case.

Here is the how code looked like:

ProcessDescriptor processDescriptor = bpelProcessHandle.getDescriptor();
ConfigurationsDescriptor configurationsDescriptor = processDescriptor.getConfigurations();
configurationsDescriptor.setPropertyValue("handleTopLevelFault",appropriateValue);  // set or add
bpelProcessHandle.updateDescriptor(processDescriptor);

Another side effect of handleTopLevelFault

If your process has handleTopLevelFault = true, it will show the Faulted Instance in Flow view and Audit view in BPELConsole

if your process has handleTopLevelFault = false (or don't have this property in 10.1.3.5), it will not show the instance in flow view and will only show in Audit view. That was some new enhancement added in 10.1.3.5.

I fixed the JavaScript issue so it shows up the faulted instance in Flow view even if you have handleTopLevelFault=false. Details can be found at http://chintanblog.blogspot.com/2011/01/bpelconsole-faulted-instance-in-10135.html

BAM Filtering / Governance 11g

2011-02-03T12:59:00.000-08:00

In one of the project, we were trying to push lot of data to BAM (some of the trace data along with data required for reports). We needed a solid approach were we can control the amount of data going to BAM at run time, just like logging level (info/debug). I agree that for infrastructure and instance tracing, CAMM, Amberpoint or Grid Control would be a better solution, but not all clients have luxury to invest in those tools. Sometime EM and BAM can help with basic troubleshooting.

The only out of the box approach I found was to use EM to disable monitoring at composite level. Pretty much nothing if you are using BAM adapter (e.g. in Mediator). We can also use following at composite level, but it requires redeployment:

   <property name="disableProcessSensors" many="false">true</property>

Following was the pretty intuitive approach we found to control the data going to BAM and we can stop/start/control BAM publishing at runtime. In BAM Sensor (Sensor action) there is feature called Filter. From there we can call any XSL expression, so we wrote custom XSL function which reads some configuration from JVM. We can control the BAM sensor publish event by returning true/false from this function.

We used the similar thing in Mediator, but it was quite simple as it also provides filter expression.

This way we had complete centralized control over BAM publish.

BPELConsole Faulted Instance in 10.1.3.5

2011-01-18T11:17:00.000-08:00

Recently we did upgrade of 10.1.3.5 and found that if you have faulted instance in BPELConsole, it doesn't show up the flow view. You can still see the details from Audit tab but flow tab shows following error (Javascript):

Line: 57
Char: 48
Error: 'this.wi.state' is null or not an object
Code: 0
URI: http://localhost:7777/BPELConsole/default/lib/flowviewer.js?gvpn0m

- First of all, faulted instances are the one which you want to look into BPELConsole first
- Audit tab is very busy, I believe flow view is graphical and very compact

Upon following up with Oracle, we found that it is expected behavior! (http://download.oracle.com/docs/cd/E14101_01/doc.1013/e15342/bpelrn.htm#BABCGIBB). Since when, the javascript error on the webpage become expected behavior??

Anyways, it was time back to get into dirty details of the JSP and Javascripts but I finally was able to fix the issue by just changing flowviewer.js file. Here are the steps, which should be applicable to any version:

1) open $ORACLE_HOME/j2ee/oc4j_soa/applications/orabpel/console/lib/flowviewr.js
2) You can use online beatifier as this JS is very compact, I used :
http://jsbeautifier.org/
3) Search for open.faulted, you should see something like

                } else {
                    if (this.wi.state == "4" || this.wi.state == "open.faulted") {
                        this.checkforESB(e);
                        this.templateId = "activity.faulted";
                        return
                    } else {
                        if (this.wi.state == "5" || this.wi.state == "6" || this.wi.state == "7" || this.wi.state == "8" || this.wi.state == "11" || this.wi.state == "12" || this.wi.state.substring(0, 6) == "closed") {
                            this.checkforESB(e);
                            this.templateId = "activity.closed";
                            return
                        } else {
                            this.checkforESB(e);
                            this.templateId = "activity.pending";
                            return
                        }
                    }
                }

4) Make changes as below:


                } else {
                    if (    (  this.wi.state == null &&   this.type == "4") ||    this.wi.state == "4" || this.wi.state == "open.faulted") {
                        this.checkforESB(e);
                        this.templateId = "activity.faulted";
                        return
                    } else {
                        if (   this.wi.state == null ||   this.wi.state == "5" || this.wi.state == "6" || this.wi.state == "7" || this.wi.state == "8" || this.wi.state == "11" || this.wi.state == "12" || this.wi.state.substring(0, 6) == "closed") {
                            this.checkforESB(e);
                            this.templateId = "activity.closed";
                            return
                        } else {
                            this.checkforESB(e);
                            this.templateId = "activity.pending";
                            return
                        }
                    }
                }

5) On IE or Firefox, just do ctrl+F5 to refresh your local copy of JS, and you should be able to see faulted instance just like before.

If you are using 10.1.3.5 or 10.1.3.5.2, you can just download flowviewer.js from here.

OWSM gateway clean up

2011-01-18T08:06:00.000-08:00

In OWSM fresh install, usually you get following error message in gateway.log file:

2011-01-15 18:39:37,194 FINEST [Thread-77] configuration.PolicySetWatchdog - Failed to retrieve policy set from policy manager with url http://host:7777/policymanager/services/RegistrationService
com.cfluent.policymanager.sdk.base.exception.ServerException: java.lang.Exception: Invalid component ID - C0003001
        at com.cfluent.policymanager.sdk.client.soap.SoapComponentConfigurator.getPolicies(SoapComponentConfigurator.java:185)
        at com.cfluent.agent.configuration.PolicySetWatchdog.getPolicySetFromPolicyManager(PolicySetWatchdog.java:168)
        at com.cfluent.agent.configuration.PolicySetWatchdog.pollFromPolicyManager(PolicySetWatchdog.java:207)
        at com.cfluent.agent.configuration.PolicySetWatchdog.run(PolicySetWatchdog.java:91)
2011-01-15 18:39:49,484 FINEST [Thread-77] configuration.PolicySetWatchdog - Checking Policy Manager
2011-01-15 18:39:49,530 WARNING [Thread-77] configuration.PolicySetWatchdog - Failed to retrieve policy set from policy manager with url http://host:7777/policymanager/services/RegistrationService: com.cfluent.policymanager.sdk.base.exception.ServerException: java.lang.Exception: Invalid component ID - C0003001
2011-01-15 18:39:49,530 FINEST [Thread-77] configuration.PolicySetWatchdog - Failed to retrieve policy set from policy manager with url http://host:7777/policymanager/services/RegistrationService

This is known issue, because during fresh install there is no gateway and there is no component called C0003001. So if you create gateway component then it will get the id of C0003001 and this error should go away.

In our environment, we had C0003001 component as gateway, but we decided to deactivate it and tried to create new one, and this error shows up again. Now deleting any component or recreating gateway doesn't ensure C0003001 id. On gateway, we started getting error: Gateway is not ready to process requests.

To resolve the issue, we cleaned up the entire OWSM repository and reset the sequence as below:

delete from SERVICES;
delete from COMPONENT_POLICY_MAPPINGS;
delete from COMPONENTS;
drop SEQUENCE COMPONENT_ID_SEQ;
CREATE SEQUENCE "ORAWSM"."COMPONENT_ID_SEQ" MINVALUE 1 MAXVALUE 999999999999 INCREMENT BY 1 START WITH 3001 NOCACHE NOORDER CYCLE

After that if we recreate gateway as first component, it will have id of C0003001 and issue will be resolved right away.

XSL Refresh 10g

2010-12-22T11:29:00.000-08:00

It is always good practice to have all XSD outside the BPEL and ESB project and refer them via http url (or oramds in 11g). All XSL remains in the BPEL or ESB, and changes to XSL requires redeployment.

We have lot of logic in XSL especially in AIA, and lot of XSL can be reused. We thought of exploring the option of doing real time changes to XSL.

Option 1: XSL embedded in BPEL process
If XSL is embedded in BPEL process, the XSL is cached inside the transformation engine. Fortunately, XSL function used by BPEl and ESB provides ways to initialize the cache. com.collaxa.cube.xml.xpath.functions.xml.GetElementFromXSLTFunction provide initializeCache function. So if you deploy a Java class in the same JVM as BPEL RT, then it can call this method and reinit the XSL cache. I did the same, created a Java class which calls GetElementFromXSLTFunction initializeCache and then exposed that piece of code as a WebService.

Here are detailed steps:

(one time) Deploy the the XSL Refresh web service (download) to same container as BPEL
Change the XSL from $ORACLE_HOME/bpel/domains/domain-name/tmp/.bpel.../.. as your need
go to: http://host:port/XSLTRefresh/RefreshXSLTPoolSoapHttpPort and hit invoke

This will ensure that new XSL changes will be in effect immediately. This approach came quite handy during development phase as I could avoid redeploying AIA processes for XSL changes and could have better turn around.Option 2: XSL on Apache Server I think cleaner way would be to Putting XSLs outside the BPEL project and deploy them separately from the BPEL and ESB deployment. E.g.

<from expression="ora:processXSLT('http://host:7777/xsl/Transform_IP_OP_Sample.xsl',bpws:getVariableData('inputVariable','payload'))"/>

BPEL XSL engine doesn't cache the XSL handle if it is http based, but you can use the xsl refresh webservice if you see any issue depending on BPEL version you are using.

IE8 with Oracle ESB 10g

2010-12-21T15:43:00.000-08:00

We were having issue with ESB console when we upgraded from IE6 to IE8 browser. BPELConsole/AIA/OWSM Console just worked fine with IE8. If you use 10.1.3.5+, I think this issue is fixed, but for lower version of Oracle ESB you can use following to make it compatible with IE8 browser.

For ESB Console:

- Modify following files:

esb-dt-container\applications\esb-dt\esb_console\esb\widgets\InstancesView.js and
esb-dt-container\applications\esb-dt\esb_console\picasso\widgets\widgets.Perspective.js

- In these files look for a line this.myEvents = [""] and comment this line out with "//"

How to check JDBC driver version

2010-12-15T11:19:00.000-08:00

Oracle Support Note: 396187.1 provides instructions on how to install a sample web application to test the database driver. It works great, and it is pretty simple process as shown below:

1) download drvtest.ear from the Oracle support
2) Deploy it desired container
3) Goto http://:/drvtest/dbdetails.jsp
enter username/password/url
it provide information like below:

JDBC Driver details
=============
JDBC Driver Name is ........ Oracle JDBC driver
JDBC Driver Version is ..... 10.1.0.5.0
JDBC Driver Major Version is 10
JDBC Driver Minor Version is 1
=============
Database Details
=============
Database Product Name is ... Oracle
Database Product Version is Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options

I found couple of issues with drvtest.ear file which forced me to write another sample app:
1. This application opens connection and doesn't close
2. This application is struts based and causes problem when you deploy oc4j_soa container (at least in 10.1.3.3.x). It works fine on other container
3. Some time you want to use data source instead of direct connection

I decompiled the code and modified for my need, and it was very easy to expose that piece of java code as web service and provide two separate operations:

InitialContext initialContext = new InitialContext();
DataSource dataSource = (DataSource) initialContext.lookup(dataSourceName);
connection = dataSource.getConnection();
DatabaseMetaData meta = connection.getMetaData();

1. You can deploy the CustomDriverTest.ear to any oc4j container
2. Goto http://host:port/CustomDriverTest/DriverCheckService?WSDL or http://host:port/CustomDriverTest/DriverCheckService
it provides both operation - test by direct database connection or test by data source.
3. It certainly closes connection after providing db information

JDBC Driver Upgrade for SOA/AIA 10g

2010-12-15T11:10:00.000-08:00

Oracle SOA suite comes with 10.1.0.5 version of JDBC driver, which is pretty old I believe. If you are using newer version of database, it is worth upgrading the JDBC driver of SOA Suite or AIA to the database version you are using.

If you are using 11g database, then 11g version of database driver does cause some problem with AIA console and EBS WF_Event complex payload, but as long as 10g version is concerned it is quite easy migration.

For instance, we were using 10.2.0.5 version of database and wanted to upgrade to 10.2.0.5 JDBC drivers:

1. Download 10.2.0.5 specific JDBC drivers from below
http://www.oracle.com/technetwork/database/features/jdbc/index-091264.html

2. Copy all jar files to $ORACLE_HOME/JDBC10205/lib directory

3. Add following to $ORACLE_HOME/j2ee/CONTAINER_NAME/config/server.xml:

    <shared-library name="oracle.jdbc" version="10.2.0.5">
        <code-source path="/aiaapp/oracle/product/10.1.3.1/OracleAS_1/JDBC10205/lib/classes12.jar"/>
        <code-source path="/aiaapp/oracle/product/10.1.3.1/OracleAS_1/JDBC10205/lib/classes12_g.jar"/>
        <code-source path="/aiaapp/oracle/product/10.1.3.1/OracleAS_1/JDBC10205/lib/classes12dms.jar"/>
        <code-source path="/aiaapp/oracle/product/10.1.3.1/OracleAS_1/JDBC10205/lib/classes12dms_g.jar"/>
        <code-source path="/aiaapp/oracle/product/10.1.3.1/OracleAS_1/JDBC10205/lib/ojdbc14.jar"/>
        <code-source path="/aiaapp/oracle/product/10.1.3.1/OracleAS_1/JDBC10205/lib/ojdbc14_g.jar"/>
        <code-source path="/aiaapp/oracle/product/10.1.3.1/OracleAS_1/JDBC10205/lib/ojdbc14dms.jar"/>
        <code-source path="/aiaapp/oracle/product/10.1.3.1/OracleAS_1/JDBC10205/lib/ojdbc14dms_g.jar"/>
        <code-source path="/aiaapp/oracle/product/10.1.3.1/OracleAS_1/j2ee/home/lib/ojms-provider.jar"/>
        <code-source path="/aiaapp/oracle/product/10.1.3.1/OracleAS_1/jdbc/lib/ocrs12.jar"/>
        <code-source path="/aiaapp/oracle/product/10.1.3.1/OracleAS_1/opmn/lib/ons.jar"/>
        <code-source path="/aiaapp/oracle/product/10.1.3.1/OracleAS_1/rdbms/jlib/aqapi.jar"/>
        <import-shared-library name="oracle.dms"/>
        <import-shared-library name="oracle.gdk"/>
    </shared-library>

4. Change $ORACLE_HOME/j2ee/CONTAINER_NAME/config/system-application.xml file:
(add : min-version="10.2.0.5)

    <import-shared-library name="oracle.jdbc" min-version="10.2.0.5"/>

5. bounce the server

6. check the driver version (http://chintanblog.blogspot.com/2010/12/how-to-check-jdbc-driver-version.html)

OPMN Ping timeouts

2010-12-02T13:17:00.000-08:00

I did this exercise quite a while ago and thought of posting as I encountered this issue again. Basically, opmn manages all containers and does health check. It can restart the container if container doesn't respond in timely fashion. In opmn log file, it shows up as : [libopmnoc4j] Forcefully Terminating Process. During heavy load or temporary network failure, this is not desired behavior so it is better to control and fine tune such parameters.

It is pretty scattered documentation on different parameters, so thought of putting them together. OPMN algorithm documentation can be found at :

By default opmn just comes with stop/start and restart timeout, which are used just during start/stop and restart as name suggest, but the run time algorithm uses different ping parameters:

Different ping option from OPMN
1. OS Process Check: Every 2 seconds OPMN queries the OS with the managed process id to see if it has terminated.
2. Forward Ping: Periodically, 20 seconds by default, OPMN sends a ping message to the managed process and expects a result within 20 seconds.
3. Reverse Ping: Every 20 seconds managed process sends OPMN a ping notification.

Forward ping can be fine tuned with ping attribute in opmn.xml
Reverse ping can be set using "reverseping" parameters in opmn.xml

BAM 11g JVM Filter/Sort

2010-11-24T19:54:00.000-08:00

I was working with BAM 11g and saw that I was not able to apply filter or sort when creating chart.

No error in log file makes even hard to troubleshoot. Finally, one of the metalink note (1077533.1) helped me out. Downgrading BAM JVM to jdk160_14_XXXX resolved the issue. You can either just change the JDK version of BAM or change it for whole domain by changing $DOMAIN_HOME/bin/setDomainEnv.sh.

BPEL Fault Generation - adapter framework

2010-09-29T11:13:00.000-07:00

Recently we had to dig into how BPEL was generating Runtime or Binding Fault. E.g. we were calling BRM adaper and based on BRM logs we could see, it was returning and error code. On BPEL console, we saw that error code was prefixed with BRM-. We had no idea what part was doing this conversion and that brought us to the point to analyze BPEL and Adapter code to figure out how it works with EIS system and how EIS fault gets converted to BPEL fault. It should be generic across all the adapter, as BPEL PM would be just calling the adapter and getting generic exception.

It was quite obvious that BRM- would be somewhere in the BRM adapter and that exception would be converted to Fault XML, but we wanted to find exact place where it would happen. Upon analyzing multiple files, we ended up with following findings.

Summary

oracle.tip.adapter.brm.BRMInteraction had hard coded BRM- when it was throwing PCResourceException
com.collaxa.cube.ws.WSIFInvocationHandler.WSIFInvocationHandler is getting this exception and converting this in to Fault XML message which is seen in the BPELConsole

Details

Each adapter entry point would be Interaction, and all adapter will implement that interface
com.collaxa.cube.ws.WSIFInvocationHandler.WSIFInvocationHandler.invoke seems to be the master piece which calls all the adapters via WSIF - especially you can see invokeRequestHeaderHandlers and invokeLocalService calls here
invokeLocalService converts adapter level exception (PCResourceException) to WSIFException
invoke constructs SOAP fault XML message based on WSIFException

AIA deployment error: Please set the ORACLE_HOME environment variable

2010-09-27T10:38:00.000-07:00

When you deploy AIA project from Jdeveloper (10.1.3.x release), you usually get an error: Please set the ORACLE_HOME environment variable. Easiest way to solve is to setup dummy Oracle and AIA home on your machine. You don't need to install entire SOA or AIA suite but need to setup the directory structure as below and copy the appropriate directory content from actual AIA installation. It comes to be about 40 to 50 MB.

Once it is setup, you can setup following environment variables:

AIA_HOME D:\Profiles\DummyHomes\DummyAIAHome
ORACLE_HOME D:\Profiles\DummyHomes\DummyOracleHome
BPEL_HOME D:\Profiles\DummyHomes\DummyOracleHome\bpel

Once Jdeveloper is restarted, this error should go away.

Note: it was quite obvious that something else will fail as soon as I hard code my ORACLE_HOME. It didn't take long me to find, my Oracle XE stopped working after my work station restart. I think it is good practice to not set ORACLE_HOME and set only when you need to do critical AIA deployment.

As another work around, you can only try to set AIA_HOME and BPEL_HOME, and based on build.xml file, it might just work fine, that way it doesn't cause problem with existing SOA or ORACLE DB installation on your machine.

How to install & get list of AIA patches

2010-09-27T10:34:00.000-07:00

To get list of all the patches installed in Oracle Application Server (10g release), you can run following command to get all patches:

To list patches installed in Oracle Home
set ORACLE_HOME=
cd $ORACLE_HOME/OPatch
opatch lsinventory

For AIA, there is separate OPATCH to install and to get list of patches. Here are the steps:

Download latest version of AIA Opatch from metalink (p5912518_111000_GENERIC.zip)
Set following homes

export ORACLE_HOME= (e.g. /home/oracle/product/10.1.3.1/OracleAS_1)
export AIA_HOME= (e.g. /home/oracle/aiahome )
export SOA_HOME= (e.g. /home/oracle/product/10.1.3.1/OracleAS_1)
export PATH= (e.g. /home/oracle/5912518/OPatch:$PATH)

Note: Make sure AIA Opatch is in the path and it is the first entry or prior to any other OPATCH entry.

Now you can run following command to get list of AIA patch or install AIA patch

List of AIA patch

opatch lsinventory -invPtrLoc $AIA_HOME/oraInst.loc -oh $AIA_HOME -jre $SOA_HOME/jdk/jre

Install AIA patch

Go to the directory where patch is unzipped and run:
opatch apply -invPtrLoc $AIA_HOME/oraInst.loc -oh $AIA_HOME -jre $SOA_HOME/jdk/jre

Removing WSSE Headers in OWSM

2010-09-23T15:30:00.000-07:00

Recently had chance to work with WSSE headers in OWSM. WSSE information is stored in SOAP Header. When we were getting WSSE security information in SOAP envelope (header), and we were using "Verify Signature" to verify signature. Verify Signature has step to remove the WSSE header so that this signature doesn't get propagated to BPEL, otherwise BPEL will throw exception. E.g.

If you install entire SOASuite in one container (oc4j_soa) and use above option to remove Security headers, it works great. When SOA suite is installed as per Enterprise Deployment Guide where gateway and policy manager are in separate container than oc4j_soa, OWSM doesn't remove the header even with this option.

Quick resolution to this issue is to use XSL and remove the header completely. Earlier we used following XSL to remove entire SOAP header.

We faced the problem with XSL as it removes entire SOAP header. We can use following XSL which can just remove WSSE header from entire SOAP envelope, so if there are custom element in SOAP header, it doesn't get removed.

Non intrusive set title for BPEL, AIA

2010-09-16T21:04:00.000-07:00

If you check my blog http://chintanblog.blogspot.com/2010/03/bpel-set-title.html you can see how to configure set title for BPEL process and why it is almost absolutely necessary for enterprise implementation. It also makes search in BPEL console quite useful http://chintanblog.blogspot.com/2010/03/bpel-title-wildcard-search_01.html.

We did encounter multiple issues which forced us to think non intrusive approach of setting title of BPEL process

We had to change title of more than 80 processes and writing logic and changing each process was very cumbersome (it had to go through un-necessary QA cycles)
Lot of process were already deployed and had lot of instances, we couldn't go back in time and set the title for them
Due to business requirement, if we plan to change the title, we have to go through this cycle again
We didn't want to touch OOTB process which comes with PIP, but we needed our custom title for those

Here is what I came up with:

Implementation Steps

EJB 3.0 timer which polls for new instances in BPEL DB
Via JDBC, extract Audit Information and Audit Trail (it would be configurable how much data to extract)
Once payload is available, apply XPath (based on configuration) to extract the title for a particular process
Update the title of the BPEL process

It sounded pretty complicated and resource exhaustive, but when I implemented and ran on the huge amount of data. I could see that we could set title of individual process in just 0.03 seconds, which was quite acceptable. It runs in separate JVM and doesn't talk with BPEL Runtime Engine so it doesn't increase any load on BPEL. It does increase some load on database but it is just select and update, there is no heavy processing going inside the database.

With this approach, we could achieve following

Centralized approach to configure and set the titles
We could change title of OOTB processes without touching the code
We could set the title of old instances (even stale instances)
Changing title was just matter of configuration and pure XPath. I also increased the size to title from 50 to 256, so you could put more data in title (e.g. order number, organization id, etc...)

Oracle AIA SOA - Multi JVM - Vertical Scaling

2010-09-12T18:55:00.000-07:00

It is pretty old concept and I tried earlier couple of times but this time I had much more time to do through analysis of Oracle Application Server with multi JVM container and see how it works out with SOA and AIA in general.

Configuration of multi JVM is fairly straight forward. All you need to do is change number of JVM for container from EM or change numprocs from opmn.xml for a particular container. You can get more information from http://mike-lehmann.blogspot.com/2006/09/scaling-oracleas-with-multiple-jvms.html

(Note: Although you can not change number of JVM for the container where EM is running, so if you install basic SOA suite, adding JVM would cause problem with EM.)

So we added 2 JVM to OC4J_SOA container and tried couple of test cases:

1) Test case: Make sure load is distributed evenly
I believe it is due to mod_oc4j config or default option, but load was distributed in exactly round robin fashion as per log file.

2) Functionality testing
Well, I was not involved much into functional stuff, but seems like functional stuff worked like a charm. We didn't see any issue with BPEL or ESB communication.

At this point we were pretty satisfied with results and about to move forward and then we thought of checking couple of configuration and deployment stuff for AIA and things started looking pretty disappointing:

3) AIA Configuration change
Test case: Changing AIA configuration file and using AIA console to reload the configuration. Expected output would be to have new configuration in both JVM.
Result: Only one JVM loads the AIA Configuration. I tried loading multiple times using UI, it still loads only on one JVM. May be http connection had some static stuff with OC4J. Removing caching or trying from different machine might fix this issue but I would still consider it as failure scenario.
Note: Restarting server certainly forces both JVM to load new AIA configuration file.

4) BPEL Configuration change
Test case: Changing BPEL configuration (e.g. audit level or logging level) from BPEL console. Expected output would be to have new configuration in both JVM.
Result: Only one JVM loads the BPEL configuration.
Note: Restarting server certainly forces both JVM to load new BPEL configuration

5) BPEL deployment
Test case: Redeploying BPEL process with same version with minor code change. Expected output would be to have new BPEL code in both JVM.
Result: Only one JVM picks up new BPEL deployed suitcase. Second JVM was still running with old BPEL code.
Note: Restarting server forces both JVM to load latest version of BPEL suitcase.

6) ESB Configuration and Deployment testing.
I was really hoping this test to work smooth due to ESB-DT and ESB-RT concept. So as long as ESB-DT is in separate container with only one JVM and ESB RT (may be in OC4J_SOA) can have multiple JVM and deployment and configuration change would still work fine. Unfortunately I didn't have access to correct environment where ESBDT and ESBRT were database persisted, and ESBDT in separate container.

With ESB-RT and ESB-DT in same container (oc4j_soa) and having them in-memory topics caused the similar effect as BPEL during configuration chagne and deployment.

In future, I might publish results on ESB with DB persisted topics and some interesting stuff I saw on Java based application (webapp and webservices).

CAVS Configuration

2010-09-10T13:16:00.001-07:00

I have been using CAVS for quite a while but didn't see lot of documentation on how to configure it so just thought blogging about it. I have also been using iTKO Lisa for a long time which is much richer in functionality than CAVS, and maybe I will write an article about it someday.

Brief Summary

CAVS (Composite Application Validation System) is mainly used for simulating your integration points for composite testing. It can basically mimic the end system and validate your request and send valid response upon successful validation.

Technical details

It is a servlet. URL (http://host:port/AIAValidationSystemServlet/syncresponsesimulator). You can configure multiple simulator instances and it get recognized by simid parameter.
It is part of AIAApplication ($CONTAINER_HOME/applications/AIAApplication/ValidationSystemServlet.war)
Execution logs can be found at ($CONTAINER_HOME/application-deployments/AIAApplication/oc4j_soa_default_group_1/application.log)

How to use:

Scenario:
Let's say we have simple Echo service, which returns upper case data back. We want to validate the request that, it has input and return the static response back.

Configuration steps:

Login to AIA console (http://host:port/AIA)
Click on Validation System (GO)
Click on Create Simulator
Enter the request and response as below (Make sure to include namespace for both request and response and don't remove CAVS namespace)
Click Next
Click on Request and Generate XPath (this will generate xpath for all possible combinations)
Make changes to validation as necessary. E.g. I made one nodekey (which is required) and just changed "equal to" to "is valid" operation for input.
Click on Save and record simulator id number
Now it is ready to tested with SOAP UI and end point would be http://host:port/AIAValidationSystemServlet/syncresponsesimulator?simid=<<>> or http://host:port/AIAValidationSystemServlet/syncresponsesimulator (generic). If you use generic one, it will compare against all simulator to get exact match

AIA Configuration

Out of the box always have support for real or CAVS endpoint for any partnerlink. If you are implementing custom code make sure you use dynamic parterlink for AIA so that it has CAVS support (more information at: http://chintanblog.blogspot.com/2010/03/aia-bpel-dynamic-partnerlink-lookup.html)
In AIAHome/config/AIAConfigurationProperties.xml file, make following changes:

AIA Error Codes

2010-08-03T17:00:00.000-07:00

I was working on AIA error handling and found very weird thing about how AIA was handling its error codes and description. Error codes are pretty much sporadic in .bpel or .xsl files, and aia:getErrorMessage is used to get error description for the error code.

AIA gets error messages from following file:
$oracle_home/bpel/system/classes/oracle/apps/aia/core/eh/i18n/EHResourceBundle.class

Error Code	Error Message
AIA_ERR_AIAINF_1001	Input log level for IsLoggingEnabled function is incorrect
AIA_ERR_AIAINF_1002	No corrective action string specified for the given corrective action code
AIA_ERR_AIAINF_1003	No error message has been specified for the input key
AIA_ERR_AIACOMBMPI_0001	Maxitems requested is less than the Threshold/Number of records returned from the billing system
AIA_ERR_AIACOMBMPI_0002	Account/Billing Profile being sent is not synced to the billing system
AIA_ERR_AIACOMCMPI_0001	EBMHeader/Sender/ID is required
AIA_ERR_AIACOMCMPI_0002	EBMHeader/Target/ID is required
AIA_ERR_AIACOMCMPI_0003	Account Sequence Error: Pay-From Accounts and Billing Profiles must appear before dependent/subordinate Accounts and Billing Profiles
AIA_ERR_AIACOMCMPI_0004	Subordinate account cannot have multiple parent accounts
AIA_ERR_AIACOMCMPI_0005	Ambiguous subordinate bill profile update: There are multiple distinct Pay-From-Party Billing Profile references associated with a single Prior Pay-From-Party Billing Profile reference
AIA_ERR_AIACOMCMPI_0006	All existing subordinate bill profiles are not included in the move account request
AIA_ERR_AIAINF_2001	Error message for getErrorMessage XPath verification
AIA_ERR_AIAO2C2_1001	Timeout while waiting for a response from the InterfaceCustomerToFulfillmentEBF service.
AIA_ERR_AIAO2C2_1002	Timeout while waiting for a response from the InterfaceSalesOrderToCustomerEBFV2 service.
AIA_ERR_AIAO2C2_1003	Timeout while waiting for a response from the CreateSalesOrder EBS service operation.
AIA_ERR_AIAO2C2_1004	Timeout while waiting for a response from the UpdateSalesOrder EBS service operation.
AIA_ERR_AIAO2C2_1005	This order has already been synced.
AIA_ERR_AIAO2C2_1006	Account could not be queried. Please make sure the Account exists in the system
AIA_ERR_AIAO2C2_1007	Timeout while waiting for a response from the SyncCustomerPartyList EBS service operation.
AIA_ERR_AIAO2C2_1008	Credit Check failed. Please contact your System Administrator.
AIA_ERR_AIAMDM_1001	Timeout while waiting for a response from the SyncCustomerPartyList EBS service operation.
AIA_ERR_AIAMDM_1002	Timeout while waiting for a response from the FetchCustomerPartyEBF service.
AIA_ERR_AIAMDM_1003	Invalid QueryCode. Valid values are: MATCH_PERSON or MATCH_ORGANIZATION.
AIA_ERR_AIAMDM_1004	Invalid QueryCode. Valid values are: FETCH_PERSON or FETCH_ORGANIZATION.
AIA_ERR_AIAMDM_1005	Cross-Reference ID lookup failed or returned no associated IDs
AIA_ERR_AIARTL2PSFT_1001	Invalid Requesting System; Requesting Systems must be RESA
AIA_ERR_AIARTL2PSFT_1002	Invalid Set Of Books Id; Set Of Books Id must in BUSINESS_UNIT DVM
AIA_ERR_AIARTL2PSFT_1003	Invalid Currency Conversion Type Code; Currency Conversion Type Code is not in CURRENCYEXCHANGE_CONVERSIONTYPECODE DVM
AIA_ERR_AIACOMOMPI_0001	Dates Validation Falied : Either one of the Purchase Date/Cycle Start Date/ Usage Start Date should be set to future
AIA_ERR_AIACOMOMPI_0002	Date Validation Falied : Purchase Date should be set to future
AIA_ERR_AIACOMOMPI_0003	Purchased promotion instance does not exist for a promotion that was previously purchased.\240\240 A data upgrade script was not run.
AIA_ERR_AIACOMOFMPI_0001	Data Insufficient for Trouble Ticket Creation. Order Originating System Code not available
AIA_ERR_AIACOMOFMPI_0002	Data Insufficient for Trouble Ticket Creation. Order ID not available
AIA_ERR_AIAAGILEJDEE1_0001	For additional information go to /agile/ais/fileResult and review the following file:
AIA_ERR_AIAAGILEJDEE1_0002	There are branch plants in the input message that do not have a mapped value in the AGILE_TARGET_SITE_MAPPING DVM
AIA_ERR_AIAAGILEJDEE1_0003	None of the items selected have been integrated
AIA_ERR_AIAAGILEJDEE1_0004	None of the engineering change orders selected have been integrated
AIA_ERR_AIAAGILEJDEE1_0005	Unable to find a Agile Site to branch plant mapping. Ensure all of the sites for the items on the Change are properly mapped or a default is set in AGILE_SITE_TARGET_MAPPING DVM
AIA_ERR_AIAAGILEJDEE1_0006	Invalid Batch Quantity: All Bill of Materials Batch Quantity must be equal to Zero.
AIA_ERR_AIAAGILEJDEE1_0007	Invalid Bill Of Materials Type: All Bill of Materials Type must be equal to 'DEFAULT_BOM_TYPE' AIA Configuration Property value.
AIA_ERR_AIAAGILEJDEE1_0008	Invalid Bill Of Materials Input: Multiple Bill of Materials for the same Parent Item is not allowed when 'MULTISITE_ENABLED' property is set to 'FALSE'.
AIA_ERR_AIAAGILEJDEE1_0009	Invalid Bill Of Materials Input: Bill Of Materials Components with 'Non-Stock' Stocking type is not allowed when 'MULTISITE_ENABLED' property is set to 'TRUE'.
AIA_ERR_AIAAGILEJDEE1_0010	Failed
AIA_ERR_AIAAGILEJDEE1_0011	Release ECO SDK Execution Failed:
AIA_ERR_AIAAGILEJDEE1_0012	Could not find Change Order workflow:
AIA_ERR_AIAAGILEJDEE1_0013	File does not exist:
AIA_ERR_AIAAGILEJDEE1_0014	None of the Items selected have associated Branch Plants
AIA_ERR_AIAAGILEJDEE1_0015	The Change Order Not found -
AIA_ERR_AIAAGILEJDEE1_0016	Invalid Item Revision Level: Make sure 'ITEM_REVISION_LEVEL' Property in AIA Configuration Properties is not Blank
AIA_ERR_AIAEBSI_0001	Failed to fetch the Sales Representative Resource ID. Please make sure the Sales Representative email addresses are identical in both systems

It was quite disappointing to see that all error codes and error messages are defined in a Java class instead of a property file. if someone needs to change error message or add new error code and error message, it is impossible unless you hack the Java code and recompile it.

It does makes sense to create a custom framework to store all your error codes and error messages so that you don’t get bound to the AIA boundaries.

Siebel Web Service with SoapUI

2010-07-07T22:02:00.001-07:00

I was working with Siebel webservices for Query Customer and found that it was working fine from BPEL but when I try from soapUI, it returns junk characters. Upon intercepting both traffic using tcpmon, we found the issue was in one of the HTTP header. Basically "Accept-Encoding: gzip,deflate" which was sent from SoapUI was causing the problem.

In soapUI, you can disable this particular header via following option. Once the option is enabled, you can invoke Siebel web service doesn't return junk data and returns valid XML data.

Java forever

2010-06-30T09:22:00.000-07:00

Not related with Oracle SOA. It was just fun to watch this after a long busy day .. Java forever

BPEL Version Redirect

2010-06-16T18:19:00.001-07:00

I was working with the client and we had situation were multiple BPEL process versions were deployed with one of them being the default version (e.g. 1.0, 2.0*, 3.0). If client is using is version agnostic WSDL and end point URL, (as below), then it is not major concern because BPEL will route it to default version and everything will work like a charm.

http://host:port/orabpel/<domain>/<process>/<process>?wsdl

http://host:port/orabpel/<domain>/<process>

That was not the case we had. One of the client was using version specific URL, and we didn’t have time and control over the client to change their URL right away. Of course, the long term solution would be use version agnostic URL, gateway or UDDI, but here is quick and dirty way if we just want to route old version request to new version using Apache redirect rule.

This basically routes the request from old-version to new-version for WSDL or End-Point request. If we want to make it generalize, to redirect all versions to a specific version (doesn't matter if that is default or not), it would be following. This can come pretty handy if version management goes out of hand..

After making it generalized, I thought of taking this approach to the next level. As all of us know that to utilize ESB->BPEL native binding (which is highly recommended by Oracle, especially in AIA world), we have to configure routing rules to point to specific version of BPEL process. Every time new version of BPEL is deployed it causes routing rule to change. I thought may be I can fool ESB with this redirect even if it is using native binding, but this trick but didn’t work out with native binding.

XSLT Troubleshooting

2010-05-03T13:23:00.001-07:00

May be it is just based on my experience, but I feel in general XSLT doesn’t have lot of troubleshooting support. E.g. in Java, no matter what JVM you use, you can pretty much get stack trace (even in NPE errors). In XSLT, based on XML parser you use, sometime you get line numbers and some time you don’t, plus it doesn't provide lot of details in log. Moreover there are not really any good tools to do much troubleshooting. In AIA world, it gets even crazier. Most of the XSLT functions come with AIA just runs only on server side, so you are hopeless and helpless when you want to do some troubleshooting on your local box. It gets even worse, when you get FOTY0001 type errors. Sometime you get more details in log file when there is FOTY0001, and sometime you don't.

Here are couples of ideas which can make your life easier.

1) XSL comment
You can put XSL comment in your code, so it will show up when XSLT is completed. It does help a lot when you are trying to just print values of variables. Pretty much all credit goes to Mahesh Narayanaswamy (absolute XSL genius) for this suggestion.

2) XSL out
This was very cool idea came out of disucssion with Amol Vaidya. XSL comment didn’t help much when XSLT was completely breaking and we were unable to get the line number or the root cause. All we needed is the similar capability like System.out inside the XSL, so we know how far in XSL we went and what are the variables. We just want to get all information before the point of failure. A custom XSLT log function came pretty handy for putting trace log in your XSLT. This way we can get as much information about the XSLT before the point of failure.
We wrote custom XSL function called XSLTLog, and registered that function in BPEL RT and ESB XSL Library.

3) Java code
I also wrote a Java code which can take XML input and execute the XSL. We have to register this Java code to the server as lot of AIA XSL functions relies on server side components. We also took it further to find exact line number in XSL using binary search approach.

BPEL ReadOnly Console

2010-04-28T14:28:00.001-07:00

It was my first blog which I wrote almost three years ago. http://chintanblog.blogspot.com/2007/12/i-saw-numerous-people-asking-about-bpel_290.html Since then, pretty much all my clients and a lot of other companies used this approach. I also saw many other blogs copied this idea or put enhancement to it. Anyways, one of my client pointed out this metalink article: 852840.1. Metalink had reference to my blog!! Wow, it certainly makes me feel good when multi billion dollar organization put reference to my tiny little site.

On a smaller scale, but it still reminds what Margaret said: "Never doubt that a small group of thoughtful, committed citizens can change the world. Indeed, it is the only thing that ever has"

TCP Tunnel - Java Port Forwarding - TCP kill

2010-04-08T22:39:00.000-07:00

You know, sometime you just get amazed when you find an absolutely simple solution for a overly complicated problem. I was working with one client remotely and banging my head against the wall to get the access to their valuable servers. After a so much frustration I could get access to one or two servers via ssh, and that's pretty much it. I could ping to everywhere from that UNIX box but can not connect anything from my local laptop.

To make long story short, I tried using SSH tunnel, hacked routing tables of UNIX/windows but nothing came as handy as a simple Socket program which basically listens on a port and make socket connection to remote client. It is very much like TcpMon or Obtunnel but it is just bare bone version of it. Pretty much 99% of my laptop traffic was going through this tiny unix box and never had to beg for access.

java -jar springportforward.jar com.springsoa.socket.Forward localport remotehost remoteport &

Now as this become the absolutely life savor, I thought I can I can leverage something more out of it. I think we potentially can use this for tcpkill workaround. Basically try to forward all traffic through portforward, and kill the portforward to achieve same effect as tcpkill.

BPEL Set Title

2010-03-31T12:38:00.001-07:00

BPEL titles are extremely important for troubleshooting anything from BPELConsole. It is basically providing title-name to BPEL instance. I also wrote blog on how to do wildcard search on those BPEL titles (http://chintanblog.blogspot.com/2010/03/bpel-title-wildcard-search_01.html).

Now the part comes is how to set titles. I am still working on implementing non-intrusive way of setting titles without ever changing BPEL code, but for now let's talk about how it is done Out of the Box:

Recommended Approach is:

I didn't like the idea of Java embed, just as it conflicts with other best practices Oracle has and it creates an additional workItem overhead for BPEL RT. Correct way of doing this would be to write BPEL function to do this. I wrote BPEL function as below:

ora:setProcessTitle()

This function sets the title of BPEL process
The signature of this function is : ora:setProcessTitle ('title', 'auditTrail'?)
auditTrail: true/false, it is optional and default value is true.

With this, you can set the title as native assign operation:

It doesn't reduce the code footprint that much, but still provides neater BPEL code and centralized way of handling Audit.

AIA BPEL Dynamic Partnerlink Lookup

2010-03-31T11:51:00.001-07:00

BPEL dynamic partnerlink lookup concept is quite old and been there since 10.1.2 and it works great. AIA took that concept little far and forced the design pattern. E.g. all endpoint URLs should be strictly in AIAConfigurationProperties.xml file.

Example would be:

Step 1: put the endpoint URL in AIAConfigurationProperties.xml file:

Step 2: Change the BPEL code to look up this URL at run time using Java Embed.

Step 3: Follow the same old rule for BPEL dynamic partner-link once we have the right endpoint URL.

So far so good, but the problem is 20 lines of "non reusable" java embed code in my BPEL. If you talk with any decent Oracle SOA guru, they will ask you to avoid Java Embed as much as possible. AIA design pattern didn't stop here, they have recommended not to put EndPoint (or Partner-link) name directly in your Java Embed. Reason seems to quite obvious, you just specify the partner-link name, and based on SystemID or CAVS parameter, we can route it to any place without doing any code change. At the end your Java Embed looks something like this:

AIAConfigurationProperties.xml:

Java Embed:

Functional point of view it looks great, but technically - holy crap! 93 lines of non reusable code copied for every partner-link and every BPEL process.

I took initiative to eliminate the code for and obviously the best option was to use Oracle BPEL XPath Function. Initial functionality didn't take long time to develope BPEL function and register them to BPEL RT engine, but making addAuditTrailEntry working in my custom function was an amazing experience. Rather than going too much into details, would just go with final product. I created two separate bpel functions:

1) ora:getSimpleDynamicURL (simpler version)

This function gets dynamic URL from AIAConfigurationProperties.xml.
The signature of this function is : ora:getSimpleDynamicURL ('serviceName', 'partnerlinkName', 'auditTrail'?)
auditTrail: true/false, it is optional and default value is true.

2) ora:getDynamicURL (richer version)

This function gets dynamic URL from AIAConfigurationProperties.xml.
The signature of this function is : ora:getDynamicURL ('serviceName', 'partnerlinkName', 'ebmMsgBpelVariableName', 'ebmMsgPartName', 'auditTrail'?)
auditTrail: true/false, it is optional and default value is true.

If I look at my BPEL process now, it looks like the way it supposed to. I can get TargetEndpointLocation from AIAConfigurationProperties.xml based on parameters, and hallelujah!

It just reminds me the most basic programming tip which my friend told me: "Don't do copy and paste, just do cut and paste.

Namespace agnostic XPath/XQuery

2010-03-04T23:11:00.001-08:00

Problem Description

I have two different type of XML files, both have multiple namespaces in it. First one contains prefixes along with namespaces, where second one doesn't. The question is how do I find title (Organization->Person->title) using XPath or XQuery?

Solution - 1 (Aware of Namespace)

The standard way (which I don't recommend) of doing this would be to create XPath resolver and register all the namespaces with it. I created external properties file with all the namespace like below (Note: Namespace prefix doesn't have to match between XPath and XML Input Payload (e.g. org to org1). Only namespaces has to match.):

Now, I started writing a method which does Xpath resolution.

For this method, I had to create a class for Namespace resolution. I wrote a class which resolves namespaces based on property file:

That's it, now if you provide /org1:Organization/person1:Person/person1:title as XPath to this program, it can evaluate it as "Engineer"! I was not too impressed with this approach because everytime payload comes up with change/addition in namespace, I have to maintain my property file. In small organization it might workout OK, but in rapidly changing world I can not imagine it would work at all.

Solution - 2 (Namespace Agnostic XPath/XQuery)

For this, I wrote method which doesn't require any Namespace Context. E.g.

Now the trick is in the XPath, I wrote XPath: /*[contains(name(),'Organization')]/*[contains(name(),'Person')]/*[contains(name(),'title')]. This XPath resolves to "Engineer" given XML payload and it doesn't require any namespace. Correct way of writing this XPath would be to [name()='Organization' or ends-with(name(),':Organization')], but unfortunately ends-with is not supported with Oracle XML parser, but I don't see any huge issue with current Xpath.

Test

Finally I just wrote test case to test this out, e.g.

Ran the test, and I could not be more happier after looking at the results:

FYI, just found two more approaches to write name space agnostic XPath:
a) /*[local-name()='Organization']/*[local-name()='Person']/*[local-name()='title']
b) //*[local-name()='title']

JAZN Security - Hacked

2010-03-03T11:30:00.001-08:00

I guess pretty much everybody in Oracle community knows that Oracle supports indirect password (for data-sources.xml) via file based security system (Jazn). All the container passwords e.g. oc4jadmin, bpeladmin are also stored in Jazn. File name is $ORACLE_HOME/j2ee/<<container_name>>/config/system-jazn-data.xml

There are multiple ways to implement indirect password, and you can go through http://download.oracle.com/docs/cd/B31017_01/web.1013/b28957/deploysimple.htm#BABCCIFH if you really using it first time.
- You can use EM to use indirect password
- You can change data-sources.xml and put ! infront of password.
- You can put "->USERNAME" and create that USERNAME in Jazn with indirect password using EM

Anyways, back to the main problem, passwords are stored in encrypted form in system-jazn-data.xml, so people who has access to the file system would not know clear text password. The encryption algorithm used was way too weak to break. I would avoid putting the solution on my blog just to avoid hacking encouragement.

I would assume the person who wrote encryption algorithm for Jazn was either too lazy or didn't want to implement it correctly, because I believe an average Java programmer can decrypt system-jazn-data.xml in less than 30 minutes.

ESB Interceptor

2010-03-02T19:07:00.001-08:00

I guess I am pretty late on this blog, I had this working 2 years ago, and implemented it for more than 2 clients over last two years and now probably does not value that much. I still thought to share it and it is a little bit documented in http://download.oracle.com/docs/cd/E12524_01/relnotes.1013/e12523/esb.htm as well.

Basically, implementing IEsbMessageInterceptor interface, which looks something like this (You probably have to tweak it little bit as IEsbMessageInterceptor has been changed from 10.1.3.4 to 10.1.3.5).

public class CustomESBInterceptor implements IEsbMessageInterceptor {

public CustomESBInterceptor() {

System.out.println("Initializing Custome Interceptor");

}

public void invoke(FlowId pFlowId, String pInstanceId, Element pPayload, Element pHeader, String pServiceName, String pSystemName, boolean isOutbound) {

System.out.println("----------------------------------------");

System.out.println("Custom Interceptor ");

System.out.println("FlowId: " + pFlowId + " subflowid: " + pFlowId.getSubFlowId() );

System.out.println("instanceid: " + pInstanceId);

System.out.println("Header: " + XMLHelper.toXML(pHeader));

System.out.println("payload: " + XMLHelper.toXML(pPayload));

System.out.println("Service: " + pServiceName);

System.out.println("System: " + pSystemName);

System.out.println("----------------------------------------");

}

Once we compile this class, we can put the class in bpel/system/classes directory and add following to $ORALCE_HOME/integration/esb/config/esb_config.ini file:

inboundMessageInterceptor = CustomESBInterceptor

outboundMessageInterceptor = CustomESBInterceptor

Just to clarify, just like any other interceptor this is extreamly dangerous point. We should keep following things in mind:

Call from ESB runtime to this interceptor is made in synchronous fashion, so if CustomESBInterceptor.invoke errors out, then entire ESB RT will have issues.
Another issue is if we are doing any heavy processing in CustomESBInterceptor.invoke, that will affect performance of entire ESB RT. The code in invoke should be absolutely error free and as quick as possible. We should use AQ or another JMS implementation to avoid real time processing if needed.
You can possibly change the payload and header values if you want to (it should be avoided for best practices)

BPELConsole Timeout Issue

2010-03-02T06:27:00.001-08:00

In 10.1.3.5 Oracle SOA Suite we realized that BPELConsole to load the flow page, often timeouts. It gets worse if you are accessing the BPELConsole remotely or from different country. Error seems something like: failed to load resource: xmlGetAuditTrail.jsp?referenceId=bpel%3A//localhost/<<domain-name>>/<<process-name>>%7E1.15/431669&id3j0son2=1

After doing extensive digging in the JSP code a little bit, I found that one of the Java Script which was making server call had 20 second timeout!! After bumping up that to higher number, I was able to resolve that issue. Here is the fix:

$ORACLE_HOME\j2ee\oc4j_soa\applications\orabpel\console\ui\util2\1001.js
(search for rInfo.timeout)
increase timeout from 20 to 200

BPEL Title WildCard Search

2010-03-01T18:05:00.001-08:00

I probably would be blogging on setting title for BPEL processes in non-intrusive way, but have to get through couple of hurdles before I get there. It is quite common practice to set Titles for BPEL process so that you can search from BPELConsole.

e.g. setTitle("Title = " + title); //in embedded Java (you can also look at http://blogs.oracle.com/pt/2009/02/searching_bpel_process_instanc_1.html if you are interested on how to use title in intrusive way)

The problem people complain is that BPEL title search doesn't support wild card search. It only supports exact search. While looking through BPELConsole JSP, I found this was quite easy change it from exact search to wild card search. The query was getting generated as part of $ORACLE_HOME/j2ee/oc4j_soa/applications/orabpel/console/ngInstanceList.jsp. To change exact title search to wild card search, here is the change required:

- In $ORACLE_HOME/j2ee/oc4j_soa/applications/orabpel/console/ngInstanceList.jsp, search for keyword called instanceTitle, you should see something like:

String instanceTitleQ = request.getParameter( "instanceTitle" );
if ( instanceTitleQ != null && instanceTitleQ.length( ) != 0 )
{
buf.setLength( 0 );
tmpWhere.setClause( buf.append( n++ > 0 ? " AND " : "" )
.append( SQLDefs.AL_ci_title )
.append( " = ? " )
.toString() );
tmpWhere.setString( 1, instanceTitleQ );
where.append( tmpWhere );
}

You can change it to:

String instanceTitleQ = request.getParameter( "instanceTitle" );
if ( instanceTitleQ != null && instanceTitleQ.length( ) != 0 )
{
buf.setLength( 0 );
tmpWhere.setClause( buf.append( n++ > 0 ? " AND " : "" )
.append( SQLDefs.AL_ci_title )
.append( " LIKE ? " )
.toString() );
tmpWhere.setString( 1, instanceTitleQ );
where.append( tmpWhere );
}

Once it is changed, you can search for Title in BPELConsole using % as wild card character.

GreeseMonkey

2009-02-24T20:54:00.001-08:00

I was working at client site and their web based authentication for Wireless keep expiring every 10 minutes. It was such a pain to login every single time. I tried a Java program but didn't go that well with unknown issuer X509 certificate.

I found GreeseMonkey pretty helpful, just a few lines of javascript code and you can ask Firefox to do whatever you want. I just wrote a few line of script to get page, put username/passwords and submit every few minutes, and never login again..

Also found ready made script for Outlook Webmail Autologin, saved tons of my keystrokes and refresh.

Spring SOA

2008-10-29T18:47:00.001-07:00

Haven't got time to write any Blog entry in past 3 to 4 months. Working on my dreams: building my own company (http://www.springsoa.com/), learning taxes, hiring employees/contractors, bidding for projects. Not everything goes smooth and successful, but I am quite happy with what I could accomplish in last couple of months especially at the same time when I am still working on my own projects. Hopefully things will go better, also will add some reusable component library shopping cart for Fusion Middleware (FMW) and may be FMW training sessions.

Refreshing Connection Pool via MBeans

2008-06-27T16:43:00.000-07:00

You can easily refresh the connection pool via Enterprise Manager. If you look at MBean Browser (for more information please look at my earlier blog on JMX http://chintanblog.blogspot.com/2007/12/it-is-all-about-jmx-i-worked-on-jmx.html, it is very comprehensive blog) you can search for your Connection Pool it provides management interface to refresh the connection pool as well. I believe EM internally must be using MBean interface.

Anyways, here is programmatic way of accessing MBean related to Connection pool and doing refresh using MBean client API.

To get Mbean Name, you can look at MBean Browser as displayed here. You can also see it supports testConnection, refreshConnection, etc.. methods.

You can see Mbean name and method name in the Mbean browser, you invoke here if you want also.

Here is the code to execute this method using Java API:

Here MBean name and method name I have got it from System MBean Browser. This code works great and does invoke the refreshConnectionPool method.

Testing the connection pool refresh

To replicate the scenario where connection pool get broken and remain broken without calling my refresh method was quite tricky to figure out. I tried multiple different scenarios and here is the one which worked great and as expected.

Here is the use case and steps I came up with:

1) Start database and create connection pool and data source
2) Test the connection pool using Mbean browser "testConnection" method
Expected Result: it should work without any isuse.
3) Stop database and Start the database
4) Test the connection pool using Mbean browser "testConnection" method
Expected Result: it should not work, and should show error (at least for a while) because of stale connection
5) Stop database and start the database
6) Run refresh Mbean Client routine
7) Test the connection pool using Mbean browser "testConnection" method
Expected Result: it should work without any issue.

Execution of the usecase:

Step 1) Start database and create connection pool and data source:
I created connection pool called TestPool which points to Test Schema

Step 2) Test the connection pool using Mbean browser "testConnection" method:
As show below it was successful.

Step 3) Stop database and Start the database

Step 4) Test the connection pool using Mbean browser "testConnection" method
As we already know, testConnection method failed with four different error message each time we invoke test connection method. After these four messages, connection was successful.

Exception occurred testing connection. Exception: java.sql.SQLException: No more data to read from socket.
Exception occurred testing connection. Exception: java.sql.SQLException: OALL8 is in inconsistent state.
Exception occurred testing connection. Exception: java.sql.SQLException: Io exception: Software caused connection abort: socket write error.
Exception occurred testing connection. Exception: java.sql.SQLException: Closed Connection.

Step 5) Stop database and start the database

Step 6) Run refresh Mbean Client routine

Step 7) Test the connection pool using Mbean browser "testConnection" method

This time it just works fine...

Code for MBean Client can be downloaded from here.

DB Adapter Tricks

2008-06-26T22:51:00.000-07:00

1. Dealing with Special Characters in Table/Column name

Recently saw the issue on forum that Database does allow creating table or column with special characters inside the name e.g. $, #, etc.. Based on W3C standard (http://www.w3.org/TR/xmlschema-1/#cElement_Declarations) not all special characters are allowed in XSD element/attribute names. $ is certainly not allowed. Oracle Database adapter creates one-to-one mapping between database column/table name and element/attribute name.

Sample Table: I created table named "mytable" with following script:

create table mytable (
id int,
company varchar2(100),
my$comments varchar2(100),
processed varchar2(100),
processed_time date
);

a) Custom SQL : No toplink mapping is used, in this case conversion from $ value to _ is done automatically. Means, when DB adapter creates XSD file it creates attribute with my_comemnts. That's cool, no work.

b) Insert/Select operation: That uses Toplink mapping which is basically <<SERVICENAME>>_table.xsd file and <<SERVICENAME>>_toplink_mappings.xml file. We can change _table.xsd file and convert my$comments to my_comments as shown below:

Only these changes are required in order to make things work. The same thing can achieved for table name and other types of special characters.

2. Dealing with Dynamic Queries - Query By Example

For simple use case, you can always use custom query with multiple parameters e.g. "select * from TABLENAME where COL1 = ?1 and COL2 = ?2". It works perfectly fine and input parameters shows up in XSD and also in wsdl contract.

Sometimes, some richer query is required, usually the dynamic where clause. Here is the example:

sometimes, you need query, e.g. select * from tablename where col1 = ?1
sometimes, you need query, e.g. select * from tablename where col1 = ?1 and col2 = ?2

Here you can see the where clause is getting build dynamically. It is supported by Oracle DB adapter and it is called Query By Example. It was very well supported in 10.1.2. but somehow it got deleted by some Jdev developer, the good thing is that it is still supported at runtime and you can make small changes to SELECT operation to convert it into Query By Example.

What it does: It takes lets say object as input (e.g. Employee), and returns collection of Employee as output. Here the trick is, where clause is created based on Employee attribute you provide. So if pass null Employee object it would "select * from Employee" querey. If you provide Employee input with Company name "Oracle", it will return all employee matching company name oracle...

How to implement it:

I created DynamicQueries BPEL process and DBSelect partnerlink which does simple SELECT operation (not custom sql), and no parameters on table name Test. Now, we need to modify two files:

DBSelect_table.xsd: Create an Element called Test.

DBSelect.wsdl: Specify keyword called IsQueryByExample="true" in your JCA operation as shown below:

That's all it needs. You need to create input variable of Element "Test" and assign it to Partnerlink Input variable and deploy the process.

3. Inserting multiple records in Database in single transaction

Well, I know that is supported out of the box and nothing you need to do special to achieve it. If you create database adapter in your BPEL process with INSERT operation it is exposed as collection of object as Input. You can use XSLT to assign that Collection and all records will be inserted in one atomic transaction.

Working example for all three can be found here.

File Size from File Adapter

2008-06-19T17:03:00.000-07:00

Well, reading/writing file name and file directory are very well documented in File Adapter guide and blogs. Just to recap, in order to read/write file name and directory you are supposed create a header variable of element InboundFileHeaderType (it will be different if it is outbound header variable). The WSDL file is created when you create FileRead or FileWrite operation.

After you create Variable, we just need to specify that variable as part of Header variable in Adapter.

That's all required, and during file polling you will get the fileName and directoryName already populated.

Wait a minute, but if you look at Variable_FileHeader at runtime in Assign Activity, you can see 5 elements are populated rather than 2. E.g.

It means, this header variable is populating 5 elements, where wsdl file is exposing only two! How to get remainder three, or let;s say SIZE which is more interesting to all of us.

Two ways:

1) Hack the wsdl file and put size as one of the element as shown below. (Note fileAdapterInboundHeader.wsdl file is created with ReadOnly attribute so you have to change the attribute prior to modification).

After doing this size attribute is populated and you can assign to any other variable if you like.

2) This is non-intrusive way. As we know that Header Variable (XML Element) already has size, it is just matter of extracting it, so why not to convert Header Variable in String and do string search? e.g. I did following assign activity to get the size and assign it to some string variable called Variable_File_Size. You can see, that I am converting xml element to string and then doing search for string between <size> </size>.

Things didn't end here, as somebody asked me that how to know how many elements/attributes are really supported. I couldn't find the documentation anywhere, so back to the JAD, and I could find the file name called "oracle/tip/adapter/file/FileAgent.java" which had following lines of code:

Based on code, it looks like variables are just getting inserted on the fly, as it is not JAXB but just being treated as DOM element. Ofcourse, it showed me a way what I need to do if I need to insert my custom header in File Adapter.

Here is the entire code used for sample.

Business Rules WebDav Repository

2008-06-18T13:39:00.000-07:00

I was used to configure WebDav on Oracle Database for Oracle Business Rules, it was pretty hard to configure and very unstable. Recently found out that I can configure WebDav on Oracle Apache.

Usually to install WebDav on vanilla Apache server, we need to install couple of dll file and load those module during Apache startup, but for Oracle Apache all configuration are done out-of-the-box, only thing we need to specify is location and type of repository.

Configuration file:

%soasuite%/Apache/oradav/conf/moddav.conf

To enable default WebDav repository:

<Location /dav_public>
DAV on
</Location>

Once you specify on, http://host:port/dav_public (which located under %soasuite%/Apache/Apache/htdocs/dav_public is ready for use as webdav repository.

To create new WebDav repository:

1) Add following entry in %soasuite%/Apache/oradav/conf/moddav.conf file

<Location /my_webdav_repository>
DAV on
</Location>

2) Create directory called %soasuite%/Apache/Apache/htdocs/my_webdav

3) Restart the Apache server and that's all required for configuring custom repository.

To create password for WebDav repository:

1) Create authentication file with different user, I created

%soasuite%/Apache/Apache/bin/htdigest -c %soasuite%/Apache/oradav/conf/webdav.access webdav-authentication oc4jadmin
%soasuite%/Apache/Apache/bin/htdigest %soasuite%/Apache/oradav/conf/webdav.access webdav-authentication ruleauthor

2) Change %soasuite%/Apache/oradav/conf/moddav.conf to provide the authentication mode:

<Location /my_webdav_repository>
DAV on
AuthType Digest
AuthName "webdav-authentication"
AuthDigestFile %soasuite%/Apache/oradav/conf/webdav.access
Require valid-user
</Location>

3) Restart the server and now my_webdav_repository will be accessible only after username and password authentication.

Flex Field Mapping Migrator

2008-06-17T12:13:00.000-07:00

Nothing special, just putting together some BPEL client API to create useful utility for BPEL Human Workflow migration.

Flex field mappings are very useful for creating custom views/queues and reports. Flex-Field mapping in worklist are persisted in Workflist Schema with MD5 encoded guid. It is not advisable to promote them from one server to another server via database scripts. BPEL workflow client API supports extracting, creating and deleting such mappings for worklist application.

Here is the utility which I created by using worklist client API, which I have released it as JAR file, and I wrote wrapper BAT file to execute the command.

As mentioned earlier I support three COMMANDNAME export, import and clean. Export will generate workflowmappings.xml file in the current directory and import will take workflowmappings.xml file and import all the payload mappings to the target server. My import program is not very intelligent, it cleans all the mapping on target server and creates them according to workflowmappings.xml file, it is not smart update.

It can be downloaded from here.

Can you create a domain with upper case name?

2008-05-27T15:41:00.000-07:00

Realized that you can not create domain with upper case, and if you do then you have login in BPELConsole in very weird way. I saw the meta-link and this issue been around for more than a year. I looked into exploded version of BPELConsole, it took me couple of min to fix it, and here is my hack:

Either you can get modified JSP files from here and dump it to %soasuite%/j2ee/oc4j_soa/applications/orabpel/console, or follow the steps described here:

1. %soasuite%/j2ee/oc4j_soa/applications/orabpel/console/index.jsp

Search for lines, with IBPELDomainHandle ch = l.lookupDomain(), and comment it out 2 lines. It is about line number 15 or 16.

Now if you login to BPEL console, you can see the Domain Picker jsp file and I thought it will just work fine, but I had to modify domainPicker jsp file as well to get thing done.

2. %soasuite%j2ee/oc4j_soa/applications/orabpel/console/domainPicker.jsp

a) Search for string called “alreadyAuthed.add( aDomainId )”, it will be about line number 39, and add following line of code

b) Need to change Java Script at the end of page:

c) Finally Change the submit button, and that’s all.

No need to bounce the server to see the magic! It feels like hacking is my business and consulting is byproduct.

Rule Engine Tricks

2008-05-27T15:39:00.000-07:00

While working with Oracle Business Rules I found some tricks:

1) Enable Logging

Provide following properties in decisionservice.desc file in your bpel process:

<properties>
<property name="watchRules">true</property>
<property name="watchActivations">true</property>
<property name="watchFacts">true</property>
<property name="watchCompilations">true</property>
</properties>
You have to provide this right in <ruleEngineProvider> and right after the <repository> tag. If you have worked with Rule Engine SDK, then you may realize that it is not really providing all debugging information which you can see via ruleSession.executeFunction("watchFacts"). Well, be happy with what it provides, it is good enough. If you want to look at more information then look at my article on Custom Decision Service.

2) Additional Libraries

If your BPEL process is having decision service and your Rule is calling external Java program, then you will need to include those Jar files in to your path. The directory would be: decisionservices\DecisionService\war\WEB-INF\lib under your BPEL root directory.

3) RuleSet invoking other RuleSets

Hmm, this was cool one. If you have ruleSet and inside that ruleSet you are calling another ruleSet via pushRuleSet functioin, it just works fine in RuleAuthor, but decision service interface doesn't let you execute it. It throws exception something like "Rule Set xyz is undefined". Well, again if you are familiar with how Rule SDK works, then you can guess why this is the case. Developer of decision service probably loading only one rule set and that is what causing the issue.

Resolution: Rule function is quite different, if you execute rule function and if rule function calls multiple RuleSet that just works fine. When I created my custom decision service, I did the same for both RuleSet and Rule Functions. Therefore, for out-of-the box rule decision service, we have to create function interface in order to interact with multiple rule-sets.

4) Deploying to home container

As shown in below diagram, if you specify oc4j_soa while deploying decision service, it just works fine. If you specify home during creating Jdeveloper connection while deploying Decision service it throws following exception. Therefore, we have to use oc4j_soa while making Jdeveloper AS/Integration server connection.

[deployDecisionServices]
[deployDecisionServices] 08/05/14 18:20:26 Notification ==>application : rules_default_TestBPEL1_1_0_DecisionService is in failed state

[deployDecisionServices]

[deployDecisionServices] Exception in thread "ConfigurableThreadImpl::" java.lang.NoClassDefFoundError: oracle/classloader/util/LocalizedText
[deployDecisionServices] at com.evermind.server.rmi.RMICall.EXCEPTION_ORIGINATES_FROM_THE_REMOTE_SERVER(RMICall.java:109)
[deployDecisionServices] at com.evermind.server.rmi.RMICall.throwRecordedException(RMICall.java:125)
[deployDecisionServices] at com.evermind.server.rmi.RMIClientConnection.obtainRemoteMethodResponse(RMIClientConnection.java:517)
[deployDecisionServices] at com.evermind.server.rmi.RMIClientConnection.invokeMethod(RMIClientConnection.java:461)
[deployDecisionServices] at com.evermind.server.rmi.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:63)
[deployDecisionServices] at com.evermind.server.rmi.RecoverableRemoteInvocationHandler.invoke(RecoverableRemoteInvocationHandler.java:28)
[deployDecisionServices] at com.evermind.server.ejb.StatefulSessionRemoteInvocationHandler.invoke(StatefulSessionRemoteInvocationHandler.java:31)
[deployDecisionServices] at __Proxy2.getEvents(Unknown Source)
[deployDecisionServices] at oracle.oc4j.admin.jmx.client.MBeanServerEjbRemoteSynchronizer.getEvents(MBeanServerEjbRemoteSynchronizer.java:530)
[deployDecisionServices] at oracle.oc4j.admin.jmx.client.CoreRemoteMBeanServer.getEvents(CoreRemoteMBeanServer.java:319)
[deployDecisionServices] at oracle.oc4j.admin.jmx.client.EventManager.run(EventManager.java:217)
[deployDecisionServices] at oracle.oc4j.admin.jmx.client.ThreadPool$ConfigurableThreadImpl.run(ThreadPool.java:303)

How to start Enterprise Manager

2008-05-27T15:36:00.000-07:00

If you shutdown Enterprise Manager from Console or any other way here is the trick to start the Enterprise Manager:

soasuite/j2ee/home/config/default-web-site.xml
<application name="ascontrol" path="../../home/applications/ascontrol.ear" parent="system" start="true" />
<application name="javasso" path="../../home/applications/javasso.ear" parent="default" start="true" /> (not sure if it is required all the time)

soasuite/j2ee/home/config/server.xml
<web-app application="ascontrol" name="ascontrol" load-on-startup="true" root="/em" ohs-routing="true" />

Bounce the home container using following command

opmnctl restartproc process-type=home

Custom Decision Service

2008-05-23T10:10:00.000-07:00

I am not very big fan of Decision Service which comes out of the box with Oracle BPEL. There are couple of issues I found

- It creates whole bunch JAXB classes in my bpel process, which are .java and .class files (I have to keep both of them, .class files are not generated during compilation of bpel process!)

- I can NOT change the created Decision Service, it is not like I can click on DB adapter and reconfigure it. It make my life extremely difficult when have to accommodate changes from client

- I can not call the service from ESB or other WS consumer, the decision service exposed takes millions of parameters which are not relevant to web service consumer (e.g. bpelInstanceId? why in the hell rule engine needs to know bpel instance id?)

- There is a decision service for each partner link! We have 20 BPEL processes making almost 30 different type of calls, that ends up in 30 decision services. That make my life impossible to manage.

- It is way too complicated to implement WSIF with the decision service.

I love the idea of BAM, where all Active Data cache operations are exposed as one service. I had very naive and romantic idea of implementing my custom decision service and talk to any type of rule set, rule function, do the connection pooling of rules via one web service interface. It can take any type of parameter and using Java reflection I can convert to right type and that's all I need. My client did buy this idea and I implemented very nice and generic Custom Decision Service.

1. Implemented POJO

- I implemented a small Java POJO service, which does takes configuration

e.g. RuleRepositoryType (webdav, files), File Repository Connection Information, WebDav Repository Connection Information, Whole bunch of debug flags which are supported by Rule Engine, Dictionary Name, Rule Function or Rule Set, Input types and output types.

2. Implemented EJB 3.0 and exposed as Web service via annotations

- Just created EJB 3.0 session facade on top of POJO so that it is container managed, I can easily expose as Web Service via Annotations, and for implementing Connection pool.

3. Implemented Connection Pooling in EJB

- I implemented custom connection pool of Rule Repository and Rule Dictionary. I initiate new Rule Session each time. I also noticed that pooling/caching Rule repository was not just good enough so I had to cache Rule Dictionary as well. Yeah, I implemented

4. WSIF Interface via annotations

- I just followed http://chintanblog.blogspot.com/2007/12/although-wsif-seems-to-be-hack-to-plug.html to create WSIF on top of my EJB 3.0 session facade.

5. Performance Matrix

Test case: Used WebDav repository, Rule Function interface which executes more than 10 rule sets and creates whole bunch of external connection. It usually takes about 2 seconds to complete the Rule Session itself. I used about 10 threads and loop of 50 with 1 second delay in each (so total number of instances will be 500).

Default Decision Service over SOAP: I created sample BPEL process and created decision service with decide activity (which comes with jaxb java and classes, jar, ear, war, and so on...). Here were the results:

Average execution time: 3.1 second

Number of failures: 15

Custom Decision Service over SOAP: To be fair I didn't use WSIF interface. Calling my custom decision service over soap protocol. It is just like normal partner link.

Average execution time: 3.5 second

Number of failures: 10

Notes:

- Reason behind slowness is because of I am doing more reflection which is a bit expensive.

- Because of EJB and container managed stuff, I am getting less number of failures.

- Out-of-box decision service caches the Rule Connection where the custom decision service manages the entire pool which is shared across the enterprise. I believe if I test with 20 different BPEL processes with different pattern of calls, I will be far ahead in comparison with default decision service.

I am more than willing to get your feedback on this. I am not sure if I did right thing, but I believe I reduced (rather completely eliminated) overhead for managing far complex default decision service.

ESB Display Older Instances

2008-05-22T14:30:00.000-07:00

I used to have my own interceptor logging all required thing into the database, so never had to look at ESB console.

Issue 1 :

Recently when I was working on ESB console I realized that it displays only a day old instances! While looking into the database I saw all instances where there.

Yeah, of course the ESB client API was causing the issues. As mentioned in my earlier blogs, to debug ESB Console related issues, the best things is to use obtunnel to intercept all your request and then simple wingrep/grep to search for what you interested in.

The reason behind day old instance was easy to find:

File Name: %soasuite%/j2ee/oc4j_soa/applications/esb-dt/esb_console/esb/model/model.InstancesSearch.js

Where: Search for key word "InstanceSearchModel.prototype.init" about the line number: 150 in 10.1.3.1

Content:

Original:

InstanceSearchModel.prototype.init = function(serviceConfigROTID)
{
this._trackingSC = new TrackingSearchCriterion(serviceConfigROTID);
this._serviceEntityInfo = null; // ESBEntityInfo
this._status = "Any";
this._flowID = "";
this._timePeriod = 1;
this._timeUnit = "days";
this._timeZoneString = InstanceSearchModel.getTimezoneString(new Date(), false);
}

Basically InstanceSearchModel creates Rest Webservice request to ESB server, and timePeriod is configurable parameter to this object. Based on my observation, other javascript which are using InstanceSearchModel object are not passing this parameter, so default parameter will be used which is 1 day. I changed the default to 20 days and it showed 20 days worth of instances...

Modified:

InstanceSearchModel.prototype.init = function(serviceConfigROTID)
{
this._trackingSC = new TrackingSearchCriterion(serviceConfigROTID);
this._serviceEntityInfo = null; // ESBEntityInfo
this._status = "Any";
this._flowID = "";
this._timePeriod = 20;
this._timeUnit = "days";
this._timeZoneString = InstanceSearchModel.getTimezoneString(new Date(), false);
}

Issue 2 :

ESB has limitation on returning only 100 results. It was an amanzing experience to rip off the entire oraesb.jar and go through the hierachy of java files,

soasuite/j2ee/oc4j_soa/applications/esb-dt/esb_console/WEB-INF/web.xml
oraesb/oracle/tip/esb/configuration/servlet/CommandServlet.jad
oraesb/oracle/tip/esb/configuration/servlet/command/GetInstancesCommand.jad (xmlInstanceManager.getInstances)
oraesb/oracle/tip/esb/console/XMLInstanceManager.jad
oraesb/oracle/tip/esb/console/XMLInstanceManagerImpl.jad
oracle/tip/esb/monitor/manager/ActivityMessageStore.jad
oraesb/oracle/tip/esb/monitor/manager/database/DBActivityMessageStore.jad (InstanceListXMLBuilder)
oraesb/oracle/tip/esb/monitor/manager/database/InstanceListXMLBuilder.jad
oraesb/oracle/tip/esb/monitor/manager/database/FilterParser.jad (String s = RepositoryFactory.getRepository("RUNTIME").getESBParameter("MaxInstanceCount"))

Bingo! Finally I found the last file which was reading the parameter from ESB parameter table, now it was piece of cake!

I added, insert into esb_parameter (PARAM_NAME, PARAM_VALUE) values ( 'MaxInstanceCount', '1000' ); in ORAESB schema, and bounced the server and it started working like magic!

MQ Adapter Anatomy

2008-05-13T13:59:00.000-07:00

Recently I had chance to work with MQ Adapter, and it was pretty nice experience and I pretty much tried all different types of settings supported by MQAdapter. Here I would like to share something.

If you religiously follow the document: http://download.oracle.com/docs/cd/B32110_01/web.1013/b28956.pdf then there is not much value I will be adding, but still you might find something useful and accumulated information.

1) Different Settings

Now let's take a look at each parameter:

Scheme: Three types of scheme
1. dynamic : use max connections, if all are in use, then give new one and don't worry about maxConnections limit
2. fixed: use max connections, if all are in use, then throw exception for new request
3. fixed_wait: use max connections, if all are in use, then do fixed amount of wait for newly requested connection

Best practice is to use fixed_wait with higher timeout so that we can restrict the number of connection

minConnections/maxConnections/initial-capacity: Decided number of connections and initial connections during start up.

waitTimeout : This is the timeout used during fixed_wait scheme for new requests

inactivity-timeout-check: When to check for expired or inactive connections. Supported values are "never","periodic","piggyback" (when a new connection is fetched), and "all" (periodically and when a new connection is fetched).

We kept it “all” as we don’t want to keep connections idle longer than certain time period.

inactivity-timeout: The desired connection timeout, in seconds, as a positive integer, or 0 for connections to never expire. Negative values are disallowed.

We have kept it 60 seconds.

2) "Garbage Collector" - clean up routine

Basically three parameters are used to configure for releasing the idle connection held by OC4J connection pool.

inactivity-timeout-check: has to be "all" or "piggyback", all is desired if you want to periodically run clean up routine.

inactivity-timeout: The desired connection timeout, in seconds, as a positive integer, or 0 for connections to never expire. Negative values are disallowed.

Server.xml (taskmanager-granularity): When inactivity-timeout-check is configured to be periodic, this value specify how specified how frequently we want to run the periodic recycling routine.

Here is sample server.xml entry:

3) Different Level of Caching and Related Errors

Actually, after setting 1 and 2, connections were still not getting cleared, and we were able to see on MQ server side (using Tivoli) that connections are not getting closed. We found from Oracle product managers, that Adapter itself manages cache for MQ (or any type of) connections.

We need to provide, cacheConnections=false property in BPEL Partnerlink or ESB end point property. After that ESB and BPEL releases connection nicely and clean up routine described in No 2, works great and cleans up all idle connections.

4) Inbound and Outbound MQ adapter

Another finding I found was, difference between inbound and outbound adapters. If you have MQ outbound adapter with "cacheConnections=false", it releases connection right after completing the unit of work, and connection is available to OC4J connection pool. But if you have MQ inbound adapter with "cacheConnections=false", it still holds the connection. Probably it is because of continuous polling.

I have not tried it with changing the polling frequency to see if connection get releasesd.

Big News - Resigned Oracle

2008-05-09T21:57:00.001-07:00

Joined on Nov 29, 2005 and finally completed my last day at Oracle today. Feel pretty sad and same time very exciting to face new challenges by my self. Now I am becoming independent consultant in Oracle SOA space and without any expert's support.

I am may be putting my career at risk but as we all know: The biggest risk in life is not to take one!

ESB (and BPEL) purge instances

2008-04-09T11:55:00.000-07:00

There are multiple ways we can use to purge ESB instances.

Database Scripts
For 10.1.3.3.1 SOASuite ESB purge instances scripts are provided as part of installation (%soasuite%/integration/esb/sql/other):
- purge_by_date.sql
- purge_by_instance_id.sql
- purge_by_id.sql

ESB Client API
Database scripts are always good, but it should be best practice to rely on API provided by product. Same as BPEL Client API, whatever you can do in ESB console you can do via ESB client API. I used %soasuite%/bpel/bin/obtunnel.bat to check what XML based payload it is using. For cleaning the ESB instances, here is the sample code:

purgeInstance.consoleClient = purgeInstance.getESBClient();
String purge = "<instanceManage enable='true' userPurgeTimePeriod='14400000'/>";
HashMap<String, String> requestProps = new HashMap<String, String>();
requestProps.put("root", purge);
purgeInstance.consoleClient.perform("UpdateTrackingConfig", requestProps);

Here time period is in milliseconds and all instances older than timestamp will be purged. If we need to purge all instances, we need to change userPurgeTimePeriod='0'.

Custom ANT Task
I saw the custom ANT tasks for managing BPEL domain, purge instances, etc.. It is an amazing job done by Ramkumar Menon (http://blogs.oracle.com/rammenon/2007/11/26#a74 ). I used his framework and added some tasks for managing ESB.

It is really easy to use this tasks, e.g. for managing BPEL domain you can specify following in build.xml:

<BPELServerAdmin providerurl="${bpelProviderURL}" username="${bpelUsername}" password="${bpelPassword}">
<manageDomain domain="${bpelDomainName}">
<purgeInstances select="all" fromdatetime="${bpelPurgeInstanceFromDateTime}" todatetime="${bpelPurgeInstanceToDateTime}"/>
<undeployProcesses select="BPELProcess.*" type="all" revision="all"/>
</manageDomain>
</BPELServerAdmin>

This will purge instances based on timestamp and undeploy the processes.

For ESB, this is what I added:

<ESBServerAdmin username="${esbUsername}" password="${esbPassword}" hostname="${esbHostName}" port="${esbPort}">
<purgeESBInstances todatetime="${esbPurgeInstanceToDateTime}"/>
</ESBServerAdmin>

You can purge ESB instances based on timestamp, from and to is not supported. Probably it can be enhanced if there is requirement.

Here is the link for downloading Ant task with readme file, sample build.xml and sample build.properties file.

Performance with ESB and BPEL UDDI runtime lookups

2008-04-08T14:57:00.000-07:00

Well, nothing new with it, as long as you are using SOASuite 10.1.3.3.1 you should have both of them out of the box. Just to recap:

BPEL run-time look-ups:

1) You have to define/publish wsdl in service registry. If you publish wsdl through User Interface, it creates unique sequence id for registry key which you can change using registry user interface. You can also use registry API to publish wsdl of the service.

2) Define registryServiceKey as property of partnerlink

You can either edit bpel.xml or edit the partnerlink to achieve it:

3) Define registry URLs in domain.xml

URL for uddiLocation should be http://host:port/registry/uddi/inquiry.

ESB run-time look-ups:

2) Define registryServiceKey as end point property for service

3) Define registry URLs in integration/esb/config/esb_config.ini file

I have tried both ESB and BPEL run-time look-ups and they work great. Although there are some really bad performance issues I saw:

After enabling run-time look-ups ESB or BPEL calling any service via UDDI was taking longer than 1 minutes to invoke (it was couple of milliseconds earlier). Upon analysis we find out the actual invocation was not the issue but preparing the call (get-wsdl, parse wsdl and store in specific cache) was taking the longest time.

Also we saw that if load on BPEL and ESB if not very high, I don't see many performance issues but as soon as load increases (with number of threads) things get really slow. We did look at service registry logs and it's response time, service registry was extremely fast.

Here is what we found the reason behind slowness: ESB and BPEL both caches WSDL into their cache in specific format. If we do run-time look-ups BPEL and ESB gets the WSDL using service registry (which is pretty fast), but analyzing those wsdl and storing them in structure which BPEL and ESB run-time engine can understand takes a long time.

In conclusion, all this stuff BPEL, ESB, OSR - run-time look-ups looked like an ideal approach but had some performance drawbacks. All that glitters is not gold :)

ESB read-only Console

2008-03-31T15:14:00.000-07:00

I know this functionality is supported out of the box, but there are some special touches added by me, and thought to share with everybody.

To create ESB read-only user account:

- Create user in EM with ascontrol_monitor, and that's all is required for creating read-only account for ESB read-only.

Now if I login to ESB console using esbreadonly user, everything works fine. There are some gotchas here:

- You can delete any service using esbreadonly user!

- Sometime you want to enhance functionality, e.g. not all readonly user should have access to routing rules, etc...

I published the article for BPEL Read-only Console , it was fairly easy because BPEL read-only console are all JSP and I used Servlet filter to achieve that functionality. Now ESB console is all Java Script underneath. From directory names I can see it is Picasa framework, not sure how that works, but debugging JS is hell lot of time unless you have good editor.

Disable Delete button for Readonly Users:

The best way to debug Javascript is just to use wingrep and find the pattern which you are looking for. For disabling DELETE button, here is the trick:

- open %soasuite%/j2ee/oc4j_soa/applications/esb-dt/esb_console/esb/commands/controller.ServiceNavigator.js

- Search for following two entry (it should be about 399 or 400 line in the file:

var a=ActionController[gActionDeleteService];
a.enable();

- Add following line of code below the a.enable(); line:

/* cshah changes : added code to disable delete button */
if(!isAdminUser()) {
a.disable();
}

That's all and DELETE button will be disabled for non-admin users.

Enhance functionality for ESB console, disable tabs e.g. Routing Rules

- open %soasuite%/j2ee/oc4j_soa/applications/esb-dt/esb_console/esb/commands/controller.ESBController.js

- Search for following two entry (it should be about 614 and 615 line in the file:

tabmodel.enableTab(TabModel.TAB_RTNGRULES_IDX);
tabmodel.enableTab(TabModel.TAB_TACKINGFLDS_IDX);

- Add the following line of code right below tabmodel.enableTab(TabModel.TAB_TACKINGFLDS_IDX);

/* cshah -- disable routing rules if the user is not admin */
if(!isAdminUser()) {
tabmodel.hideTab(TabModel.TAB_RTNGRULES_IDX);
}

This will remove Routing Rules tabs for read-only (non-admin) users.

OWSM Offline Agents

2008-01-10T10:17:00.000-08:00

Per architecture OWSM Agents intercepts the requerst, calls the policy manager to retrieve the policy and then executes those policy where agent is installed. It introduces lot of traffic on policy manager. Policies are stored in database and policy manager is single point of contant for them.

Although we can break this dependency by offline agent installation, we can export all the policy from database as an XML file and provide that XML file to OWSM agent during the time of installation. In this case, owsm agents will look into the file, and wouldn't contact policy manager. We can update this file and policies at run time without bouncing the server.

Here are steps I followed:

Step 1: Create a client agent using OWSM ccore User Interface and defined the policies as I would for any other client agent.

Step 2: I expored those policies to an XML file using "save" button:

Step 3: Now it is time to install this agent with policy file already exported. Client agent installation is explained pretty well in http://www.oracle.com/technology/obe/fusion_middleware/owsm/index.html , but here I will just provide required informaiton to make offline.

I provided following agent.properties and then used "wsmadmin installAgent" to install the client agent.

agent.componentType=OC4JClientInterceptor
agent.containerType=OC4J
agent.containerVersion=10.1.3
client.home=d:/soasuite/j2ee/oc4j_soa
agent.component.id=C0003007
agent.policymanager.enabled=false
agent.policySet.file=D:/temp/owsm/OfflineAgentPolicySet.xml

Here I had to provide the policy file and disable the policy manger.

Step 4: Final testing: After this, I just create two BPEL process Calle and Caller to test my client agent, and it worked simply great. Upon changing the file name my client agent didn't work and upon updating the file client agent picked up new changes. It worked..

You can download agent.properties, policy file and test code from here.

OWSM Client Agents

2008-01-10T09:13:00.000-08:00

It took me a while to configure all client agents, here is some information I would like to share. OWSM supports 3 types of client agents (http://download.oracle.com/docs/cd/E10291_01/doc.1013/e10298.pdf page 6-6):

- J2EE Client Agents: Configured when Servlet/EJB makes call to WebService and web service clients are maintained by Container

- J2SE Client Agent: Configured when Stand-Alone Java is making call to WebService.

- WSIF Client Agent: To leverage the WSIF framework used by ESB and BPEL. Configured when ESB/BPEL making outbound call to webservice.

Client agent intercepts the outbound requests and executes the policies defined by policy manager.

WSIF Client Agent: It is very well explained at Oracle OBE: http://www.oracle.com/technology/obe/fusion_middleware/owsm/index.html

J2SE Client Agent: Here are the steps I followed to create J2SE client agent.

Step 1: I created BPEL webservice called HelloBPELAgentTestingCalle. Deployed that process to BPEL PM and secured it using Server Agent which requires WSSE authentication information.

Step 2: I copy the WSDL link and created J2EE webservice proxy from JDeveloper.

It creates pretty much all code required to invoke the webservice, although I had to add couple of lines of code to test the way I wanted:

I tried to create fake security token from wizard (and also per documentation), I created HelloBPELAgentTestingCalleBinding_Stub.xml and put that file in src and also under src/../proxy/runtime directory. The content of this file is very well explained in WSIF client agent.

Now after putting all libraries in classpath as mentioned in document: client.home/owsm/lib/extlib, ORACLE_HOME/owsm/lib/cfluent-log4j.jar, JDBC jar file, ORACLE_HOME/jlib/orai18n.jar, ORACLE_HOME/jlib/ojmisc.jar

Now if I execute the client using JDeveloper, I can see that it is executing the client agent before it goes to real service.

J2EE client agent:

I tried to create J2EE client by creating Web Service proxy mentioned earlier and then wrapping it up in servlet or JSF page.

I found out that if I have to follow http://download.oracle.com/docs/cd/B31017_01/web.1013/b28974/j2eewsclient.htm to create webservice client and then follow the document http://download.oracle.com/docs/cd/E10291_01/doc.1013/e10298.pdf for configuring my war or jar file. I believe that's too much work and per my personal experience I would avoid recommeding such complicated way to client.

I went for little easy solution, although I my client agent will not be managed by container, but I can get same functionality served. I used my J2SE client which _stub.xml file and all other good stuff. I created Servlet wrapper around it and deployed to application server, things works just fine....

In a nutshell, you can use J2SE client approach and still deploy to container. It will serve the the purpose but won't be managed by container.

You can download the code here for WSIF, J2SE and J2EE client agent over here.

OWSM Field Level Encryption

2008-01-09T22:06:00.000-08:00

I had chance to work on OWSM agents and different type of encryption. Here I would like to present the solution for OWSM field level encryption. I already had encryption/decryptioin example working for full payload, so here I will just provide tricks on how to configure OWSM for field level encryption.

It is basically doing XPATH encryptiong, as described in http://download.oracle.com/docs/cd/E10291_01/doc.1013/e10299/policy_steps.htm#sthref612.

Step 1.

I created a BPEL process called DemoOWSMFieldLevelEncryption which pretty much returns the input string. Here is how it looks like:

I have input payload with SSN number in it, which I am interested in encrypting:

Please note down the namespace, it is used when we encrypt the message.

Step 2:

Just for testing purpose I registered that service in OWSM gateway, and start creating policies for the service as shown below:

If we look at the XML encrypt in more detail :

Here I am using existing utility to create JKS files. Interesting thing to note down is:

Encrypted Content: XPATH

Encrypt XPATH: /soap:Envelope/soap:Body/ns1:DemoFieldLevelEncryptionProcessRequest/ns1:SSN
Encrypt namespaces: soap=http://schemas.xmlsoap.org/soap/envelope/,ns1=http://xmlns.oracle.com/DemoFieldLevelEncryption

As you can see, I am using soap and ns1 namespaces in my XPATH, so I have to define them in namespaces section as comma seperated values.

If we look into XML decrypt, it remains pretty much the same. XML encrypt of body/header/envelope or xpath doesn't change XML decrption part.

I created LOG before and after each policy step as part of best practices.

Step 3:

Now time for testing. I used OWSM test page to test my registered service and used Execution Logs to check if messages are getting encrypted and then decrypted back to the original content. Here is what I saw:

First log (SSN is encrypted)

Second Log: SSN is decrypted back to the original value

It seems like it is encrypting and decrypting field level variables. Source code can be downloaded at here.

BPEL Client API

2007-12-31T17:48:00.000-08:00

I had chance to work on both BPEL and ESB client APIs, and I found very little documentation on BPEL Client API, that made me write this BLOG entry.

BPEL Client API:

Everything we can do in BPELConsole, we can do through BPEL client API as well. BPEL client API is JAR file shipped with SOASuite, I imporeted that JAR file and I could manage my BPEL domain using stand alone Java program which calls this client APIs. Here is the usecase we had to implement: Activating and Deactivating a BPEL process using stand-alone Java program or Servlet.

Here is what I started with:

Here we can see the establishConnection, which takes oc4j server name, port, username and port to connect with BPEL server. I have couple of test service to list all domains, list all services in a domain. I have high-lighted startService and endService which has logic for activating and deactivating the service.

I had to include all libraries from, bpel/lib, opmn/lib and j2ee/home/lib. Once I included these libraries, I was able to run this Java program and activate/deactivate the process successfully.

BPEL Client API source code can be downloaded from here.

ESB Client API:

I found ESB Client API are extreamly well explained and documented, so I avoided repeating that information.

Oracle BPEL Header Interceptor

2007-12-31T15:38:00.000-08:00

I saw that BPEL interceptor has been only documented as header handler, I found it can do much more than just handling Headers. It is perfect interceptor and works just like Servlet Filter. It is sensitive part of code, because interceptor is called synchronously from main BPEL process. Any uncatched exception or time consuming delays in interceptor code will be propagated to main BPEL process.

OK, let's look at how to implement it:

We can register two different interceptor, one for request and one for response. I pretty much copied the initial part from documentation and enhance it for 10.1.3.x SOASuite. I created CustomHeaderHandler which implements HeaderHandler, and then I created two different classes InboundHandler and OutboundHandler which extends CustomHeaderHandler.

CustomHeaderHandler:

Here we can see invoke operation with Parternlink, Header, CallProperties. I have also written dumpProperties method which will dump all the properties from Map.

Here is InBound and OutBound interceptor which merely just calls CustomHedaerHandler's invoke method:

InboundHandler

OutboundHandler

I compiled all three classes and put it in bpel/system/classes directory and restarted SOASuite. I created couple of BPEL Processes to test my interceptors, I put following lines of code in bpel.xml to activate the interceptors:

Now if I look into soasuite/opmn/log/default_group~oc4j_soa~default_group~1.log, I can see the intercepted log:

Source code can be downloaded at here.

It is all about JMX

2007-12-27T17:02:00.000-08:00

It is all about JMX

I worked on JMX stuff on OC4J lately, and found information are scattered at many places. Thought of writing blog to combine information which might be useful.

- Simple MBean registered in Stand-Alone JVM, and monitered using JConsole

- Simple MBean registration to stand-alone OC4J, Stand-Alone OC4J MBean Browser, Stand-Alone OC4J MBean Client

- Simple MBean registration to cluster OC4J, OC4J Cluster MBean Browser, Cluster OC4J MBean Client

- Monitor OC4J MBeans using JConsole

1. Stand-Alone JVM

As per MBean spec, we need to define interface which is suffixed by MBean. I created interface called HelloServiceMBean and implemented called HelloService.

Here getMessage and setMessage will expose my message as an attribute to the MBean and printMessage will be exposed as an operation to MBean. To register this MBean in stand-alone JVM, I wrote another class which register this MBean in JVM and I could manage it using JConsole MBean Browser.

I compiled those classes using JDeveloper and executed through following command line:

cd D:projectsJMXSamplesSampleJMXclasses

java -Dcom.sun.management.jmxremote jmx.trial.SimpleAgent

On second window, I started JConsole using: %JAVA_HOME%jdkbinJConsole, and I can see my MBean with Attributes and Operations. I can invoke any operation but I am not able to change any attribute using JConsole. Not sure why I am not able to change any attribute value using JConsole, it works just fine with OC4J MBean Browser.

2. MBean with Stand-Alone OC4J

There are two options we can try, Registering JAR (static registration) file to OC4J using deployment descriptor, and registering MBean from servlet (dyanmic registration). For current Blog entry, I will use deploying through descriptor.

I used the same project which I used for Stand-Alone JVM. I created 3 artifacts in my JDeveloper project:

- Created EJB-JAR deployment profile for jar file called: helloservicejar.deploy

- Created EAR deployment profile: helloservice.deploy (had dependency on helloservicejar.deploy)

- Created OC4J deployment descriptor called - orien-application.xml

I deployed this ear file in stand-alone OC4J, I can see my application and MBean Browser right in EM console. (We can also register this Jar file in j2ee/config/application.xml file then we don't need to have orien-application.xml and ear file.)

Here I can change attribute values and invoke operation without any issues. I used java client to test MBeans deployed on OC4J.

Here getMBeanConnection method is pretty generic, it will get us connection based on the URL specified. printAllMBeans will iterate through all available MBeans and prints all MBeans. The testStandAlone() method does, invocation of method, get attribute and set attribute.

I had to put following libraries in my classpath in order to execute my client:

J2EE
j2eehomelibjmx_remote_api.jar
j2eehomelibjmxcluster.jar
j2eehomelibjmxri.jar
j2eehomelibadminclient.jar
j2eehomeoc4jclient.jar
opmnlib all six jar files.

3. MBean with Cluster OC4J

Registration as JAR: Registration through JAR in Cluster is pretty much similar as registering in Stand-Alone. I use JDeveloper to deploy my EAR file in Cluster Group. I created a group called default_group and registered all my OC4J instances in that group.

Dynamic Registration: Through servlet or POJO residing in container, I could register only to specific container. It looks like, all instances has to do the same to achieve cluster level MBeans deployment.

For testing purpose, I manually deployed a JAR on OC4J group. Here is little introduction about Cluster MBean:

Cluster MBean is actually a proxy to the OC4J group. Client (Java/Browser) can get hold on Cluster MBean, and pass the specific MBean and it's method/attribute as paraemter. Cluster MBean takes care of invoking those method across cluster. Cluster MBean knows about all OC4J instances through OC4J group configured in EM. For each OC4J group there will be one Cluster MBean proxy.

OC4J Cluster MBean Browser: following are the screen shots which explains how did I use cluster MBean browser for getAttribute, I can use it for setAttribute or invoke operation exactly the same way.

Upon invoking the operation I get following return values.

Now to test through Java code, it is pretty much the same concept:

We can see that I got access to Cluster MBean which is "ias:j2eeType=J2EEServerGroup,name=default_group', and after that I am just invoking method of that proxy same as MBean browser. Here tricky part is method signatures, OC4J MBean browser is able to know all those signature through reflection but in my code I had to provide those signature before hand.

4. Monitor OC4J MBeans using JConsole

Basically by adding few libraries to JConsole, you can use it to browse OC4J MBeans. Here is the thing I used to start my JConsole:

set ORACLE_HOME=d:soasuite

jconsole -J-Dcom.sun.management.jmxremote.ssl=false -J-Dcom.sun.management.jmxremote.authenticate=true -J-Djmx.remote.protocol.provider.pkgs=oracle.oc4j.admin.jmx.remote -J-Djava.class.path=;%JAVA_HOME%libjconsole.jar;%JAVA_HOME%libtools.jar;%ORACLE_HOME%j2eehomelibadminclient.jar;%ORACLE_HOME%j2eehomeoc4jclient.jar;%ORACLE_HOME%j2eehomelibjavax77.jar

Upon the window, I added the same URLs which I used in my Java program, I can use any of the following URLs:

STAND_ALONE_OC4J_URL (service:jmx:rmi://localhost:23791/oc4j)
SOASUITE_HOME_URL (service:jmx:rmi:///opmn://mycomputer:6003/home)
SOASUITE_OC4J_SOA_URL (service:jmx:rmi:///opmn://mycomputer:6003/oc4j_soa)
SOASUITE_CLUSTER_URL (service:jmx:rmi:///opmn://localhost:6003/cluster)

And it works great except one weird exception on DOS prompt.

Source code for all can be downloaded over here.

Insert/Extract SOAP Headers in BPEL

2007-12-23T13:14:00.000-08:00

It has been very common topic to insert and extract SOAP headers with BPEL. Lately, I was working with extracting WSSE headers in BPEL, I had to apply some tricks that made me write this BLOG entry.

Inserting Headers in BPEL Outbound:

I will try to enter three different types of headers:

1) Complex XSD element

2) Simple XSD element

3) WSSE headers - which comes out-of-box with BPEL

- First of all I defined XSD element which I want to insert in SOAP headers, e.g.

Here I have created ComplexCredentialElement as XSD Complex element and and Username and Password with XSD simple type. Here namespace is extremely important, I will insert all elements in SOAP header with "http://xmlns.oracle.com/InsertHeaderVariables" namespace. A target process will need to know namespace while extracting Headers, that's why namespace is very important in this case.

Now, in BPEL process I created variable and assigned those variable with some dummy values. You can see my BPEL process as well as source code in following figure:

Here I created variable of the header elements, and assigned dummy values to them. Now when I invoke the partnerlink, I need to specify those variable as bpelx:inputHeaderVariable, and BPEL PM will take care of inserting them when creating SOAP envelope. I believe if you have to insert some SOAP headers for enterprise wide, you can use OWSM client agent to do so, and if you haven't bought OWSM product then writing custom BPEL partnerlink interceptor would be an awesome choice rather than putting header in each partner link. Stay tuned for my blog on intercepting/hacking BPEL partnerlink!

The last one is to insert some WSSE headers through partnerlink, from security aspect I have separate blog entry for this thread: http://chintanblog.blogspot.com/2007/12/have-been-playing-around-with-ws-basic.html but here I am more interested in inserting and extracting headers using BPEL partnerlink.

I deployed this BPEL process and changed my partnerlink URL port to 1234 to test using obtunnel.bat. I started soasuite/bpel/bin/obtunnel.bat and kick-off the BPEL process, here is the intercepted request payload:

I can see my custom headers as well as wsse headers in my SOAP envelope Header, therefore it is working great!!!

Extracting Headers in BPEL in-bound:

I will try to extract three different types of headers:

1) Complex XSD element

2) Simple XSD element

3) WSSE headers

To extract headers I need to define elements which I need to extract, I created following XSD file which represent my simple and complex XSD header elements.

I also need to define wsse headers. Ideally I should download the XSD defined by OASIS for WSSE (http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd) in my project and create variable of the elements already defined in this XSD. Upon loading this XSD project, I got numourous issues so left that approach and created my own XSD element for reprsenting WSSE security header as shown below.

Now I created three variables and pass them to receive activity. BPEL PM takes care of populating them from SOAP headers...

I just deployed this process and I could see both my custom headers and wsse headers are populated nicely...

Source code can be downloaded from this link.

HTTP and WS-BASIC authentication

2007-12-22T22:09:00.000-08:00

Have been playing around with WS-BASIC authentication and HTTP authentication. I just created HelloBPEL project and secured it using OWSM gateway. I defined following policy steps in OWSM for HTTP basic authentication:

Extract credentials I specified HTTP credentials and for File Authenticate I created username/password file using md5 utilies, here is details of the steps:

To test if my policies are defined fine, I tested using Tools -> test page feature.

Everything works great so far. I tried to test it using BPEL. I provided gateway URL to BPEL partnerlink. I had to specify following additional partnerlink properties so that BPEL passes correct header information for HTTP authentication.

Cool... It worked great.

Similar way for WS-BASIC authentication. I changed Extract Credential step in OWSM policy:

And now to test using BPEL, I had to provide following Partnerlink properties:

You can see in above image that you can use httpBasicUsername and httpBasicPassword as well. I found that httpUsername and httpPassword were supported in 10.1.2 but they are deprecated in 10.1.3 as per http://download-uk.oracle.com/docs/cd/B31017_01/integrate.1013/b28982/security.htm , therefore httpBasicUsername and basicUsername are our only choice for 10.1.3.3 SOASuite.

Source code can be downloaded at this link.

WSIF with EJB 2.0, 2.1 and 3.0

2007-12-19T17:47:00.000-08:00

Although WSIF seems to be hack to plug-in native protocol in web services framework, I absolutely love WSIF.

My understanding of WSIF is standard way of representing service contract (which is WSDL) for any type of service. E.g. if you have servlet, java program, .Net code or JCA operation, you can represent all of them as standard WSDL. Now for WSDL consumer (e.g. BPEL) it is just any other WSDL operation and WSIF provider will know how to interact with this service. My belief is Oracle BPEL uses open source Apache WSIF provider in BPEL engine. I did play with HTTP WSIF Binding and Java Binding. I probably will post on HTTP binding once I figure out how to pass dynamic URL (e.g. calling servlet with dynamic parameters) but now it is time to play with some EJB WSIF.

I created sample EJB 2.0, EJB 2.1 and EJB 3.0 Session Façade using Jdeveloper and deployed to my SOASuite oc4j. I deployed in different container than OC4J_SOA just to make sure Remote interface are being used. I will start with EJB 2.1 because that is the easiest one, EJB 3.0 and EJB 2.0 WSIF binding needs lot of hack and/or custom coding.

Technology Used: JDeveloper 10.1.3.3, Oracle SOASuite 10.1.3.3

EJB 2.1 WSIF: Created a sample EJB 2.1 Session Façade, one business method called demoHelloMethod as shown below:

Once you right click your EJB icon, it let’s you create stand-alone Java client for EJB pointing to Remote Application Server as shown below:

We can see that the client uses opmn URL to connect with remote EJB: “opmn:ormi://localhost:6003:home/demosessionejb21”. I found WSIF doesn’t like this URL so I tested my client with ormi url “ormi://localhost:12402/demosessionejb21”. We probably want to note down some information from this client, e.g.

Provider URL: ormi://localhost:12402/demosessionejb21

Initial Context Factory: oracle.j2ee.rmi.RMIInitialContextFactory

EJB name: DemoSessionEJB21

Now it is time to create WSDL, which can represent this EJB operation. We can Jdeveloper wizard to create such such wsdl. Once we right click on EJB we can see “create J2EE web service” option, it lets you create web-service for your EJB. Remember we are just going to use this to create WSDL but not web-service. I took backup of my project, created web-service, took wsdl and then restored my old project, as I am not interested in web-service. I just followed the wizard as shown below.

In Step 2. Please remember to select RPC Lit otherwise some extra effort will be required on XSD side.

In Step 6, this values I took from EJB Client code which was generated using JDeveloper.

Just click on Finish and Look at generated WSDL file:

We can take this wsif and restore the project from earlier backup. We can see both WSIF and HTTP binding in this WSDL. Now as this is not deployed as web-service, I commented out the http part and also put the partner-link so that BPEL doesn’t create extra ref file. The final WSDL file look something like this:

EJB 2.1 WSIF Testing through BPEL Project

I created sample BPEL project and copied this wsdl file in bpel directory. I created partnerlink and selected the binding wsdl. I am almost done, I just need to pass extra information required by EJB RMI client. Seems like these extra information (e.g. username, password) are not part of WSIF spec, but BPEL supports it very well. I provided these entries in bpel.xml:

Because it is native call, BPEL engine will need all classes for EJB client. I copied the classes from EJB session facade project to bpel/system/classes directory.

Deployed the project … and hulala… things working great.

EJB 2.0 WSIF: It is very well supported by Oracle BPEL PM, but not through JDeveloer. Overuse of Wizards and drag-drop kinda makes us handicapped!! Anyways, time to get hand dirty. Created EJB 2.0 session façade, a sample business method and deployed to Application Server. I also created EJB 2.0 Java Client the same way as described in EJB 2.1.

We can see the “Create J2EE Web Service option”. Upon clicking this option it just create web service code and wsdl file with name called “MyWebService1” without showing any prompt… I double click on MyWebService1 so I could at least change the name of WS to DemoSessionEJB20. The initial WSDL looks something like:

There is no WSIF information at all. I hand coded the part of WSDL as shown below. I copied most of the part from EJB 2.1 WSIF wsdl and just changed the stuff as required.

I created sample BPEL project and imported WSIF WSDL. I also put username and password in bpel.xml as described in EJB 2.1 and copied required EJB classes in bpel/system/classes directory. Deployed the project and things went pretty smooth…

EJB 3.0: Personally I believe EJB 3.0 made life much easier in J2EE development. It is really very nice concept eliminating need of EJB 2.x crap, lot of XMLs (like Spring) and extensive use of annotations. Ok, let’s get back to WSIF part.

I created sample EJB 3.0 Session Façade, Java Client, as EJB 2.x. JDeveloper doesn’t display “Create J2EE webservice” option for EJB 3.0 Session Façade, but there is Annotation to create WSIF bindings.

During the creation of EJB 3.0, please expose them to WebService interfacce as well as shown in above figure. That creates Doc-Lit service without any WSIF binding. I believe we need to do schemac if we stick with Doc-Lit. I changed webservice interface (DemoSessionEJB30WebService.java) as shown below to create RPC Lit service with WSIF bindings using annotations.

Now when I deploy this service using deployment profile, Oracle AS creates very nice WSDL:

I created BPEL project, imported this WSDL. After importing this WSDL file I still had to make couple of changes. I had to add partner link to avoid Ref file, remove soap bindings for safety and I also had to add more parameter in ejb:address e.g. className, jndiName and providerURL. The final WSDL with changes looks something like:

As usual, I provided username and password in bpel.xml and copied required classes in bpel/system/classes directory, and things started rocking!!!

Source code can be downloaded at this link.

I will try to update my log when JDeveloper has support for creating wsif WSDL through wizard for EJB 2.0 and EJB 3.0.

Hosting .zip File on Google/Web

2007-12-17T17:41:00.000-08:00

Nothing interesting, but it took me long time to find a place on web where I can upload all my documents reliably. Google docs didn't let me upload .zip files, so I tried Myspace, WordExpress and plenty of other Blog websites which can let me upload .zip or other .java files which I can point to my Blog. No luck!

E.g. other day I was writing blog on BPEL Read Only Console, but I couldn't find where to put the source code.

Got an idea! I went to Google page maker, it let you create a website and you can upload any files on your webpage..... Done, I created a page called http://chintanjshah.googlepages.com/ and put all my files out there, and it does give individual link for each file e.g. http://chintanjshah.googlepages.com/BPELReadOnly.zip....

Now my files are saved on Web through google (till google doesn't disable this service .... ) :)

BPEL Read Only Console - Role Based Security

2007-12-17T17:01:00.000-08:00

I saw numerous people asking about BPEL read only console, which is not supported out of the box. I guess Oracle might ship this feature in 11g, but here is the hack which will obviously come in mind of any web-developer.

First of all, BPEL Console is just a common J2EE web application which front servlet and couple of JSPs behind the scene. It is deployed in "soasuite/j2ee/oc4j_soa/applications/orabpel/console" By default it is secured by JAZN file based security which you can easily look at "WEB-INF/web.xml".

In nutshell, I just wrote servlet filter and provided non-intrusive role based security to BPEL console. Here are the steps I followed:

Step 1. I wrote standard Servlet Filter with help of java.util.regex.Pattern class.

Note: Current filter code doesn't care about user's role. Every single request will be filtered against filterlist.txt file shown below:

Step 2: I wrote to fake servlet to test this servlet Filter. Now we need to integrate this filter with BPELConsole. We can check in "/soasuite/j2ee/oc4j_soa/config/default-web-site.xml" that BPELConsole is located at "soasuite/j2ee/oc4j_soa/applications/orabpel/console".

a) I put compiled BPELFilter.class file in "soasuite/j2ee/oc4j_soa/applications/orabpel/console/WEB-INF/classes" directory. This is how it looks like: "/soasuite/j2ee/oc4j_soa/applications/orabpel/console/WEB-INF/classes/bpelfilterproject/BPELFilter", where bpelfilterproject is the package name.

b) I changed soasuite/j2ee/oc4j_soa/applications/orabpel/console/WEB-INF/web.xml file to apply this filter settings:

These changes are good enough to make BPEL console read only. Now if we restart SOASuite and click on one of the sensitive URL, e.g. Purge Instances, it should show the page with following message:

Step 3: We are done with basic component, but now it is time to do some Role based security. I had to dig into some JAZN settings in order to make this thing worked. I followed the link http://download.oracle.com/docs/cd/B12314_01/web.904/b10325/configja.htm#1007274 which was very helpful. I realized that BPELConsole is using only one Role called "PUBLIC" which is defined in Web.xml file and not mapped to any of the OC4J Role!!!

So let's follow the best practice which has been defined for ESB console already.
- I need to create two roles called "viewers" and "admins".
- I need to define this role in web.xml and map to oc4j role using orien-web.xml file.
- I need to assign admins to bpeladmin
- I need to assign viewers to guest
- Change the Filter code to do Role based filtering

Then let's do it:

a) Create viewers and admins role from EM as shown below:

b) assign "admins" role to "bpeladmin" user and "viewers" role to "guest" user. (Note: BPELDefaultDomainAdmin is always required to access anything in BPELDomain).

c) Let's change the source code. Here is what I did:

Note: Essentially I am relying on two roles viewers and admins, and if role is viewers and not admins, then I start doing filtering. As we know, bpeladmin can have both viewers and admins role.

d) Final change would be to create these roles in console's web.xml file and and assign those roles to OC4J roles. Here is how "soasuite/j2ee/oc4j_soa/applications/orabpel/console/WEB-INF/web.xml" file looks like:

And to associate these roles with OC4J roles, we need to change /soasuite/j2ee/oc4j_soa/application-deployments/orabpel/console/orien-web.xml file. Here is how orien-web.xml file looks like:

Cool. Now if I login as bpeladmin, everything works great, and when I login as guest it is all read only.

Source code along with Read-Me document can be downloaded from this link.