Workflow mailer container services are not runnig in 11.5.9/ Sloaris

2,217 views
Skip to first unread message

farook

unread,
Feb 6, 2010, 12:17:48 AM2/6/10
to Troubleshooting AppsDBA
Dear All,

Workflow mailer container services are not runnig in 11.5.9/ Sloaris
5.9. Please help me to solve this issue


Error codes:


LOG_ID_UNKNOWN : oracle.apps.fnd.cp.gsc.Logger.Logger(String, int) :
Logging to System.out until necessary parameters are retrieved for
Logger to be properly started.
LOG_ID_UNKNOWN :
oracle.apps.fnd.cp.gsm.GSMSvcComponentContainer.initializeStateMachine() :
BEGIN
LOG_ID_UNKNOWN :
oracle.apps.fnd.cp.gsc.SvcComponentContainer.getNewWorkflowContext() :
BEGIN
java.lang.NoClassDefFoundError: javax/servlet/ServletRequest
at
oracle.apps.jtf.cache.ServletSessionManagerWrapper.endStandAloneSession(Ser­
vletSessionManagerWrapper.java:
54)
at
oracle.apps.jtf.cache.CacheWorkerThread.statsWriter(CacheWorkerThread.java:
226)
at
oracle.apps.jtf.cache.CacheWorkerThread.run(CacheWorkerThread.java:
337)
LOG_ID_UNKNOWN :
oracle.apps.fnd.cp.gsc.SvcComponentContainer.getNewWorkflowContext() :
BEGIN
LOG_ID_UNKNOWN :
oracle.apps.fnd.cp.gsc.SvcComponentContainer.loadGlobalParameters() :
BEGIN
LOG_ID_UNKNOWN :
oracle.apps.fnd.cp.gsm.GSMSvcComponentContainer.loadContainerParameters() :
BEGIN
LOG_ID_UNKNOWN :
oracle.apps.fnd.cp.gsm.GSMSvcComponentContainer.loadContainerParameters() :
No GSM service-level parameters were found. Either there really are
none or they didn't come over the GSM queue for some reason
LOG_ID_UNKNOWN :
oracle.apps.fnd.cp.gsc.SvcComponentContainer.start() : Successfully
retrieved global and container parameters
LOG_ID_UNKNOWN :
oracle.apps.fnd.cp.gsc.SvcComponentContainer.validateParameterValues(Proper­
ties) :
BEGIN ({SVC_COMP_MAX_ERROR_COUNT=5, SVC_COMP_MONITOR_LOOP_SLEEP=60,
SVC_CONTAINER_LOOP_SLEEP=10, SVC_CONTAINER_READ_TIMEOUT=10,
SVC_COMP_MONITOR_ONDEMAND_FREQ=300, SVC_CONTAINER_LOG_LEVEL=4,
SVC_WRITE_DIAG_TO_GSM_LOG=Y})
LOG_ID_UNKNOWN :
oracle.apps.fnd.cp.gsc.SvcComponentContainer.start() : Successfully
validated container parameters
LOG_ID_UNKNOWN :
oracle.apps.fnd.cp.gsm.GSMSvcComponentContainer.loadDetails(Connection) :
BEGIN (oracle.jdbc.driver.OracleConnection@1ced8e)
LOG_ID_UNKNOWN :
oracle.apps.fnd.cp.gsc.SvcComponentContainer.start() : Successfully
retrieved container details
LOG_ID_UNKNOWN :
oracle.apps.fnd.cp.gsm.GSMSvcComponentContainer.startLogger() : BEGIN
LOG_ID_UNKNOWN :
oracle.apps.fnd.cp.gsc.SvcComponentContainer.startLogger() : BEGIN
[default implementation]
[Feb 4, 2010 8:13:33 AM GMT+03:00]:1265260413649:Thread[main,5,main]:
0:-1:itspdb:10.2.5.214:-1:-1:EXCEPTION:[SVC-GSM-WFALSNRSVC-1174916 :
oracle.apps.fnd.cp.gsc.SvcComponentContainer.start()]:Successfully
created new component control context
[Feb 4, 2010 8:13:33 AM GMT+03:00]:1265260413654:Thread[main,5,main]:
0:-1:itspdb:10.2.5.214:-1:-1:EXCEPTION:[SVC-GSM-WFALSNRSVC-1174916 :
oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:­
Successfully
asked for BES control connection to be established (asynchronous)
[Feb 4, 2010 8:13:33 AM GMT+03:00]:1265260413661:Thread[main,5,main]:
0:-1:itspdb:10.2.5.214:-1:-1:EXCEPTION:[SVC-GSM-WFALSNRSVC-1174916 :
oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:­
Registered
a subscription for oracle.apps.fnd.cp.gsc.SvcComponent.refresh
[Feb 4, 2010 8:13:33 AM GMT+03:00]:1265260413664:Thread[main,5,main]:
0:-1:itspdb:10.2.5.214:-1:-1:EXCEPTION:[SVC-GSM-WFALSNRSVC-1174916 :
oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:­
Registered
a subscription for oracle.apps.fnd.cp.gsc.SvcComponent.resume
[Feb 4, 2010 8:13:33 AM GMT+03:00]:1265260413665:Thread[main,5,main]:
0:-1:itspdb:10.2.5.214:-1:-1:EXCEPTION:[SVC-GSM-WFALSNRSVC-1174916 :
oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:­
Registered
a subscription for oracle.apps.fnd.cp.gsc.SvcComponent.start
[Feb 4, 2010 8:13:33 AM GMT+03:00]:1265260413665:Thread[main,5,main]:
0:-1:itspdb:10.2.5.214:-1:-1:EXCEPTION:[SVC-GSM-WFALSNRSVC-1174916 :
oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:­
Registered
a subscription for oracle.apps.fnd.cp.gsc.SvcComponent.stop
[Feb 4, 2010 8:13:33 AM GMT+03:00]:1265260413666:Thread[main,5,main]:
0:-1:itspdb:10.2.5.214:-1:-1:EXCEPTION:[SVC-GSM-WFALSNRSVC-1174916 :
oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:­
Registered
a subscription for oracle.apps.fnd.cp.gsc.SvcComponent.suspend
[Feb 4, 2010 8:13:33 AM GMT+03:00]:1265260413666:Thread[main,5,main]:
0:-1:itspdb:10.2.5.214:-1:-1:EXCEPTION:[SVC-GSM-WFALSNRSVC-1174916 :
oracle.apps.fnd.cp.gsc.SvcComponentContainer.startBusinessEventListener()]:­
Have
waited 0 seconds for BES system to establish connection to the
control
queue. Waiting another 2 seconds
Failed to establish Java Business Event System control connection:
databaseId = itspdb_prod, ownerName = APPLSYS, queueName =
WF_CONTROL,
subscriberName = WFBESCL_329375, selector = null, exception =
oracle.apps.fnd.wf.bes.BusinessEventException: A JMS error occurred;
see the linked exception for details., linkedException =
oracle.jms.AQjmsException: ORA-24026: operation failed, queue
APPLSYS.WF_CONTROL has errors
ORA-06512: at "SYS.DBMS_AQADM_SYS", line 4418
ORA-06512: at line 1
ORA-06512: at "SYS.DBMS_AQJMS_INTERNAL", line 110
ORA-06512: at line 1
ORA-06512: at "SYS.DBMS_AQJMS", line 101
ORA-06512: at line 1

Syed Irfan

unread,
Feb 6, 2010, 3:13:29 AM2/6/10
to troubleshoo...@googlegroups.com
Ora-24026 in Workflow Notification Mailer After IMAP/SMTP Change [ID 375986.1]  

  Modified 05-OCT-2008     Type PROBLEM     Status MODERATED  
Please check metalink note id

In this Document
  Symptoms
  Cause
  Solution


This document is being delivered to you via Oracle Support's Rapid Visibility (RaV) process, and therefore has not been subject to an independent technical review.

Applies to:

Oracle Workflow Cartridge - Version: 11.5.9
This problem can occur on any platform.

Symptoms

The orkflow mailer does not run because of errors in the log file

ORA-20002: 3825: Error '-24026 - ORA-24026: operation failed, queue APPLSYS.WF_CONTROL has
errors' encountered during execution of 'WF_RULE.ERROR_RULE' for event
'oracle.apps.fnd.cp.gsc.SvcComponent.start'.
FUNCTION_NAME=WF_EVENT_OJMSTEXT_QH.enqueue()

Cause

Either the queue WF_CONTROL or its data is corrupted, probably due to the change of the IMAP/SMTPserver settings.

Since the WF_CONTROL queue is used by the notification mailer to hold runtime data, and it is not valid, the mailer will not be able to start up

Solution

To solve this problem please proceed as follows:

1. Stop Workflow Agent listeners

2. Run the following:
cd $FND_TOP/patch/115/sql
sqlplus APPS/<passwd> @wfctqrec.sql APPLSYS <passwd>

3. Grant required privileges:
sqlplus apps/passwd @afwfqgnt.sql APPS <passwd> APPLSYS <passwd>

Note: APPLSYS password is the same as APPS'.

4. Start Workflow Agent listeners.

The Workflow mailer should start now

Hint: to make sure the WF_CONTROL queue was created successfully, the Concurrent request 'Workflow Control Queue Cleanup' should run without errors.




--
You received this message because you are subscribed to the Google Groups "Troubleshooting AppsDBA" group.
To post to this group, send email to troubleshoo...@googlegroups.com.
To unsubscribe from this group, send email to troubleshooting-a...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/troubleshooting-appsdba?hl=en.




--
+966-55-9764543

Farook Issudeen

unread,
Feb 7, 2010, 5:23:28 AM2/7/10
to troubleshoo...@googlegroups.com
I don't have wfctqrec.sql script in my $FND_TOP/patch/115/sql. What is my problem?

2010/2/6 Syed Irfan <siis...@gmail.com>

Farook Issudeen

unread,
Feb 7, 2010, 7:23:42 AM2/7/10
to troubleshoo...@googlegroups.com
Dear Syed,
 
It worked for me. Thanks a lot

2010/2/7 Farook Issudeen <farook....@gmail.com>

Syed Irfan

unread,
Feb 8, 2010, 12:35:51 AM2/8/10
to troubleshoo...@googlegroups.com
Dear
 
Please check weather you applied patch 4157741
Also check below note from Metalink.
 
Troubleshooting WF_CONTROL Agent issues [ID 469045.1]  

  Modified 26-JAN-2010     Type TROUBLESHOOTING     Status PUBLISHED  

In this Document
  Purpose
  Last Review Date
  Instructions for the Reader
  Troubleshooting Details
     Common issues with WF_CONTROL Queue  
     1) WF_CONTROL queue cleanup concurrent program not running regulary
     2) Performance due to huge volume of events in WF_CONTROL queue.
     3) Detecting and resolving WF_CONTROL queue corruption
     4) Detecting and Resolving Technology Stack related issues
  References


Applies to:

Oracle Workflow Cartridge - Version: 11.5.9 to 12.0.3
Information in this document applies to any platform.
WF_CONTROL Agent
Checked for relevance on 26-JAN-2010

Purpose

Provide steps to Diagonise and Troubleshoot issues with WF_CONTROL Agent that may be the reason why Workflow Agent Listeners would not start.

Last Review Date

November 29, 2007

Instructions for the Reader

A Troubleshooting Guide is provided to assist in debugging a specific issue. When possible, diagnostic tools are included in the document to assist in troubleshooting.

Troubleshooting Details

WF_CONTROL Agent handles control events for service component containers and service components, as well as other Oracle Applications internal processing.

Oracle Workflow contains a standard Business Event System agent named WF_CONTROL, which is associated with a standard queue that is also named WF_CONTROL. This queue has a payload type of JMS Text message. The WF_CONTROL agent is used for internal processing only, and is not meant for customer use.  You should not place custom event messages on this queue. The Generic Service Component Framework uses WF_CONTROL to handle control events for containers and service components, such as notification mailer or agent listener service components. WF_CONTROL is also used for other Oracle Applications internal processing.

You do not need to schedule propagation for the WF_CONTROL agent, because the middle tier processes that use WF_CONTROL dequeue messages directly from its queue.

Common issues with WF_CONTROL Queue  

  • WF_CONTROL queue cleanup concurrent program not running regulary
  • Performance due huge volume of events in WF_CONTROL queue
  • WF_CONTROL queue corruption
  • Technology Stack setup  

1) WF_CONTROL queue cleanup concurrent program not running regulary

It is required that the dead/inactive subcribers to WF_CONTROL queue be cleaned up regularly. A concurrent program named Workflow Control Queue Cleanup is automatically scheduled to perform this cleanup for you. When a middle tier process for Oracle Applications starts up, it creates a durable JMS subscriber to the queue. Then, when an event message is placed on the queue, a copy of the event message is created for each subscriber to the queue. If a middle tier process dies, however, the corresponding subscriber remains in the database. For more efficient processing, you should ensure that WF_CONTROL is periodically cleaned up by removing the subscribers for any middle tier processes that are no longer active. The Workflow Control Queue Cleanup concurrent program sends an event named oracle.apps.wf.bes.control.ping to check the status of each subscriber to the WF_CONTROL queue. If the corresponding middle tier process is still alive, it sends back a response. The next time the cleanup program runs, it checks whether responses have been received for each ping event sent during the previous run. If no response was received from a particular subscriber, that subscriber is removed. The recommended frequency for performing cleanup is every six to twelve hours. In order to allow enough time for subscribers to respond to the ping event, the minimum wait time between two cleanup runs is thirty minutes. If you run the procedure again less than thirty minutes after the previous run, it will not perform any processing. As per the above, the recommended frequency is 6 to 12 hours. The minimum frequency is 30 min + the time that the cleanup program takes to complete.


2) Performance due to huge volume of events in WF_CONTROL queue.

This was a common problem prior to patchset 11i.FND.H Patch 3262159. This patchset is included by default with Oracle Applications 11.5.10.  The performance issue was caused by cache invalidation feature which was introduced starting 11.5.9.

Starting 11.5.10 or patchset 11i.FND.H, Cache Invalidation feature was completely re-written and it is unlikely that you would encounter this problem in 11.5.10.

If you are on 11i release below 11.5.10 and you are experiencing performance problem suspected to be related to WF_CONTROL queue, then see Note 295016.1 and follow the instructions on this note.


3) Detecting and resolving WF_CONTROL queue corruption


The work of Workflow Control Queue Cleanup concurrent program is to cleanup all dead subscribers from the queue. However, for some unknown reasons, WF_CONTROL queue can get corrupt and the cleanup concurrent program is unable to remove all dead subscribers. The following are some SQLs that can be used to check the status of the queue.  Please note that the results are just examples and does not mean that this is what will be returned from your system. 

SQL> select subscriber_type, count(*) from applsys.aq$_wf_control_s group by
subscriber_type;


SUBSCRIBER_TYPE COUNT(*)
-----------------------  ----------

                                2   4611 
                                4   2
                              65   554 
                                     1

Our main concern would be to ensure that the number of durable subscribers i.e subscribers of type 65 are reflecting the true status of Workflow Service Component Containers and Service Components.   When Workflow Service Component Containers and Service Components are running, we should have at least a minimum count of 1.  A count such as in the example above, may indicate that there are lots of Durables Subscribers in the WF_CONTROL queue of which the middle tier services may no longer be active.

RDBMS AQ imposses a limit of 1024 maximum number of durable subscribers per queue at any given time and hence the importance of keeping the queue off limits.

Running the Workflow Control Queue Cleanup concurrent program should clear these dead subscribers under normal circumstances.  If the cleanup job is ran and the number of durable subscribers is not cleared, it is possible that the queue is corrupt.  The following selects can be used to confirm this:

SQL> select count(*), status from WF_BES_SUBSCRIBER_PINGS group by status;

COUNT(*) STATUS
---------- ------------------------------
554          PINGED
941          REMOVED
100          RESPONDED

When Workflow Control Queue Cleanup concurrent program is submitted, it sends an event named oracle.apps.wf.bes.control.ping to check the status of each subscriber to the WF_CONTROL queue. This status of each ping is recorded in WF_BES_SUBSCRIBER_PINGS table. The status of REMOVED and RESPONDED are good. It is the status of  PINGED that you should be concerned about.

In the above example, we can see that 554 subcribers have been pinged and have not responded which then ties with the number of  durable subscribers returned by the above first query from applsys.AQ$_WF_CONTROL_S  table and its corresponding view applsys.AQ$WF_CONTROL_S below.


SQL> select count(*) from applsys.AQ$WF_CONTROL_S;

COUNT(*)
----------
554

To confirm corruption, try removing one of the PINGED subscribers manually as follows:

SQL> select subscriber_name,status from WF_BES_SUBSCRIBER_PINGS
where status in ('PINGED')

SUBSCRIBER_NAME STATUS
------------------------------ ---------
WFBESCL_13670 PINGED
WFBESCL_13671 PINGED 

Pick one of the subscribers returned by the sql above and try removing:

SQL> declare
begin
dbms_aqadm.remove_subscriber(
queue_name => 'APPLSYS.WF_CONTROL',
subscriber => sys.aq$_agent('<WFBESCL13670', null, null));
end;
/

If removal fails with errors, then WF_CONTROL queue is corrupt and needs to be dropped and recreated.

Before this queue can be dropped and recreated, the Middle Tier needs to be brought down, i.e. all concurrent managers, apache and all services stoped. 

a) Drop and Recreate the queue with admin script wfctqrec.sql which is available in Patch 4157741
   Patch is available on top of base 11.5.9 / Patchset 11i.OWF.G

   SQL> $FND_TOP/patch/115/sql/wfctqrec.sql APPLSYS <APPLSYS_PASSWD> 

b) Create required grants/synonyms for Workflow Advanced Queues using $FND_TOP/patch/115   /sql/afwfqgnt.sql script.

4) Detecting and Resolving Technology Stack related issues


If  Control messages are getting raised successfully and event processing is not erroring but the messages are staying  in READY state in WF_CONTROL queue but not getting dequeued, this may indicate some problem with technology stack (appsborg2.zip) issue.

This sql can be ran to verify if events are getting dequeued from the Control Queue.

select substr(wfc.user_data.GET_STRING_PROPERTY('BES_EVENT_NAME'),1,50) EVENT_NAME,
decode(wfc.state,
0, '0 = Ready',
1, '1 = Delayed',
2, '2 = Retained',
3, '3 = Exception',
to_char(substr(wfc.state,1,12))) State,
count(*) COUNT
from applsys.wf_control wfc
group by wfc.user_data.GET_STRING_PROPERTY('BES_EVENT_NAME'), wfc.state


If the number of READY messages is not decreasing then verify if tech-stack is setup correctly:

a) That $APPL_TOP/admin/adjborg2.txt contents are correct and that aqapi and jmscommon jar files from the AIS_HOME are included. 

   grep jar $APPL_TOP/admin/adjborg2.txt

   The output should include

   /d01/oracle/visora/iAS/rdbms/jlib/aqapi.jar
   /d01/oracle/visora/iAS/rdbms/jlib/jmscommon.jar

b) That appsborg2.zip contains JMS classes. If not and the contents of adjborg2.txt is correct as above, regenerating appsborg2.zip using FORCE option will ensure that it contains the JMS classes. You can verify the contents of appsborg2.zip file using the following command:

   unzip -l $JAVA_TOP/appsborg2.zip | grep ".class" | wc -l

   On a typical autoconfigured enviroment, appsborg2.zip contains 1455 classes or more.

   List the contents of appsborg2.zip to a text file to confirm which classes are included and which ones are missing:

   unzip -l $JAVA_TOP/appsborg2.zip > allclasses.txt


c) That AF_CLASSPATH is set correctly in that appsborg2.zip jar file is in the path:

   ECHO $AF_CLASSPATH should return something as follows:

/usr/java/jdk1.6.0_03/jre/lib/rt.jar:/usr/java/jdk1.6.0_03/lib/dt.jar:/usr/java/jdk1.6.0_03/lib/tools.jar:/usr/java/jdk1.6.0_03/jre/lib/charsets.jar:/d01/oracle/viscomn/java/appsborg2.zip:/d01/oracle/visora/8.0.6/forms60/java:/d01/oracle/viscomn/java

d) Verify that JDK (Java Development Kit) 1.3 is installed on the concurrent processing node, Patch 2356331 has been applied, and you have performed all post-installation steps described in Note 130091.1, Upgrading to JDK 1.3 with Oracle Applications 11i. Ensure that environment variables AF_JRE_TOP and AFJVAPRG are set correctly in the adovars.env.

 

Regard

Irfan

Farook Issudeen

unread,
Feb 10, 2010, 5:29:18 AM2/10/10
to troubleshoo...@googlegroups.com
Dear Irfan,
 
Thanks. It is done

2010/2/8 Syed Irfan <siis...@gmail.com>

Syed Irfan

unread,
Feb 11, 2010, 2:15:35 AM2/11/10
to troubleshoo...@googlegroups.com
thats great

Farook Issudeen

unread,
Feb 13, 2010, 11:40:15 PM2/13/10
to troubleshoo...@googlegroups.com
Dear Irfan,

Iam getting this error often 1 or 2 times in a week. Everytime i have
to cleanup the queue as per the note u sent me. What is the reason for
this?. Any permanent solutions?

2010/2/11, Syed Irfan <siis...@gmail.com>:


> thats great
>
> On Wed, Feb 10, 2010 at 1:29 PM, Farook Issudeen
> <farook....@gmail.com>wrote:
>
>> Dear Irfan,
>>
>> Thanks. It is done
>>
>> 2010/2/8 Syed Irfan <siis...@gmail.com>
>>
>> Dear
>>>

>>> Please check weather you applied patch *4157741*


>>> Also check below note from Metalink.
>>>

>>> *Troubleshooting WF_CONTROL Agent issues [ID 469045.1]*
>>> ------------------------------
>>> *Modified* 26-JAN-2010 *Type* TROUBLESHOOTING
>>> *Status*PUBLISHED
>>>
>>>
>>> *In this Document*
>>>
>>> Purpose<https://mail.google.com/mail/html/compose/static_files/blank_quirks.html#PURPOSE>
>>> Last Review
>>> Date<https://mail.google.com/mail/html/compose/static_files/blank_quirks.html#LASTREVIEW>
>>> *Instructions for the
>>> Reader*<https://mail.google.com/mail/html/compose/static_files/blank_quirks.html#ADDINFO>
>>> Troubleshooting
>>> Details<https://mail.google.com/mail/html/compose/static_files/blank_quirks.html#TRBLSHOOT>


>>> Common issues with WF_CONTROL Queue

>>> <https://mail.google.com/mail/html/compose/static_files/blank_quirks.html#aref1>


>>> 1) WF_CONTROL queue cleanup concurrent program not running regulary

>>> <https://mail.google.com/mail/html/compose/static_files/blank_quirks.html#aref2>


>>> 2) Performance due to huge volume of events in WF_CONTROL queue.

>>> <https://mail.google.com/mail/html/compose/static_files/blank_quirks.html#aref3>


>>> 3) Detecting and resolving WF_CONTROL queue corruption

>>> <https://mail.google.com/mail/html/compose/static_files/blank_quirks.html#aref4>


>>> 4) Detecting and Resolving Technology Stack related issues

>>> <https://mail.google.com/mail/html/compose/static_files/blank_quirks.html#aref5>
>>>
>>> References<https://mail.google.com/mail/html/compose/static_files/blank_quirks.html#REF>
>>> ------------------------------


>>>
>>> Applies to: Oracle Workflow Cartridge - Version: 11.5.9 to 12.0.3
>>> Information in this document applies to any platform.
>>> WF_CONTROL Agent
>>> Checked for relevance on 26-JAN-2010 Purpose
>>>
>>> Provide steps to Diagonise and Troubleshoot issues with WF_CONTROL Agent
>>> that may be the reason why Workflow Agent Listeners would not start.

>>> Last Review DateNovember 29, 2007 *Instructions for the Reader*
>>> *A Troubleshooting Guide is provided to assist in debugging a specific


>>> issue. When possible, diagnostic tools are included in the document to

>>> assist in troubleshooting.*


>>> Troubleshooting Details
>>>
>>> WF_CONTROL Agent handles control events for service component containers
>>> and service components, as well as other Oracle Applications internal
>>> processing.
>>>
>>> Oracle Workflow contains a standard Business Event System agent named
>>> WF_CONTROL, which is associated with a standard queue that is also named
>>> WF_CONTROL. This queue has a payload type of JMS Text message. The
>>> WF_CONTROL agent is used for internal processing only, and is not meant
>>> for
>>> customer use. You should not place custom event messages on this queue.
>>> The
>>> Generic Service Component Framework uses WF_CONTROL to handle control
>>> events
>>> for containers and service components, such as notification mailer or
>>> agent
>>> listener service components. WF_CONTROL is also used for other Oracle
>>> Applications internal processing.
>>>
>>> You do not need to schedule propagation for the WF_CONTROL agent, because
>>> the middle tier processes that use WF_CONTROL dequeue messages directly
>>> from
>>> its queue.
>>> Common issues with WF_CONTROL Queue
>>>

>>> - WF_CONTROL queue cleanup concurrent program not running regulary
>>> - Performance due huge volume of events in WF_CONTROL queue
>>> - WF_CONTROL queue corruption
>>> - Technology Stack setup

>>> 3262159<https://updates.oracle.com/ARULink/PatchDetails/process_form?patch_num=3262159>.


>>> This patchset is included by default with Oracle Applications 11.5.10.
>>> The
>>> performance issue was caused by cache invalidation feature which was
>>> introduced starting 11.5.9.
>>>
>>> Starting 11.5.10 or patchset 11i.FND.H, Cache Invalidation feature was
>>> completely re-written and it is unlikely that you would encounter this
>>> problem in 11.5.10.
>>>
>>> If you are on 11i release below 11.5.10 and you are experiencing
>>> performance problem suspected to be related to WF_CONTROL queue, then see
>>> Note

>>> 295016.1<https://mail.google.com/CSP/main/article?cmd=show&id=295016.1&type=NOT>and

>>> *a)* Drop and Recreate the queue with admin script wfctqrec.sql which is
>>> available in Patch
>>> 4157741<https://updates.oracle.com/ARULink/PatchDetails/process_form?patch_num=4157741>


>>> Patch is available on top of base 11.5.9 / Patchset 11i.OWF.G
>>>
>>> SQL> $FND_TOP/patch/115/sql/wfctqrec.sql APPLSYS <APPLSYS_PASSWD>
>>>

>>> *b)* Create required grants/synonyms for Workflow Advanced Queues


>>> using $FND_TOP/patch/115 /sql/afwfqgnt.sql script.
>>> 4) Detecting and Resolving Technology Stack related issues
>>>
>>>
>>> If Control messages are getting raised successfully and event processing
>>> is not erroring but the messages are staying in READY state in
>>> WF_CONTROL
>>> queue but not getting dequeued, this may indicate some problem with
>>> technology stack (appsborg2.zip) issue.
>>>
>>> This sql can be ran to verify if events are getting dequeued from the
>>> Control Queue.
>>>
>>> select substr(wfc.user_data.GET_STRING_PROPERTY('BES_EVENT_NAME'),1,50)
>>> EVENT_NAME,
>>> decode(wfc.state,
>>> 0, '0 = Ready',
>>> 1, '1 = Delayed',
>>> 2, '2 = Retained',
>>> 3, '3 = Exception',
>>> to_char(substr(wfc.state,1,12))) State,
>>> count(*) COUNT
>>> from applsys.wf_control wfc
>>> group by wfc.user_data.GET_STRING_PROPERTY('BES_EVENT_NAME'), wfc.state
>>>
>>>
>>> If the number of READY messages is not decreasing then verify if
>>> tech-stack is setup correctly:
>>>

>>> *a)* That $APPL_TOP/admin/adjborg2.txt contents are correct and that


>>> aqapi and jmscommon jar files from the AIS_HOME are included.
>>>
>>> grep jar $APPL_TOP/admin/adjborg2.txt
>>>
>>> The output should include
>>>
>>> /d01/oracle/visora/iAS/rdbms/jlib/aqapi.jar
>>> /d01/oracle/visora/iAS/rdbms/jlib/jmscommon.jar
>>>

>>> *b)* That appsborg2.zip contains JMS classes. If not and the contents of


>>> adjborg2.txt is correct as above, regenerating appsborg2.zip using FORCE
>>> option will ensure that it contains the JMS classes. You can verify the
>>> contents of appsborg2.zip file using the following command:
>>>
>>> unzip -l $JAVA_TOP/appsborg2.zip | grep ".class" | wc -l
>>>
>>> On a typical autoconfigured enviroment, appsborg2.zip contains 1455
>>> classes or more.
>>>
>>> List the contents of appsborg2.zip to a text file to confirm which
>>> classes are included and which ones are missing:
>>>
>>> unzip -l $JAVA_TOP/appsborg2.zip > allclasses.txt
>>>
>>>

>>> *c)* That AF_CLASSPATH is set correctly in that appsborg2.zip jar file is


>>> in the path:
>>>
>>> ECHO $AF_CLASSPATH should return something as follows:
>>>
>>> /usr/java/jdk1.6.0_03/jre/lib/rt.jar:/usr/java/jdk1.6.0_03/lib/dt.jar:/usr/java/jdk1.6.0_03/lib/tools.jar:/usr/java/jdk1.6.0_03/jre/lib/charsets.jar:/d01/oracle/viscomn/java/appsborg2.zip:/d01/oracle/visora/8.0.6/forms60/java:/d01/oracle/viscomn/java
>>>
>>>

>>> *d)* Verify that JDK (Java Development Kit) 1.3 is installed on the
>>> concurrent processing node, Patch
>>> 2356331<https://updates.oracle.com/ARULink/PatchDetails/process_form?patch_num=2356331>has


>>> been applied, and you have performed all post-installation steps
>>> described in Note

>>> 130091.1<https://mail.google.com/CSP/main/article?cmd=show&id=130091.1&type=NOT>,


>>> Upgrading to JDK 1.3 with Oracle Applications 11i. Ensure that
>>> environment
>>> variables AF_JRE_TOP and AFJVAPRG are set correctly in the adovars.env.
>>>
>>>
>>>
>>> Regard
>>>
>>> Irfan
>>>
>>> On Sun, Feb 7, 2010 at 1:23 PM, Farook Issudeen <
>>> farook....@gmail.com> wrote:
>>>
>>>> I don't have wfctqrec.sql script in my $FND_TOP/patch/115/sql. What is
>>>> my problem?
>>>>
>>>> 2010/2/6 Syed Irfan <siis...@gmail.com>
>>>>

>>>>> *Ora-24026 in Workflow Notification Mailer After IMAP/SMTP Change
>>>>> [ID 375986.1]*
>>>>> ------------------------------
>>>>> *Modified* 05-OCT-2008 *Type* PROBLEM *Status* MODERATED


>>>>> Please
>>>>> check metalink note id
>>>>>

>>>>> *In this Document*
>>>>>
>>>>> Symptoms<https://mail.google.com/mail/html/compose/static_files/blank_quirks.html#SYMPTOM>
>>>>>
>>>>> Cause<https://mail.google.com/mail/html/compose/static_files/blank_quirks.html#CAUSE>
>>>>>
>>>>> Solution<https://mail.google.com/mail/html/compose/static_files/blank_quirks.html#FIX>
>>>>> ------------------------------
>>>>>
>>>>> *This document is being delivered to you via Oracle Support's Rapid
>>>>> Visibility<https://mail.google.com/mail/html/compose/static_files/blank_quirks.html#>(RaV)


>>>>> process, and therefore has not been subject to an independent

>>>>> technical review.*


>>>>>
>>>>> Applies to: Oracle Workflow Cartridge - Version: 11.5.9
>>>>> This problem can occur on any platform.

>>>>> SymptomsThe orkflow mailer does not run because of errors in the log


>>>>> file
>>>>>
>>>>> ORA-20002: 3825: Error '-24026 - ORA-24026: operation failed, queue
>>>>> APPLSYS.WF_CONTROL has
>>>>> errors' encountered during execution of 'WF_RULE.ERROR_RULE' for event
>>>>> 'oracle.apps.fnd.cp.gsc.SvcComponent.start'.

>>>>> FUNCTION_NAME=WF_EVENT_OJMSTEXT_QH.enqueue() CauseEither the queue


>>>>> WF_CONTROL or its data is corrupted, probably due to the change of the
>>>>> IMAP/SMTPserver settings.
>>>>>
>>>>> Since the WF_CONTROL queue is used by the notification mailer to hold
>>>>> runtime data, and it is not valid, the mailer will not be able to start
>>>>> up

>>>>> SolutionTo solve this problem please proceed as follows:

>>>>>> troubleshooting-a...@googlegroups.com<troubleshooting-appsdba%2Bunsu...@googlegroups.com>


>>>>>> .
>>>>>> For more options, visit this group at
>>>>>> http://groups.google.com/group/troubleshooting-appsdba?hl=en.
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> +966-55-9764543
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Troubleshooting AppsDBA" group.
>>>>> To post to this group, send email to
>>>>> troubleshoo...@googlegroups.com.
>>>>> To unsubscribe from this group, send email to

>>>>> troubleshooting-a...@googlegroups.com<troubleshooting-appsdba%2Bunsu...@googlegroups.com>


>>>>> .
>>>>> For more options, visit this group at
>>>>> http://groups.google.com/group/troubleshooting-appsdba?hl=en.
>>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups
>>>> "Troubleshooting AppsDBA" group.
>>>> To post to this group, send email to
>>>> troubleshoo...@googlegroups.com.
>>>> To unsubscribe from this group, send email to

>>>> troubleshooting-a...@googlegroups.com<troubleshooting-appsdba%2Bunsu...@googlegroups.com>


>>>> .
>>>> For more options, visit this group at
>>>> http://groups.google.com/group/troubleshooting-appsdba?hl=en.
>>>>
>>>
>>>
>>>
>>> --
>>> +966-55-9764543
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "Troubleshooting AppsDBA" group.
>>> To post to this group, send email to
>>> troubleshoo...@googlegroups.com.
>>> To unsubscribe from this group, send email to

>>> troubleshooting-a...@googlegroups.com<troubleshooting-appsdba%2Bunsu...@googlegroups.com>


>>> .
>>> For more options, visit this group at
>>> http://groups.google.com/group/troubleshooting-appsdba?hl=en.
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Troubleshooting AppsDBA" group.
>> To post to this group, send email to
>> troubleshoo...@googlegroups.com.
>> To unsubscribe from this group, send email to

>> troubleshooting-a...@googlegroups.com<troubleshooting-appsdba%2Bunsu...@googlegroups.com>

Syed Irfan

unread,
Feb 14, 2010, 1:32:22 AM2/14/10
to troubleshoo...@googlegroups.com

Dear please check below concurrent program is running on Production.
 
(1) Workflow Background Process
every 10 min with below parameter
Process Deferred No
Process Timeout Yes
Process Stuck No
(2) Workflow Control Queue Cleanup
Run every 3 hours "without any parametrer"
(3) Workflow Mailer Statistics Concurrent Program
Run every day "without any parametrer"
(4) Workflow Work Items Statistics Concurrent Program
Run every day "without any parametrer"

(5) Workflow Agent Activity Statistics Concurrent Program
Run every day "without any parametrer"
Workflow Background Process
every 3 days with below parameter
Process Deferred No
Process Timeout No
Process Stuck Yes
 
Regards
Irfan
Reply all
Reply to author
Forward
0 new messages