Slow app deployment with RAD 7,5 and WAS 7.0.1

[g...@miami.edu]

Has anyone ever found a clear cause/solution to this problem? My shop is accustomed to deploy times measured in seconds with earlier versions of these products, but now see deployment times of several minutes for quite small applications. Among other things, we've tried:
- unselecting the "Deploy J2EE Modules" publishing option
- setting the "Run Server with resources on Server" option
- disabling RMI and using SOAP
- setting osgi.bundlefile.limit=100 (to try and eliminate zip file errors)
- disabling automatic publishing
- other solutions suggested here and elsewhere.

Any suggestions are most welcome.

Thanks!

G. Phillips
g...@miami.edu

[Ben_]

Tried applying latest fixes already ?

[Brian S Paskin]

Hi, Please download FP3 for 7.0.0.0 and see if it fixes the issue.

http://www-01.ibm.com/support/docview.wss?rs=180&uid=swg27014463

Regards,
Brian

[philipp...@de.ibm.com]

I've opened a PMR for the same symptoms two months ago. It is still under investigation. I see the problem with a newly developed app and with an app already in production on V6.1. It worked perfectly until we put it on V7. My guess is that this is due to heavy "annotation processing" which is new in JEE5. I just downloaded FP3 hoping that the issue might be fixed.
I will keep you up2date...

[Davanum Srinivas (dims)]

Can you please try switching off by setting UseWSFEP61ScanPolicy to false - http://www.google.com/search?hl=en&q=UseWSFEP61ScanPolicy

thanks,
dims

[Brian S Paskin]

Hi, this issue has been resolved and should be part of FP3.

Regards,
Brian

[philipp...@de.ibm.com]

Sorry, neither of both suggestions worked for me. 7.0.0.3 is as slow as before:-( Already have a PMR open on this since a while..

[stev_m...@hotmail.com]

I have this same problem with a project migrated from V6. I tried a succession of RAD 7.5 fixes (7.5/7.5.1/7.5.2) and WTE 7.0 fixes (7.0.0.0/7.0.0.1/7.0.0.3)

The app seems to publish ok and quickly when global security is turned off. However once global security is turned on (Using an LDAP repository) the publishing never seems to do a smart publish.

In my case I'm simply starting the app server and then adding a space to a comment on a class, the app is never invoked on the browser. The workspace builds and then the app publish kicks in, this completes quickly, however a second app publish appears to trigger which sits at 0% for a long time, before going to 50% and sitting for a long time and finally hits 100% and sits there for a long time before the publish completes. At this point the whole app stops and restarts. Under 6.0 app server the whole publish takes 5 seconds after workspace build and does not restart the app. It looks to me as though the app server is waiting for something and times out and then continues with the publish process and then restarts the app. I understand that a app restart is required under certain conditions. However this does not explain the long pauses before all this happens

Is there a permission that the server userid or group requires that I've missed, or is there something we need to enable once global security is turned on ?

I've tried publish with resources in workspace( both minimize resources turned on & off) which takes 5 minutes and publish with resources on the server. The only difference is that the publish with resources on server exports an ear and then hits this same delay, and hence takes even longer.

[stev_m...@hotmail.com]

OK just discovered that I can shorten the publish to about 1 minute down from 5 minutes. It appears some new features have been added to app server security called Security Domains. It appears that you have to create a security domain for the either the node or server. All I did was select Security Domains and click the new button and in my case ensure each of the settings was the same as global security (I guess this allows you to customize security and override global security setting per node/cluster and server. Without the security domain the publish will pause and wait i.e 0%....50%... 100%.

However my app is still restarting for a relatively simple change. I'm not sure why this is happening but I imagine if I could get this solved the app publish times would be comparable to V6.0

[Albert Collet]

Hi,
I got exactly the same problem.
Products: RAD 7.5.2 and WAS 7.0.0.3 with Portal Server 6.1.0.1.

When publishing the times are:
0% for a long time, 50% for a long time and finally 100%.

Connection made via SOAP protocol.

And also there is another issue, the remote console view doesn't display and in SystemOut.log there are lots of messages like:
"Caller is not in the required role to access restricted document(s)"
among other messages related to the same cause.

Also I saw that installing manually (via web console) a WAR app there are some problems related to schema documents that cannot be found (*.xsd or *.xsl). I could check that these problems dissapeared if I give access the WAS to reach the internet. It seems that the XML Catalog problem has not been correctly fixed in WAS 7.0.0 fixpack 3.
Take a look into: http://www-01.ibm.com/support/docview.wss?rs=180&uid=swg1PK76017

Albert.-

[Davanum Srinivas (dims)]

Albert,

PK76017 adds xsd's/wsdl's only for the WSN use case. Please switch on "org.apache.axis2.*=all" and "org.apache.xml.resolver.*=all" in the trace settings and look at which xsd(s)/wsdl(s) are being loaded from the internet. Once you know which ones are being picked up, you can add a jax-ws-catalog.xml with the ones that are being picked up from the internet.

If you do see specific ones that you think should be in WAS itself, please raise a support/enhancement request via normal channels.

thanks,
dims

[1] http://www.google.com/#q=jax-ws-catalog.xml

[Albert Collet]

Hi dims, thanks for your help.
The schema that the server is trying to locate is http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd

When I publish my portlet through RAD 7.5.2 there's no problem (it takes long time) but the server never accesses to the Internet to locate that schema.

If I publish the portlet (WAR file) using the WebSphere Portal Administration web console (not the Integrated Solutions Console WAS 7.0.0.3) I get this error:

org.xml.sax.SAXParseException: schema_reference.4: anomalía al leer el documento de esquema 'http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd', porque 1) no se ha podido encontrar el documento; 2) no se ha podido leer el documento; 3) el elemento raíz del documento no es .
at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
at org.apache.xerces.util.ErrorHandlerWrapper.warning(Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
at org.apache.xerces.impl.xs.traversers.XSDHandler.reportSchemaWarning(Unknown Source)
at org.apache.xerces.impl.xs.traversers.XSDHandler.getSchemaDocument(Unknown Source)
at org.apache.xerces.impl.xs.traversers.XSDHandler.parseSchema(Unknown Source)
at org.apache.xerces.impl.xs.XMLSchemaLoader.loadSchema(Unknown Source)
at org.apache.xerces.impl.xs.XMLSchemaValidator.findSchemaGrammar(Unknown Source)

The previous error disappears if I give access the server to reach the Internet.
In the firewall log I get an HTTP entry to java.sun.com so the server really tries to locate that schema in the Internet.

For the moment this is not a problem because I publish using RAD and I have given access the server to the internet but I was wondering if this could be the cause of this high publishing times.

One more thing, I couldn't find jax-ws-catalog.xml file in the server filesystem. I don't know how IBM has implemented the XML Catalog but I couldn't find anything. It could be interesting to know how to add a new catalog entry on server side, on client side (RAD) I just know ho to do this via (Window/Preferences/XML/XML Catalog).

thank you very much.
Albert.-

[Davanum Srinivas (dims)]

Albert,

Did you see this technote? that may help you with the slowdown - http://bit.ly/jE7yo

Direct Link : http://www-01.ibm.com/support/docview.wss?rs=180&context=SSEQTP&dc=DB560&dc=DB520&uid=swg21381764&loc=en_US&cs=UTF-8&lang=en&rss=ct180websphere

You may also want to raise a PMR against Portal for web-app_2_5.xsd issue. You can also try removing the xsi:schemaLocation attribute in your web.xml which is causing this lookup at runtime.

thanks,
dims

[Albert Collet]

Hi Dims,

Thanks a lot. I think that this technote may help but such processing time is too high.
On the other hand I have opened a PMR for web-app_2_5.xsd issue.

thanks and regards,
Albert.-

[qixin...@yahoo.com]

I encountered the same issue in RSA 7.0 with WAS6.1 test environment, at that time, i had to do a step below for getting our application can be running in WAS61.

Alter the server settings to ensure that the necessary metadata is copied to the server so that the Admin Console can operate properly.
a.In the Servers view, double-click the server on which the application is being published
b.The Server Overview editor comes up. In the Publishing section, uncheck the option, "Minimize application files copied to the server"
Note: This step is used to solve an issue in RSA7.0, in IBM support website, the topic is “Error ADMA017EE received when associating shared library in Admin Console”. (URL: http://www-01.ibm.com/support/docview.wss?uid=swg21260016 )

Now, i begin to use RSA7.5 with WAS6.1 test environment, and i perform the same step as above, then i still saw the issue of slower publishing. But now i check the option "Minimize application files copied to the server", then the publishing looks very quickly (At least as normal as before), and i restarted the server/RSA several times, and it is stable, for the above issue, maybe i still need to do when i first deploy the application, but at least i can solve the slow publishing issue by change the option later, since it is development environment.

Could you try it? if it is indeed a feasible solution, then it is really good news for us. Thanks

[Albert Collet]

Hi,
well, finally this worked for me, it was hard to find. Thanks to this thread:

http://www.ibm.com/developerworks/forums/thread.jspa?threadID=202353

Check out the doc for PK71818 and run the wsadmin redeployFileTransfer.jacl fileTransferAuthenticationOn task

http://publib.boulder.ibm.com/infocenter/radhelp/v7r5/index.jsp?topic=/com.ibm.ws.ast.st.v6.ui.doc/topics/rredeployscript_v6.html

After doing this task, RAD publishing times are better (not the best but better) and remote console view in RAD works.

Albert.-

[stev_m...@hotmail.com]

Ok

My problems are with RAD 7.5 publishing to a WAS 7.0.1/7.0.3 local server. So in my case 'minimise files published to the server' was already ticked and I'm not sure if the redeployFileTransfer.jacl file is relevant to WAS 7. I think from what i've some people have issues with a JEE 5 app with annotations being published and this appears to be to do with the annotation parsing... however my app is a J2EE 1.4 app.

What I've noticed is a two fold problem.

Firstly RAD7.5 has issues publishing to the V7 server if security is turned on, this is when the publishing pauses ie 0%.....50%.....100%. I found that turning on a security domain the issues seems to disappear and a publish quickly gets to 100%.

Secondly and this issue I've not been able to solve, is that RAD7.5 insists on restarting the application each time a publish happens. I'm aware that this is required under certain circumstances, for example if a class i'm changing is already loaded by the classloader then obviously the app needs restarting to reload the class. However what I've noticed is that it is restarting all the time. To test this out I did the following

1) created a new class not referenced by anything or loaded by the classloader....on saving the app is restarted
2) added some fields and set/getters to the class.... on saving the app is restarted.
3) add comments.... on saving the app is restarted.
4) add a few blank lines.... on saving the app is restarted.... do this while the app is publishing
5) add a few blank lines whilst the app is being published... compile phase is paused till the app is published and then compile phase is restarted once the first publish is finished and another publish and restart occurs.
6) repeated the same tests under V6 and V6.1 local app servers and each time the class is saved the app is not restarted after a publish.

In each case after your compilation and validation phases, the publish phase kicks in, then a serious of messages appear on the console ie User {admin account} modified document blah.xml. In V6 and V6.1 thats then end of it, under V7 after those messages the app is restarted, adding another 30-45 seconds to the publishing.

The end result under V7 is it takes 1-1 1/2 minutes per control-s (save) if you include compile and validation with a running server for your changes to be compiled and hit the local server, and they are serialized too, so if you do a couple of saves you'll be hit with a sequence of these compile...validate and publish....restart... phases

So it looks like you would be quicker to stop your app and do your edits restart the app and republish. this is not very productive.

[sva...@nashen.com]

Thank God I am not alone in this situation!! The application is restarted for whatever change we do in the app, this is really a showstopper, development is slowed down so much by waiting after publishing and restarting that it is becoming ridicoulus.

[alla_s...@rediffmail.com]

Hi,
Whenver i change a java class in a web application, the application needs to be published and restarted. This was taking a lot of time. Are there any ways of overcoming this. Your help will be greatly appreciated.

Regards,
Sridhar

[PeterD...@bc.com]

I just upgraded to the Rad 7.5.3 and was 7.0.0.3 and I'm still having this problem. I've tried everything listed above and had no success. Has anyone had any other success with getting this working?

[valia...@earthlink.net]

At my previous company, I never had this problem (RSA 7.5 with WebSphere 6.0), however at my current company I've had the problem since the start. RAD 7.5 (always the latest version) on WebSphere 6.0 (always latest fixpack). I've racked my brain trying to figure it out, and haven't a clue. It must be something with how our EAR is setup?

[Karol_Nunes]

I found a solution.

The solution works for WAS 7.0.0.0, 7.0.0.3 and 7.0.0.5.

It will be added on WAS 7.0.0.7 release.

WAS7 support team has 3 solution for this issue, so they created a cumulative fix to solve it.

http://www-01.ibm.com/support/docview.wss?uid=swg24024493&myns=swgws&mynp=OCSSEQTP&mync=R

You also have to access these fix individually because each of them has some procedures after installation.

http://www-01.ibm.com/support/docview.wss?rs=180&context=SSEQTP&dc=D400&uid=swg24024386&loc=en_US&cs=UTF-8&lang=en&rss=ct180websphere

http://www-01.ibm.com/support/docview.wss?rs=180&context=SSEQTP&dc=D400&uid=swg24024389&loc=en_US&cs=UTF-8&lang=en&rss=ct180websphere

note: WAS 7.0.0.5 users don't need to download the fix it was already added to it.

In my case, my issue was because annotation scanning on jar files found in my web-project-libraries, so I had to follow the instructions here

after installing the fix:

http://www-01.ibm.com/support/docview.wss?rs=180&uid=swg1PK87053

What I did was create property on manifest.mf with my jars file names added:

---------------------------------------------
Manifest-Version: 1.0
Class-Path:
Ignore-Scanning-Archives: activation-1.1.jar, antlr-2.7.6rc1.jar, asm-hibernate-3.1.3.jar, aspectwerkz-2.0.jar,
aspectwerkz-core-2.0.jar, aspectwerkz-extensions-2.0.jar
---------------------------------------------

note: before adding this property you should install the fix that I mentioned above.

to improve publishing time, also select these publishing options on WAS configuration:

Run Server with resources within the workspace

[Jevgeni Kabanov]

On the same topic, there's a commercial tool from ZeroTurnaround that
makes application updates instant in development even for large
applications:
http://www.zeroturnaround.com/blog/how-to-install-and-use-javarebel-in-ibm-websphere-with-rational-application-developer-or-eclipse/

Jevgeni Kabanov

On Oct 21, 5:21 pm, Karol_Nunes <k-nu...@uol.com.br> wrote:
> I found a solution.
>
> The solution works for WAS 7.0.0.0, 7.0.0.3 and 7.0.0.5.
>
> It will be added on WAS 7.0.0.7 release.
>
> WAS7 support team has 3 solution for this issue, so they created a cumulative fix to solve it.
>

> http://www-01.ibm.com/support/docview.wss?uid=swg24024493&myns=swgws&...

>
> You also have to access these fix individually because each of them has some procedures after installation.
>

> http://www-01.ibm.com/support/docview.wss?rs=180&context=SSEQTP&dc=D4...
>
> http://www-01.ibm.com/support/docview.wss?rs=180&context=SSEQTP&dc=D4...