We have a WebSphere app we have developed where we have a servlet that downloads files, such as zip files or PDF's to the user. This works great when using the built-in WAS HTTP transport.
In our production site, the HTTP server is Apache. The servlet works for small files, but fails for anything over 10-16 MB, do not know the exact amount. After pulling our hair out on this all week, tweaking our code to read/write the file different ways etc., we finally got the idea of hitting the WAS transport directly, which is how we discovered that it works fine in that mode.
Could there be any settings in the Apache/WebSphere configuration that could effect this? Is it a known bug? We were about to go live with this application and this has turned into a major setback. While we do see stack traces in stdout, I do not think they are necessarily meaningful. I am not sure how to look for errors in Apache.
The site is also using SSL, but we tried SSL on the WAS transport and it worked fine.
Just as an update, I tried using the Original HTTP Server instead of Apache and have the same problem.
What happens on the downloads that fail is that almost immediately upon clicking the link that invokes the servlet I get a "Server Error", which I believe is error code 500.
So, based on what I have tested, the problem has to lie in the plugin.
OK, this problem just gets worse as I look for a solution.
Now, if I take my servlet out of the equation and just put the file I want to download in a path that ican be straight-served by my web app, I still get the same problem. In other words, even the built in WAS file-serving servlet cannot download the big files when communicating over the plugin. Nothing shows up in stdout or stderr in this scenario. The browser just gives an immediate server error. Same test with a smaller file works fine.
All works fine when talking directly to WAS HTTP transport.
By the way, I cannot use the WAS transport because it has SSL problems with IE that have not been resolved.
Yup, I finally got around to trying this out myself, I get the same failure as you describe when trying to download a large .zip file. I suggest reporting this problem to IBM support. In the meantime, you might try circumventing the problem by having your servlet stat'ing files and when they are >16MB read the file and write it back to the browser in pieces <16MB each (issue a response.flushBuffer() between writes).
Ma...@softlanding.com wrote: > OK, this problem just gets worse as I look for a solution.
> Now, if I take my servlet out of the equation and just put the file I want > to download in a path that ican be straight-served by my web app, I still > get the same problem. In other words, even the built in WAS file-serving > servlet cannot download the big files when communicating over the plugin. > Nothing shows up in stdout or stderr in this scenario. The browser just > gives an immediate server error. Same test with a smaller file works > fine.
> All works fine when talking directly to WAS HTTP transport.
> By the way, I cannot use the WAS transport because it has SSL problems > with IE that have not been resolved.
Thanks. Since you reproduced it, can you report it? If not, I guess I can post something via PartnerWorld.
Given that this fails instantly, do you really think rewriting the servlet would make a difference? I do not even think the servlet gets a chance to send any data. If you do think it would make a difference could you give me more information about what you think I should do? Are you saying to just flush the output stream before I get to 16MB? I already know the file size.
The way that I have worked around the problem, which while not ideal at least works is to do the following:
When a file over a certain size is to be download I create a unique session folder under my HTTP server, copy the file to that folder and then send a redirect to the browser. I then add an object to the session that implements HttpSessionBindingListener to delete the file and folder when the session expires. This works, but there is a slight sever-side delay while the file is created. One advantage is that I can make the download happens over a non-SSL connection.
By the way, the same problem appears to happen when posting/uploading large files. I am writing a small document management system so the user can upload files that I store in a database. I then make the files available for download.
The failure was near immediate just as you say, so I can understand why you'd think it never reached the app server. But there are indications in the plugin trace, right before the failue, that it knows the size of the large file being returned, which means it must have reached the app server. Yes, doing the flush is what I had in mind. If using the file-serving servlet to serve the large file, you wont have control to issue the flush somewhere before the 16M point. Sounds like you have your own servlet doing the download so it maybe it will be possible in you situation. I'll see how far I can get contacting IBM.
> Thanks. Since you reproduced it, can you report it? If not, I guess I > can post something via PartnerWorld.
> Given that this fails instantly, do you really think rewriting the servlet > would make a difference? I do not even think the servlet gets a chance to > send any data. If you do think it would make a difference could you give > me more information about what you think I should do? Are you saying to > just flush the output stream before I get to 16MB? I already know the > file size.
> The way that I have worked around the problem, which while not ideal at > least works is to do the following:
> When a file over a certain size is to be download I create a unique > session folder under my HTTP server, copy the file to that folder and then > send a redirect to the browser. I then add an object to the session that > implements HttpSessionBindingListener to delete the file and folder when > the session expires. This works, but there is a slight sever-side delay > while the file is created. One advantage is that I can make the download > happens over a non-SSL connection.
> By the way, the same problem appears to happen when posting/uploading > large files. I am writing a small document management system so the user > can upload files that I store in a database. I then make the files > available for download.
I am encountering the same problem. We have written a java app which is registered within WebSphere. The application deals with uploading files. When uploading a small .pdf, everything is fine. It's when I use a file bigger than 16Mb that I encounter the problems. I get the same symptoms both of you have discussed in this thread. Did we find any answers to this problem? Please let me know.
Art Smet <s...@us.ibm.com> wrote in message <news:3D3EC3BE.41C4416@us.ibm.com>... > The failure was near immediate just as you say, so I can understand why you'd > think it never reached the app server. But there are indications in the > plugin trace, right before the failue, that it knows the size of the large > file being returned, which means it must have reached the app server. Yes, > doing the flush is what I had in mind. If using the file-serving servlet to > serve the large file, you wont have control to issue the flush somewhere > before the 16M point. Sounds like you have your own servlet doing the > download so it maybe it will be possible in you situation. I'll see how far I > can get contacting IBM.
> > Thanks. Since you reproduced it, can you report it? If not, I guess I > > can post something via PartnerWorld.
> > Given that this fails instantly, do you really think rewriting the servlet > > would make a difference? I do not even think the servlet gets a chance to > > send any data. If you do think it would make a difference could you give > > me more information about what you think I should do? Are you saying to > > just flush the output stream before I get to 16MB? I already know the > > file size.
> > The way that I have worked around the problem, which while not ideal at > > least works is to do the following:
> > When a file over a certain size is to be download I create a unique > > session folder under my HTTP server, copy the file to that folder and then > > send a redirect to the browser. I then add an object to the session that > > implements HttpSessionBindingListener to delete the file and folder when > > the session expires. This works, but there is a slight sever-side delay > > while the file is created. One advantage is that I can make the download > > happens over a non-SSL connection.
> > By the way, the same problem appears to happen when posting/uploading > > large files. I am writing a small document management system so the user > > can upload files that I store in a database. I then make the files > > available for download.
> The failure was near immediate just as you say, so I can understand why you'd > think it never reached the app server. But there are indications in the > plugin trace, right before the failue, that it knows the size of the large > file being returned, which means it must have reached the app server. Yes, > doing the flush is what I had in mind. If using the file-serving servlet to > serve the large file, you wont have control to issue the flush somewhere > before the 16M point. Sounds like you have your own servlet doing the > download so it maybe it will be possible in you situation. I'll see how far I > can get contacting IBM.
> > Thanks. Since you reproduced it, can you report it? If not, I guess I > > can post something via PartnerWorld.
> > Given that this fails instantly, do you really think rewriting the servlet > > would make a difference? I do not even think the servlet gets a chance to > > send any data. If you do think it would make a difference could you give > > me more information about what you think I should do? Are you saying to > > just flush the output stream before I get to 16MB? I already know the > > file size.
> > The way that I have worked around the problem, which while not ideal at > > least works is to do the following:
> > When a file over a certain size is to be download I create a unique > > session folder under my HTTP server, copy the file to that folder and then > > send a redirect to the browser. I then add an object to the session that > > implements HttpSessionBindingListener to delete the file and folder when > > the session expires. This works, but there is a slight sever-side delay > > while the file is created. One advantage is that I can make the download > > happens over a non-SSL connection.
> > By the way, the same problem appears to happen when posting/uploading > > large files. I am writing a small document management system so the user > > can upload files that I store in a database. I then make the files > > available for download.
No, I never got any respose. Since I had developed a workaround I have not pressed the issue further.
If you have a WebSphere support contract I would recommend reporting the problem to IBM via official channels. It is very simply to duplicate so it shouldn't be difficult to get a problem opened.
I called the support line yesterday and opened up a PMR about this. I am currently waiting to see what they tell me, but the person I am dealing with has noted that he has queried Art S. and is awaiting a response.
> No, I never got any respose. Since I had developed a workaround I have > not pressed the issue further.
> If you have a WebSphere support contract I would recommend reporting the > problem to IBM via official channels. It is very simply to duplicate so > it shouldn't be difficult to get a problem opened.
The file download problem that you are seeing is fixed in WAS 4.0.4, so please get the latest ptf to fix this problem. In addition, the file upload problem has also been fixed, but due to java object size restrictions at V4R5, such large uploads will only work if you move to V5R1. In order to make them work you will also have to tweak your plugin-cfg.xml file. The tweak is present to prevent potential DOS attacks on the HTTP server.
Open your plugin-cfg.xml file in an editor, and add the attribute PostSizeLimit to all applicable server groups. Set the attribute to the mzximum value in bytes that you would like to be able to upload. For example:
> Thanks. Since you reproduced it, can you report it? If not, I guess I > can post something via PartnerWorld.
> Given that this fails instantly, do you really think rewriting the servlet > would make a difference? I do not even think the servlet gets a chance to > send any data. If you do think it would make a difference could you give > me more information about what you think I should do? Are you saying to > just flush the output stream before I get to 16MB? I already know the > file size.
> The way that I have worked around the problem, which while not ideal at > least works is to do the following:
> When a file over a certain size is to be download I create a unique > session folder under my HTTP server, copy the file to that folder and then > send a redirect to the browser. I then add an object to the session that > implements HttpSessionBindingListener to delete the file and folder when > the session expires. This works, but there is a slight sever-side delay > while the file is created. One advantage is that I can make the download > happens over a non-SSL connection.
> By the way, the same problem appears to happen when posting/uploading > large files. I am writing a small document management system so the user > can upload files that I store in a database. I then make the files > available for download.
> The file download problem that you are seeing is fixed in WAS 4.0.4, so please > get the latest ptf to fix this problem. In addition, the file upload problem > has also been fixed, but due to java object size restrictions at V4R5, such > large uploads will only work if you move to V5R1. In order to make them work > you will also have to tweak your plugin-cfg.xml file. The tweak is present to > prevent potential DOS attacks on the HTTP server.
> Open your plugin-cfg.xml file in an editor, and add the attribute > PostSizeLimit to all applicable server groups. Set the attribute to the > mzximum value in bytes that you would like to be able to upload. For example:
> > Thanks. Since you reproduced it, can you report it? If not, I guess I > > can post something via PartnerWorld.
> > Given that this fails instantly, do you really think rewriting the servlet > > would make a difference? I do not even think the servlet gets a chance to > > send any data. If you do think it would make a difference could you give > > me more information about what you think I should do? Are you saying to > > just flush the output stream before I get to 16MB? I already know the > > file size.
> > The way that I have worked around the problem, which while not ideal at > > least works is to do the following:
> > When a file over a certain size is to be download I create a unique > > session folder under my HTTP server, copy the file to that folder and then > > send a redirect to the browser. I then add an object to the session that > > implements HttpSessionBindingListener to delete the file and folder when > > the session expires. This works, but there is a slight sever-side delay > > while the file is created. One advantage is that I can make the download > > happens over a non-SSL connection.
> > By the way, the same problem appears to happen when posting/uploading > > large files. I am writing a small document management system so the user > > can upload files that I store in a database. I then make the files > > available for download.
Here is the latest reply I have gotten from the support line . Bob, Here is the word from the developer about WAS 3.5 Download does work at v3.56 across all servers & OS releases. Upload only works for apache http server at v5r1. The reasons the other uploads fails is due to non-WebSphere reasons (original http server cant handle requests 16MB+), nor can the JDK objects be greater than 16MB. . So, there is nothing to fix with 3.5. The problems are software restrictions.
This statement proves to be wrong as we are on V5R1 running WAS 3.5.3 on the Apache http server and we still have the upload barrier of 16 megs.
> We have a WebSphere app we have developed where we have a servlet that > downloads files, such as zip files or PDF's to the user. This works great > when using the built-in WAS HTTP transport.
> In our production site, the HTTP server is Apache. The servlet works for > small files, but fails for anything over 10-16 MB, do not know the exact > amount. After pulling our hair out on this all week, tweaking our code to > read/write the file different ways etc., we finally got the idea of > hitting the WAS transport directly, which is how we discovered that it > works fine in that mode.
> Could there be any settings in the Apache/WebSphere configuration that > could effect this? Is it a known bug? We were about to go live with this > application and this has turned into a major setback. While we do see > stack traces in stdout, I do not think they are necessarily meaningful. I > am not sure how to look for errors in Apache.
> The site is also using SSL, but we tried SSL on the WAS transport and it > worked fine.
> The problems happen in both IE 6 and Mozilla 1.0.
Bob, the upload was meant to apply to WAS v3.56 as well as the download statement. I'm not sure why it is failing at WAS v3.53, but that level is over a year old and it's possible the limitation got fixed somewhere along the way to WAS v3.56 or perhaps in the latest level of the apache http server itself, which WAS plugs into. Are you able to move up to WAS v3.56, if the upload still fails you should contact IBM support.
Bob Kittelson wrote: > Here is the latest reply I have gotten from the support line > . > Bob, > Here is the word from the developer about WAS 3.5 > Download does work at v3.56 across all servers & OS releases. Upload > only works for apache http server at v5r1. The reasons the other > uploads fails is due to non-WebSphere reasons (original http server cant > handle requests 16MB+), nor can the JDK objects be greater than 16MB. > . > So, there is nothing to fix with 3.5. The problems are software > restrictions.
> This statement proves to be wrong as we are on V5R1 running WAS 3.5.3 on the > Apache http server and we still have the upload barrier of 16 megs.
> <Ma...@softlanding.com> wrote in message > news:ah78f2$etk$1@news.boulder.ibm.com... > > WebSphere 4.03 AE > > OS/400 V4R5M0 > > Apache - latest Group PTF's (including recent CERT fix).
> > We have a WebSphere app we have developed where we have a servlet that > > downloads files, such as zip files or PDF's to the user. This works great > > when using the built-in WAS HTTP transport.
> > In our production site, the HTTP server is Apache. The servlet works for > > small files, but fails for anything over 10-16 MB, do not know the exact > > amount. After pulling our hair out on this all week, tweaking our code to > > read/write the file different ways etc., we finally got the idea of > > hitting the WAS transport directly, which is how we discovered that it > > works fine in that mode.
> > Could there be any settings in the Apache/WebSphere configuration that > > could effect this? Is it a known bug? We were about to go live with this > > application and this has turned into a major setback. While we do see > > stack traces in stdout, I do not think they are necessarily meaningful. I > > am not sure how to look for errors in Apache.
> > The site is also using SSL, but we tried SSL on the WAS transport and it > > worked fine.
> > The problems happen in both IE 6 and Mozilla 1.0.
