Problems with helma.Zip

[dawei]

Hello helma-group,

I have a major trouble using helma.Zip:
if I add Data larger than approx. 16MB, the process will stop with a
"java heap space" error.
The adding is a stream function and should not cause that problem, my
system monitor also doesn't recognize any significant memory usage. I
increased the java_option maximum memory in the start.bat anyway,
which, as expected, didn't change anything.
Here is the snippet of my code:

var zip = new helma.Zip (zip_file_name);
zip.addData (upload_file_1.getContent(), file_name_1, 9);
zip.addData (upload_file_2.getContent(), file_name_2, 9);
zip.save(zip_file_name);

upload_file_1/_2 are MimeParts sent by a form. Getting the contents
itself is not the problem, as it's working with smaller sizes. I sent
a large file with helma.Ftp and that was working out OK.
I also used a fixed path-string for the file instead of this
'getContent()' which came to the same result.

Either this is a bug or me handling the Zip wrong. I assume it's the
latter. If you could give advice or have a snippet working for large
files, I would greatly appreciate it! Maybe it's just one more
property-setting?

Thank you very much, have a great day,

dawei

[Hannes Wallnoefer]

This is due to the pretty bad practice of allocating byte arrays as
large as whatever comes our way, which I think happens in a few places
in the modulues and maybe even Helma core. The crazy thing is we
actually have Apache Commons IO bundled with Helma, which provides a
utility copy() method to copy from an input stream to an output stream
[1]

[1] http://commons.apache.org/io/api-release/org/apache/commons/io/IOUtils.html#copy(java.io.InputStream,%20java.io.Writer)

I opened a new bug to change all instances of stream copying in Helma
to use Commons IO: http://helma.org/bugs/show_bug.cgi?id=653

Hannes

[dawei]

Hi,

so until now, files of that size were not tested, right?
As you described it, I guess I should work around the recent
helma.Zip.

Thank you very much for your investigation on that. I'd be happy if
you could post here once the problem is officially solved.

bye, dawei