chacham
unread,Sep 22, 2011, 6:18:58 PM9/22/11Sign in to reply to author
Sign in to forward
You do not have permission to delete messages in this group
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to Spreadsheet::ParseExcel
i am processing multiple text files (testing with a few thousand right
now). the method is basically:
1) open an excel template
2) read format from second row (first row is header, only two rows,
one sheet, 15-30 columns)
3) put data from data file (text) into template. (all different sizes,
a few k to 128k)
4) loop
this is using a lot of memory on the (2GB 2008, running activestate
perl 64) server, it steadily grows to 1.7GB, hangs out there for a
while, then starts giving out of memory errors. it processes ~1000
files before generating errors.
i figured it was the parser instantiation:
$template = Spreadsheet::ParseExcel::SaveParser->new->Parse($_[0]);
IOW, never bothered capturing the parser object itself, and instead
went straight to the file.
So, (and i'm not too experience in perl, so i have what to learn) i
figured that would be created thousands of parser objects for no
reason, so instead i created just one, and reused it:
once: my $parser = Spreadsheet::ParseExcel::SaveParser->new();
reused: $template = $template = $parser->Parse($_[0]);
that didn't seem to help, so i tried once more by undef, making the
open sub:
undef $parser;
$parser = Spreadsheet::ParseExcel::SaveParser->new();
$template = $parser->Parse($_[0]);
that didn't help either.
i am making an assumption that it is the parser using all the memory,
i am guessing i need to help it get destroyed. i have no idea if this
is correct, or even if it is, what to do. please point me in the right
direction.
when perl terminates, the memory is released immediately, and i have
quite a few excel files.