Creating Large zip files in php

[Arkady Renko]

Gday Guys

I'm attempting to create zip files on the fly for some highly
compressible, yet very large files stored on my Web server. At present
I'm using a class from the Zend library by Eric Mueller which I've
modified to suit my purposes. The problem occurs when I have files of
an arbitrarily large size.. say > 100mb, whereby I end up with high
memory usage. At the moment I'm using a workaround whereby the scripts
may use a very large amount of memory and work on no time limit but
obviously this is something that must not be done in a production
environment.

What I'm hoping is that someone has written a class, or solved this by
'streaming' a zip file as it's being created to the user to avoid high
memory usage during the process. My understanding is that using the
deflate algorithm, you can create sequential 'blocks' of compressed data
which I imagine would play directly into the hands of what I'd want to
achieve

Time is not an issue for my clients, as the 2 minutes they may have to
wait for the script to execute are far better than the extra 200mb they
would have to download for an uncompressed file. (Example, reduction
from 420mb to 78mb), however I would like to be able to handle several
of these operations occurring at once.

Failing that.. looks like I'll have to write something to suit my needs.
--
--------------------
Arkady Renko
Network Admin
To email me directly
Remove all capitals
and underscores from
my posting address

[pete...@gmail.com]

Hi,

You can use command-line zip to do this. For example, to zip a file to
stdout in unix:

zip - file.txt

Here's a function to zip and stream a file on the fly in PHP using the
zip command:

<?

// This function will zip and transmit a file on the
// fly using the zip command.
//
// Usage example:
//
// zipfly('/tmp/data.txt');

function zipfly($path)
{
// Try to disable the browser cache

header('Pragma: public');
header('Expires: 0');
header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
header('Cache-Control: private', false);

// Run the zip command

$zipErrorFile = tempnam('/tmp', 'zipfly-');

$cmd = 'zip - ' . escapeshellarg($path) .
' 2> ' . escapeshellarg($zipErrorFile);

$proc = popen($cmd, 'r') or die();

// Stream the zip data

if (is_resource($proc) && !feof($proc)) {
if (($data = fread($proc, 8192)) !== false && $data !== '') {
header('Content-Transfer-Encoding: binary');
header('Content-Type: application/zip');
$fileName = preg_replace('/\..*?$/', '', basename($path)) .
'.zip';
header('Content-Disposition: attachment; filename="' .
"$fileName\";");
echo $data;
if (!feof($proc)) {
fpassthru($proc);
}
}
}

// Get the exit code of the command

$exitCode = pclose($proc);

if ($exitCode != 0) {
$msg = "Command \"$cmd\" failed with exit code $exitCode";
$errors = is_file($zipErrorFile) ? file_get_contents($zipErrorFile)
: false;
$msg .= $errors !== false ? ": $errors" : '.';
trigger_error($msg);
}
}

?>

Best Regards,

John Peters