Option to split output file into multiple chunks when compressing; can reassemble for decompression

[Frank Hileman]

We have a need to compress a file and break it into chunks no bigger than a certain size. We have been using 7-zip for this which has a "volumes" switch to limit the size of each chunk. Each chunk has a suffix added such as 001, 002, etc. Decompression needs only the name of the first chunk to work. 

Is there any intention to add something similar to the lz4 command line executable?

Here is the documentation for the 7zip switch:

-v (Create Volumes) switch 

Specifies volume sizes.
Syntax-v{Size}[b | k | m | g]
{Size}[b | k | m | g] Specifies volume size in Bytes, Kilobytes (1 Kilobyte = 1024 bytes), Megabytes (1 Megabyte = 1024 Kilobytes) or Gigabytes (1 Gigabyte = 1024 Megabytes). if you specify only {Size}, 7-zip will treat it as bytes. 

Thanks,

Frank

[Yann Collet]

Hello Franck

It is currently out of scope.

Of course, I could create a layer which would act the same way as 7-zip does.

But I feel it would not be at its proper location.

Usually, between the compression layer, and the file splitting layer, stands the archive layer (the one which agglomerate multiple files and directories).

For LZ4, as for gzip, the archive layer is typically done by tar.

So, to be on the clean side, the splitting should happen *after* tar, not before.

One possible solution would be to use the "split" program, for example :

# compress and cut into chunks
$ lz4c -c my_large_file - | split -b 1024MiB - my_large_file_split.lz4_
# combine and decompress
$ cat my_large_file_split.lz4_* | lz4c -d > my_large_file

This solution avoids the need to use any intermediate large file when (de)compressing. 

Regards

Yann