So, if you have a contemporary multi-processor, multi-core machine and want to compress larger files as quickly as possible while utilizing all of your CPU cores, pigz is an excellent option! Give it a shot and let us know what you think of the Pigz compression application in the comments below.
If you are using gzip, you use mostly one processor core (well, some parts of the task, like reading and writing data are kernel tasks and kernel will use another core). Have a look at some multicore-capable gzip replacements, e.g. MiGz ( ) or Pigz ( , for some longer explanation see also e.g. -sh/pigz-a-faster-alternative-to-gzip-for-big-files-d5909e46d659).
The three main tools in question here are pigz, pbzip2, and pxz. There are some subtle differences between the tools, but the differences lie between gzip, bzip2, and xz. In that respective order, the compression levels increase, meaning that an archive compressed with gzip will be larger than one compressed with xz, but gzip will naturally take less time than xz will. bzip2 is somewhere in the middle.
By itself, Pigz does not have options to compress a folder, it only compresses single files. As a workaround, pigz iused in conjunction with tar command to zip directories.
Since we start we a fresh installation, we will need to install everything, i.e, all packages are marked as [-]. The ouput of the spec can be quite complex to read but is really exhaustive. You see all compilations options as well as the architecture and the compiler versions. There is no notion of toolchains (see for Easybuild).Spack automatically detects the architecture. You can explicitely obtain it with the command spack arch, i.e., on Aion you will get linux-rhel8-zen.