How To:Use pigz to speed up compression
pigz, which stands for parallel implementation of gzip, is a fully functional drop-in replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data. Using it our new 16 thread (dual Xeon 5520 quad core multithreaded cpus) capable backup server has speed up our backups dramatically.
How I use it is by defining the following dump type:
define dumptype scomp-user-tar { user-tar compress server custom server_custom_compress "/usr/bin/pigz" }
You could also run the compression on the clients:
define dumptype comp-user-tar { user-tar compress client custom client_custom_compress "/usr/bin/pigz" }
but be aware that using pigz is going to put a lot more load on the machine running the compression. In our case, we want to affect our disk servers as little as possible while they are getting backed up. Finally, you will need to make sure that pigz is installed on all machines that need it.