Hardware compression

Compression in hardware, i.e. in the tapedrive itself, is not done by writing the bits in a higher density, or by using more tracks, or using both sides of the tape. Hardware compression is achieved with an algorithm, built into the firmware of the tapedrive, which removes redundancy from the byte stream, resulting in less bits to write to the tape medium. Manufacturers often claim to achieve a 2:1 or even 2.6:1 reduction on "normal data", hence the advertising of e.g. 36/72 GB tape capacity. Some of those algorithms, when given uncompressable data, actually result in an expansion by 5-15%. The net amount of tape capacity for such uncompressable data is lowered as well: e.g. the expected 36 GB capacity is now around 30-32 GB for such drives. Some tapedrives have a much better compression firmware, which does not have that negative impact (e.g. LTO tape drives).

It is usually recommended to disable hardware compression with Amanda, though it may have less of a negative impact with certain modern tape formats (e.g. LTO).

There are multiple methods to appropriately set (i.e. turn off) hardware compression. They may vary between OSes somewhat, not all tape drives support this setting, and tape drives may support a few methods at a protocol level for controlling hardware compression, methods which your OS's mt(1) tool or similar may or may not handle - e.g. the "device configuration" scsi-2 mode page 0x10 (common), the "data compression" scsi-3 mode page 0x0F (sometimes) or a particular special "density" setting (rare nowadays, thankfully). If your tape drive still has hardware-level DIP switches to control settings, it can be more pleasant to use them.

Resurfacing HW compression

Note that some tape drives re-enable HW compression if they read HW compressed data, as explained e.g. here. Therefore it may be necessary to properly overwrite existing tapes to ensure HW compression stays off.

Please note that compression information is not stored on the tapes ident header block until the tape has been written to.

Procedure for turning off compression and labelling tapes

• Label the tape
• Rewind the tape
• Read the label to a file using dd command
• Turn off tape compression using mt(1) command. See next section.
• Re-write the label block and write more /dev/zero blocks to flush its buffers.

OS specific notes

Linux

• Using stinit command: stinit(8) command initializes the SCSI tape drives at the system startup by sending driver ioctl commands. Use "comp" field in /etc/stinit.def configuration file to configure hardware compression for each type of tape.

• Use (a recent version of) mt(1) command to turn off hardware compression at boot time. For example:

# mt -f /dev/st0 compression 0

• Use (a recent version of) mt(1) command to set default compression - e.g.

# mt -f /dev/nst0 defcompression 0

(some tape drives don't seem to support this)

FreeBSD

• Use mt(1) command to turn off hardware compression at boot time. For example:

# mt -f /dev/nsa0 comp off

NetBSD

• Use mt(1) command to turn off hardware compression at boot time. For example:

# mt -f /dev/nrst0 compress 0

OpenBSD

• At time of writing (OpenBSD 3.9), OpenBSD's mt(1) doesn't allow manipulation of hardware compression settings. However, one can use the swiss-army-knife scsi(8) command to change the compression setting (and more besides - be careful...) in the mode pages. This should probably be considered undocumented and subject to change, as the st(4) man page describes as "reserved" the device bit pattern enrst0 used here to put the st kernel driver into "control mode".

# EDITOR=vi scsi -f /dev/enrst0 -m 16 -e -P0 

( -m 16 selects the "Device Configuration" mode page. Change the value of "Select Data Compression Algorithm" to 0 in the text editor launched, save and exit. Using -P3 _might_ save the setting, equivalent to linux mt "defcompression" above, but some tape drives don't seem to support it. )

Solaris

Set ST_MODE_SEL_COMP in st.conf for your device and then use the correct tape device variant (do not use "c" and "u" device)

AIX

(untested, based on public man pages - someone aixy please verify)

# chdev -l rmt0 -a compress=no