Splitsize too small

This article is a part of the Troubleshooting collection.

Problem

Amcheck gives warnings about tape_splitsize or part_size:

WARNING: knuth /usr: tape_splitsize of 10240 KB < 0.1% of tape length.
 This may create > 1000 parts, severely degrading backup/restore performance.
 To remedy, increase tape_splitsize or disable splitting by setting it to 0
 See http://wiki.zmanda.com/index.php/Splitsize_too_small for more information.

or about fallback_splitsize or part_cache_max_size:

WARNING: euclid /var/lib: fallback_splitsize of 10240 KB < 0.1% of tape length.
 This may create > 1000 parts, severely degrading backup/restore performance.
 To remedy, create/set a split_diskbuffer or increase fallback_splitsize
 See http://wiki.zmanda.com/index.php/Splitsize_too_small for more information.

and sends you to this wiki page.

Solution

This is usually due to a misconfiguration - sometimes a misconfiguration that has been in place for many years! The misconfiguration can cause Amanda to produce too many parts for a dump. This has unfortunate consequences:

• Writes to tape are slowed, because the tape drive must stop to flush its buffers and write a filemark thousands of times per tape. This is less significant with vtapes.
• Amanda uses on-volume space inefficiently - filemarks and headers take up valuable space, even on vtapes
• Recoveries using amfetchdump or amrecover are slow, because they must open and read from thousands of parts
• Recoveries using amrestore are extremely painful, since it is difficult to concatenate thousands of files together with shell tools

Splitting Config

Amanda-3.2 and later

The primary splitting parameter is the tapetype parameter part_size. Set this to something greater than 0.1% of your tape length.

If Amanda may need to "rewind" a part that has failed during a write to tape, then the part cache comes into play. This is a rare circumstance, requiring:

• the data is directly from the dumper, not via holding disk
• Amanda is configured to use more than one tape (runtapes > 1)
• the underlying device cannot detect end-of-medium in advance (LEOM support)
• the part cache is only consulted if the device actually reaches EOM during the dump

If part caching is required, then Amanda will reduce the part size to part_cache_max_size. Especially when caching in RAM, setting this value to something smaller than part_size is useful to limit the amount of space required for the part cache. The part_cache_type "none" can be used to avoid caching even in cases where it would otherwise be required. In this case, if Amanda reaches end-of-medium and all of the above conditions are met, then entire DLE fails - but Amanda will retry it up to three times, so your data is still safe.

Amanda-3.1 and earlier

The three splitting parameters are tape_splitsize, split_diskbuffer, and fallback_splitsize.

When Amanda is writing to tape from holding-disk, it always creates parts of size tape_splitsize, and does not require any other buffers. However, Amanda can decide to skip holding-disk without much warning. For a dump direct to tape (also known as PORT-WRITE), the rules for the splitsize are a little more complicated:

• if split_diskbuffer is set, the dirctory exists, and it has at least tape_splitsize bytes free, then the parts are of size tape_splitsize
• if split_diskbuffer is not set, or if the directory does not exist or does not have enough free space, then
  • if fallback_splitsize is set, the parts are of size fallback_splitsize; otherwise
  • the parts are of size 10M (the default value of fallback_splitsize).

The common misconfiguration is to set tape_splitsize but not split_diskbuffer or fallback_splitsize, with the consequence that when a dump goes straight to tape, it is written in 10M parts. Unless you're watching your reports carefully, you won't notice this until you try to do a recovery - and possible not even then!

Fix

For the first warning, the recommended splitsize is something between 1% and 5% of your tape size, so you should adjust your part_size or tape_splitsize accordingly.

For the second warning, most likely you have not changed the default fallback_splitsize and have not set split_diskbuffer. If you have enough RAM to hold an entire part, set the fallback_splitsize to the desired part size. Otherwise, set split_diskbuffer to a directory with sufficient space for your tape_splitsize. If you are using part_cache_max_size, and you do not have space (either on disk or in memory) for at least 0.1% of your tape length, then you should consider acquiring more space or disabling caching altogether (part_cache_type "none").