How To:Split DLEs With Exclude Lists

This article describes a way to split your DLEs to produce smaller individual dumps. This is often useful when you only have a few DLEs, and one of them is much larger than the rest: without this technique, the tape usage spikes when Amanda runs a level 0 for that DLE. For example, many small businesses have one very large NFS-mounted /home. Using this technique, /home can be split by subdirectory, e.g., /home/[a-o]* and /home/[p-q]*. A more extended example is provided below.

With Amanda < 2.5.0 the need arises to virtually split partitions into parts so that each part remains smaller than a tape. This will lead to smaller single dumps so that Amanda can continue the backup of other dumps on the next tape, if need be.

If you have a lot of large dumps and Amanda 2.5.0 or later, then tape spanning may be more useful than this technique.

Example

Let's assume that you have two large partitions, holding several directories and subdirectories like the following:

/home/
/home/andrew
/home/anne
/home/brian
[...]
/home/wanda
/home/zeus

/share/
/share/financial
/share/financial/company-a
/share/financial/company-b
/share/reports

You created entries in your disklist to back them up:

largehost /home        my-dumptype
largehost /share       my-dumptype

That worked fine, but now on each of the partitions, there's too much data to fit onto a single tape. You will have to split up the partitions to back them up.

Splitting it up, the easy part

Considering the /home partition, it becomes obvious that you could split it alphabetically. You would create disklist entries to back up each subdirectory by using wildcards. You also want to catch anything under /home that doesn't fit into the wildcard specifications, so there:

largehost /home_a /home {
  my-dumptype
  include "./a*"
}       1
largehost /home_b /home {
  my-dumptype
  include "./b*"
}       1
[...]
largehost /home_w /home {
  my-dumptype
  include "./w*"
}       1
largehost /home_z /home {
  my-dumptype
  include "./z*"
}       1
# catch all the rest of /home:
largehost /home_rest /home {
  my-dumptype
  exclude append "./[a-z]*"
}       1

Notice the use of the 'include' and 'exclude' options. They do what you would guess, i. e. include the files specified, or exclude them. The tricky part is to use different disk names ('/home_a', '/home_b', ...) on the same partition/mount point ('/home'). You can omit specifying the spindle.

Splitting it up, the more complicated part

The /share partition is a little more complicated to split because you have to deal with the subdirectories. We assume that there's so much data in some of the directories that you have to treat them by diving into the subdirectories. You need to catch /share, /share/financial, /share/financial/company-a, /share/financial/company-b and /share/reports:

largehost /share_financial_company-a /share/financial {
  my-dumptype
  include "./company-a"
}       2
largehost /share_financial_company-b /share/financial {
  my-dumptype
  include "./company-b"
}       2
largehost /share_financial_rest /share/financial {
  my-dumptype
  exclude append "./company-a"
  exclude append "./company-b"
}       2
#
largehost /share_reports /share {
  my-dumptype
  include "./reports"
}       2
#
largehost /share_rest /share {
  my-dumptype
  exclude append "./financial"
  exclude append "./reports"
}       2

In this example, it is important to notice that each subdirectory is refered to from its immediate parent directory. It doesn't work otherwise.

To use many exclude/include arguments on one line, you must put quotes around each argument:

exclude append "./financial" "./reports"

Currently (Amanda version 2.4.x and 2.5.0) do not allow spaces in the exclude/include pattern:

exclude append "./Documents and Settings"    # Invalid!

You'll get an obscure error message about "wrong number of arguments" in that case. There is no escape character to protect the space. As workaround you can use wildcards instead:

exclude append "./Documents?and?Settings"    # workaround space in pattern

See also

There are some important notes in the article on Exclude and include lists.