How To:Backup to Virtual Tapes on a non-UNIX Filesystem

From The Open Source Backup Wiki (Amanda, MySQL Backup, BackupPC)
Jump to navigationJump to search

This article is a part of the How Tos collection.

Introduction

Amanda has the ability to use a hard disk to store the contents of backups onto virtual tapes (vtapes) using the file driver. The vtapes can be located on any filesystem, even a remote one.

If you are following the instructions under Using chg-disk in How To:Set Up Virtual Tapes and get to the point where you run

 $ ln -s slot1 data

and get

 ln: creating symbolic link `data': Operation not supported

you likely are trying to put your virtual tapes on a file system that does not support symbolic links. Unlike the older chg-multi changer script, chg-disk uses a symbolic link to point to the currently active slot of your virtual tape changer. If you can't create the link above, neither can Amanda.

A common example of this is using CIFS to store vtapes on a Windows machine. CIFS is not specific to Windows, however, many NAS devices offer it as well.

The best way to solve this would be to change anything concerning the share or file system that would permit symbolic linking. If using NAS, it may be as simple as checking a check box on the server side. On the client side, it may be possible to mount the share or file system with certain options to allow symbolic linking.

If creating symbolic links within the share or file system you want to use is not an option, two solutions are described below. The first sets up the vtape structure on a filesystem supporting symlinks, using symlinks to point into the non-UNIX filesystem. The second uses the older chg-multi changer script instead of chg-disk. For full details on this and setting up vtapes, please see How To:Set Up Virtual Tapes.

Symlink Trickery

1. If it does not already exist, create the directory space on your non-symlink-supporting share or file system where the vtapes will actually be saved. If the full share is writable by the Amanda user, you can simply create this directory as this user.

-bash-3.00$ mkdir -p /mnt/samba/vtapes/DailySet1

If you must be root to create this directory, make sure to set proper permissions for your Amanda user:

[root:/]# mkdir -p /mnt/samba/vtapes/DailySet1
[root:/]# chown amandabackup:disk /mnt/samba/vtapes/DailySet1
[root:/]# chmod 750 /mnt/samba/vtapes/DailySet1

2. Create a virtual slot directory for every vtape you will be using (in this example, 15 tapes). As the Amanda user

-bash-3.00$ cd /mnt/samba/vtapes/DailySet1/
-bash-3.00$ for ((i=1; $i<=15; i++)); do mkdir slot$i; done

Once accomplished, you should have something like

-bash-3.00$ ls -l
total 0
drwxrwxrwx 2 amandabackup disk 0 2008-11-12 15:08 slot1
drwxrwxrwx 2 amandabackup disk 0 2008-11-12 15:08 slot10
drwxrwxrwx 2 amandabackup disk 0 2008-11-12 15:08 slot11
drwxrwxrwx 2 amandabackup disk 0 2008-11-12 15:08 slot12
drwxrwxrwx 2 amandabackup disk 0 2008-11-12 15:08 slot13
drwxrwxrwx 2 amandabackup disk 0 2008-11-12 15:08 slot14
drwxrwxrwx 2 amandabackup disk 0 2008-11-12 15:08 slot15
drwxrwxrwx 2 amandabackup disk 0 2008-11-12 15:08 slot2
drwxrwxrwx 2 amandabackup disk 0 2008-11-12 15:08 slot3
drwxrwxrwx 2 amandabackup disk 0 2008-11-12 15:08 slot4
drwxrwxrwx 2 amandabackup disk 0 2008-11-12 15:08 slot5
drwxrwxrwx 2 amandabackup disk 0 2008-11-12 15:08 slot6
drwxrwxrwx 2 amandabackup disk 0 2008-11-12 15:08 slot7
drwxrwxrwx 2 amandabackup disk 0 2008-11-12 15:08 slot8
drwxrwxrwx 2 amandabackup disk 0 2008-11-12 15:08 slot9

3. On an available file system that supports symbolic linking, create a directory structure for vtapes and use this directory as the file driver for the tapedev parameter in your amanda, conf, for ex. tapedev file:/var/lib/amanda/vtapes/DailySet1. The vtape directory will result in a small directory structure containing links and will require very little space of the file system (a few Kb). You can create this right in your Amanda user's home directory, for instance. As the Amanda user, create a directory for vtapes for your backup configuration

[amandabackup~]$ mkdir -p /var/lib/amanda/vtapes/DailySet1

If you create this in a place that requires root permission, make sure you set proper permissions for the new directory for your Amanda user

[root:/]# mkdir -p /var/lib/amanda/vtapes/DailySet1
[root:/]# chown amandabackup:disk /var/lib/amanda/vtapes/DailySet1
[root:/]# chmod 750 /var/lib/amanda/vtapes/DailySet1

4. Enter this directory and create the links to the virtual slots in /mnt/samba/vtapes/DailySet1/ as the Amanda user:

-bash-3.00$ cd /var/lib/amanda/vtapes/DailySet1
-bash-3.00$ for ((i=1; $i<=15; i++)); do ln -s /mnt/samba/vtapes/DailySet1/slot$i slot$i; done

lastly, create the link that was problematic in the first place

-bash-3.00$ ln -s slot1 data

and with this accomplished, you should have something like

-bash-3.00$ ls -al
lrwxrwxrwx  1 amandabackup disk    5 Jun 15 11:27 data -> slot1
lrwxrwxrwx  1 amandabackup disk   40 Jun 15 11:26 slot1 -> /mnt/samba/vtapes/DailySet1/slot1
lrwxrwxrwx  1 amandabackup disk   40 Jun 15 11:26 slot2 -> /mnt/samba/vtapes/DailySet1/slot2
lrwxrwxrwx  1 amandabackup disk   40 Jun 15 11:26 slot3 -> /mnt/samba/vtapes/DailySet1/slot3
lrwxrwxrwx  1 amandabackup disk   40 Jun 15 11:26 slot4 -> /mnt/samba/vtapes/DailySet1/slot4
lrwxrwxrwx  1 amandabackup disk   40 Jun 15 11:26 slot5 -> /mnt/samba/vtapes/DailySet1/slot5
lrwxrwxrwx  1 amandabackup disk   40 Jun 15 11:26 slot6 -> /mnt/samba/vtapes/DailySet1/slot6
lrwxrwxrwx  1 amandabackup disk   40 Jun 15 11:26 slot7 -> /mnt/samba/vtapes/DailySet1/slot7
lrwxrwxrwx  1 amandabackup disk   40 Jun 15 11:26 slot8 -> /mnt/samba/vtapes/DailySet1/slot8
lrwxrwxrwx  1 amandabackup disk   40 Jun 15 11:26 slot9 -> /mnt/samba/vtapes/DailySet1/slot9
lrwxrwxrwx  1 amandabackup disk   40 Jun 15 11:26 slot10 -> /mnt/samba/vtapes/DailySet1/slot10
lrwxrwxrwx  1 amandabackup disk   40 Jun 15 11:26 slot11 -> /mnt/samba/vtapes/DailySet1/slot11
lrwxrwxrwx  1 amandabackup disk   40 Jun 15 11:26 slot12 -> /mnt/samba/vtapes/DailySet1/slot12
lrwxrwxrwx  1 amandabackup disk   40 Jun 15 11:26 slot13 -> /mnt/samba/vtapes/DailySet1/slot13
lrwxrwxrwx  1 amandabackup disk   40 Jun 15 11:26 slot14 -> /mnt/samba/vtapes/DailySet1/slot14
lrwxrwxrwx  1 amandabackup disk   40 Jun 15 11:26 slot15 -> /mnt/samba/vtapes/DailySet1/slot15

5. You can now either tell Amanda to automatically label the tapes for you or you can run amlabel to label them yourself.

Do it automatically:

When using Amanda 2.5.0 and later, you can let Amanda label the tapes automatically on first use. Add the parameter label_new_tapes to the amanda.conf file, and give it a template with '%' signs for the number, like:

label_new_tapes "DailySet1-%%"

Note: label_new_tapes is deprecated in Amanda version 3.1; use the parameter autolabel instead.

The usual warnings about this being dangerous because it will erase non-Amanda tapes does not apply here as we are pretty sure that all the vtapes are indeed to be used for this purpose.

Do it myself:

You can also label the tapes manually yourself, if you prefer, as the Amanda user:

-bash-3.00$ for ((i=1; $i<=9; i++)); do amlabel DailySet1 DailySet1-0$i slot $i; done
labeling tape in slot 1 (file://var/lib/amanda/vtapes/DailySet1):
rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-01, checking label, done.
labeling tape in slot 2 (file://var/lib/amanda/vtapes/DailySet1):
rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-02, checking label, done.
.
.
.
-bash-3.00$ for ((i=10; $i<=15; i++)); do amlabel DailySet1 DailySet1-$i slot $i; done
labeling tape in slot 10 (file://var/lib/amanda/vtapes/DailySet1):
rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-10, checking label, done.
labeling tape in slot 11 (file://var/lib/amanda/vtapes/DailySet1):
rewinding, reading label, not an amanda tape (Read 0 bytes)
rewinding, writing label DailySet1-11, checking label, done.
.
.
.


Performance

Performance using CIFS via Samba is known to be poorer than protocols such as FTP, HTTP and even CIFS on Windows. This is apparently due to Samba not using asynchronous writes, so each write has to be acknowledged by the server before the next one can take place.

Before deciding to use CIFS it would be advisable to test the speed of a transfer to a CIFS mount from your amanda server. Although bandwidth is an issue, latency also plays a big part.

More research on increasing performance is done and the results will be posted here.

Using chg-multi

A much older changer script is "chg-multi", which emulates a changer consisting of multiple tape drives, and -- critically -- doesn't use symlnks. If you have two tape drives of the same model, and hook them to the server, this script allows you to emulate a changer with two slots. The situation here is similar: we will set up a number of independent vtapes, and then describe each of them to chg-multi. In this example, all of the vtapes are in the same parent directory, but this is not necessary.

Create 5 vtapes for our "test" configuration:

for i in 1 2 3 4 5; do mkdir -p /mnt/samba/vtapes/test/tape$i/data; done

Now we have a server with 5 tape drives. They are virtual tapes, but Amanda isn't picky about that.

Change amanda.conf for the "test" configuration to have these values:

tpchanger "chg-multi"
changerfile "chg-multi.conf"       # name of the special configuration file
# tapedev is ignored if present, to avoid confusion, just comment it out
# changerdev is ignored too

The chg-multi script needs more configuration and uses the parameter changerfile in amanda.conf as the name of that special config file. And add a file in the same directory as amanda.conf with the name we specified above as the changerfile chg-multi.conf. Because "chg-multi" is useful in a wide context, we need to specify a lot of parameters that are irrelevant for vtapes:

multieject 0
needeject 0
gravity 0
ejectdelay 0
statefile /home/amanda/test/changerstatus
firstslot 1
lastslot 5
slot 1 file:/mnt/samba/vtapes/test/tape1
slot 2 file:/mnt/samba/vtapes/test/tape2
slot 3 file:/mnt/samba/vtapes/test/tape3
slot 4 file:/mnt/samba/vtapes/test/tape4
slot 5 file:/mnt/samba/vtapes/test/tape5

The statefile parameter in the file chg-multi.conf is now the prefix of some files that hold the status of the emulated changer. It should be a filename in a directory where the Amanda user can write.

And we label the tapes in all the slots:

$ for i in 1 2 3 4 5; do amlabel test TEST-$i slot $i; done

Or use the label_new_tapes parameter in amanda.conf (if you have at least Amanda 2.5.0) to this automatically, as explained in the chg-disk section above.

Then, as usual, the final checks:

$ amcheck test
$ amtape test show
$ amtape test reset

And we are ready to use our tape changer with 5 tape drives.