How To:Mirror Dumps to Virtual Tapes and Real Tapes using RAIT: Difference between revisions

From wiki.zmanda.com
Jump to navigation Jump to search
No edit summary
 
(24 intermediate revisions by 5 users not shown)
Line 1: Line 1:
Currently it is only integrated with the chg-manual changer script
{{How To Header}}
= Deprecated =
Using RAIT to mirror dumps is deprecated, see [[How To:Write dump to multiple storage]] for better mechanism.


RAIT is an acronym for "Redundant Array of Inexpensive Tapes", where data is striped over several tape drives, with one drive writing an exclusive-or-sum of the others which can be used for error recovery. Any one of the data streams can be lost, and the data can still be recovered.
= Overview =
The basis of a RAIT configuration is the [[RAIT Device]], which allows Amanda to mirror (with two child devices) or stripe with parity (with 3 or more child devices) data across several child devices.  See [[How To:Stripe Dumps across Tapes using RAIT]] for a bit more background on the RAIT device and RAIT-related changers.


This means that a 3-drive RAIT set will write 2 "data" streams and one "parity" stream, and give you twice the capacity, twice the throughput, and the square of the failure rate (i.e. a 1/100 failure rate becomes 1/10,000, since a double-tape failure is required to lose data).
=Mirroring a backup=


Similarly, a 5-drive RAIT set will give you 4 times the capacity, 4 times the throughput (with sufficient bus bandwidth), and the square of the failure rate.
RAIT with only two drives, creates a perfect copy of the data: a mirror of your backup.
Each of these mirrors is handled by its own device. For example, one of the mirrors can
write to a real tape device, and the other mirror can write a virtual tape (vtape) using
the VFS device.  The vtapes can be used for fast recovery, and the tapes can be stored
offsite.


This means you can back up partitions as large as four times your tape size with AMANDA, with higher reliability and speed.
This example shows how to mirror a dump to virtual tapes and real tapes.  First, get each set up and running independently.  For example, your vtape configuration (based on [[How To:Set Up Virtual Tapes]]) might look like
tpchanger "chg-disk:/u01/vtapes"
with, say, 25 slots in <tt>/u01/vtapes</tt>.  Similarly, your physical tape configuration might be
  tpchanger "chg-zd-mtx"
  changerdev "/dev/mtx1"
  changerfile "/etc/amanda/myconfig/chg-zd-mtx.conf"


====Using a RAIT====
Combining these two is fairly straightforward.  The first can be specified directly, since it has no configuration parameters, but the second will require its own changer definition:
define changer "robot" {
  tpchanger "chg-zd-mtx"
  changerdev "/dev/mtx1"
  changerfile "/etc/amanda/myconfig/chg-zd-mtx.conf"
}
Then, specify both to the RAIT changer as the Amanda tape device:
tpchanger "chg-rait:{chg-disk:/u01/vtapes,robot}"
 
Run some {{man|8|amtape}} subcommands to exercise the new changer, then an {{man|8|amcheck}} and {{man|8|amdump}}.  Easy as that!
 
== Old, chg-multi method ==
''This method is for use in Amanda-2.6.1 and earlier, before the new RAIT changer was available.''
 
Together with the "chg-multi" changer, which is suitable for a whole range of applications, this setup requires these parameters in '''amanda.conf''' (the parameters '''dumpcycle''', '''runspercycle''' and '''tapecycle''' should be adapted to your environment):
 
dumpcycle 1 week
runspercycle 5
tapecycle 10
tpchanger "chg-multi"
changerfile "chg-multi-mirror.conf"
# tapedev is not needed
# changerdev is not needed
 
We create the file '''chg-multi-mirror.conf''' in the same directory, with these contents:


If you have several tape devices on your system [currently either 3 or 5 drive sets are supported] you tell AMANDA to use them as a RAIT by listing them as a single tape device using /bin/csh curly-brace-and-comma notation, as in:
multieject 0
needeject 0
gravity 0
ejectdelay 0
statefile /home/amanda/daily/changerstatus
firstslot 1
lastslot 10
slot 1 rait:{file:/amandatapes/daily/vtape1,tape:/dev/nst0}
slot 2 rait:{file:/amandatapes/daily/vtape2,tape:/dev/nst0}
slot 3 rait:{file:/amandatapes/daily/vtape3,tape:/dev/nst0}
slot 4 rait:{file:/amandatapes/daily/vtape4,tape:/dev/nst0}
slot 5 rait:{file:/amandatapes/daily/vtape5,tape:/dev/nst0}
slot 6 rait:{file:/amandatapes/daily/vtape6,tape:/dev/nst0}
slot 7 rait:{file:/amandatapes/daily/vtape7,tape:/dev/nst0}
slot 8 rait:{file:/amandatapes/daily/vtape8,tape:/dev/nst0}
slot 9 rait:{file:/amandatapes/daily/vtape9,tape:/dev/nst0}
slot 10 rait:{file:/amandatapes/daily/vtape10,tape:/dev/nst0}


tapedev = "rait:/dev/rmt/tps0d{4,5,6}n"
This assumes we have only 1 tapedrive (/dev/nst0), which we need to manually change the tape each day.


which means that /dev/rmt/tps0d4n, /dev/rmt/tps0d5n, and /dev/rmt/tps0d6n are to be treated as a RAIT set. You can now mount three tapes, and label them with amlabel, etc.
We create the 10 virtual tapes:


Also, you want to create a new tape-type entry, which lists an n-drive RAIT set, for this RAIT-set. So if you were using an entry like:
$ for i in 1 2 3 4 5 6 7 8 9 10; do mkdir -p /amandatapes/daily/vtape$i/data; done


  define tapetype EXB-8500 {
We insert a tape in the physical tapedrive, and label the first tape. This will label both the virtual tape vtape1 and the tape in the physical drive identically:
        comment "Exabyte EXB-8500 drive on decent machine"
 
        length 4200 mbytes
$ amlabel daily DAILY-01 slot 1
        filemark 48 kbytes
$ amcheck daily
        speed 474 kbytes
 
  }
Last check, and we are ready for the normal amdump run.
 
==Using amrecover with half of the mirror==
 
When needing to restore, you can use the specific vtape, or the physical tape.
Just specify the correct device to amrecover (or amrestore) on the command line, or with the '''settape''' subcommand from amrecover.
 
# amrecover daily -d amandaserver.example.com:file:/amandatapes/daily/vtape5
 
Note that, when you specify the driver, you must specify the hostname too.
 
Alternatively, you could set up a special configuration for restoring, named "halfmirror"
that is almost identical to the normal config, except for the chg-multi.conf:
fill that file without ''rait'' option and only the virtual tape half of the mirror:
 
In the '''amanda.conf''' from the configuration named ''halfmirror'':
 
tpchanger "chg-multi"
changerfile "chg-multi-halfmirror.conf"
amrecover_changer "halfmirror"
amrecover_do_fsf true
amrecover_check_label true
 
'''chg-multi-halfmirror.conf''':
 
multieject 0
needeject 0
gravity 0
ejectdelay 0
statefile /home/amanda/daily/restorchgstatus  # different name than the normal mirror
firstslot 1
lastslot 10
slot 1 file:/amandatapes/daily/vtape1
slot 2 file:/amandatapes/daily/vtape2
slot 3 file:/amandatapes/daily/vtape3
slot 4 file:/amandatapes/daily/vtape4
slot 5 file:/amandatapes/daily/vtape5
slot 6 file:/amandatapes/daily/vtape6
slot 7 file:/amandatapes/daily/vtape7
slot 8 file:/amandatapes/daily/vtape8
  slot 9 file:/amandatapes/daily/vtape9
slot 10 file:/amandatapes/daily/vtape10


You would want to make a new one like:
By using a different name for '''statefile''' above, we do not interfere with the notion
of current tape etc from the normal daily configuration. (This would not work with chg-disk.)


define tapetype EXB-8500x3 {
And recovering from this set of vtapes goes like:
        comment "Exabyte EXB-8500 3 drive stripe on decent machine"
        length 8400 mbytes
        filemark 200 kbytes
        speed 948 kbytes
}
and change your tapetype entry to: tapetype EXB-8500x3 to tell AMANDA about the multiple drive set.


====Disaster Recovery====
# amrecover halfmirror -d halfmirror


To assist in disaster recovery (as well as changer scripts) the AMANDA package now also includes amdd, which is a simple dd(1) replacement which supports (only) the "if=xxx", "of=xxx", "bs=nnn[kMb]" "skip=nnn" and "count=nnn" options, but which can read and write RAIT tapesets.
{|
| bgcolor=yellow | Tip
|}
If you are short in diskspace, you can get fancy, and keep only the last 5 (runspercycle) vtapes online,
by removing the contents of the data directory in each vtape that is older than runspercycle,  
except the first pair of files (the label).
Even the "TAPEEND" pair of files is not strictly needed for Amanda.
An erased vtape, with correct label then looks like:


Using [[amdd]] and your usual AMANDA unpack instructions will suffice for disaster recovery from RAIT tape-sets.
$ ls -l /amandatapes/daily/vtape4
-rw-------  1 amanda disk      10 Mar 29 00:04 00000-DAILY-04
-rw-------  1 amanda disk  32768 Mar 29 00:04 00000.DAILY-04

Latest revision as of 14:35, 29 January 2018

This article is a part of the How Tos collection.

Deprecated

Using RAIT to mirror dumps is deprecated, see How To:Write dump to multiple storage for better mechanism.

Overview

The basis of a RAIT configuration is the RAIT Device, which allows Amanda to mirror (with two child devices) or stripe with parity (with 3 or more child devices) data across several child devices. See How To:Stripe Dumps across Tapes using RAIT for a bit more background on the RAIT device and RAIT-related changers.

Mirroring a backup

RAIT with only two drives, creates a perfect copy of the data: a mirror of your backup. Each of these mirrors is handled by its own device. For example, one of the mirrors can write to a real tape device, and the other mirror can write a virtual tape (vtape) using the VFS device. The vtapes can be used for fast recovery, and the tapes can be stored offsite.

This example shows how to mirror a dump to virtual tapes and real tapes. First, get each set up and running independently. For example, your vtape configuration (based on How To:Set Up Virtual Tapes) might look like

tpchanger "chg-disk:/u01/vtapes"

with, say, 25 slots in /u01/vtapes. Similarly, your physical tape configuration might be

 tpchanger "chg-zd-mtx"
 changerdev "/dev/mtx1"
 changerfile "/etc/amanda/myconfig/chg-zd-mtx.conf"

Combining these two is fairly straightforward. The first can be specified directly, since it has no configuration parameters, but the second will require its own changer definition:

define changer "robot" {
 tpchanger "chg-zd-mtx"
 changerdev "/dev/mtx1"
 changerfile "/etc/amanda/myconfig/chg-zd-mtx.conf"
}

Then, specify both to the RAIT changer as the Amanda tape device:

tpchanger "chg-rait:{chg-disk:/u01/vtapes,robot}"

Run some amtape(8) subcommands to exercise the new changer, then an amcheck(8) and amdump(8). Easy as that!

Old, chg-multi method

This method is for use in Amanda-2.6.1 and earlier, before the new RAIT changer was available.

Together with the "chg-multi" changer, which is suitable for a whole range of applications, this setup requires these parameters in amanda.conf (the parameters dumpcycle, runspercycle and tapecycle should be adapted to your environment):

dumpcycle 1 week
runspercycle 5
tapecycle 10
tpchanger "chg-multi"
changerfile "chg-multi-mirror.conf"
# tapedev is not needed
# changerdev is not needed

We create the file chg-multi-mirror.conf in the same directory, with these contents:

multieject 0
needeject 0
gravity 0
ejectdelay 0
statefile /home/amanda/daily/changerstatus
firstslot 1
lastslot 10
slot 1 rait:{file:/amandatapes/daily/vtape1,tape:/dev/nst0}
slot 2 rait:{file:/amandatapes/daily/vtape2,tape:/dev/nst0}
slot 3 rait:{file:/amandatapes/daily/vtape3,tape:/dev/nst0}
slot 4 rait:{file:/amandatapes/daily/vtape4,tape:/dev/nst0}
slot 5 rait:{file:/amandatapes/daily/vtape5,tape:/dev/nst0}
slot 6 rait:{file:/amandatapes/daily/vtape6,tape:/dev/nst0}
slot 7 rait:{file:/amandatapes/daily/vtape7,tape:/dev/nst0}
slot 8 rait:{file:/amandatapes/daily/vtape8,tape:/dev/nst0}
slot 9 rait:{file:/amandatapes/daily/vtape9,tape:/dev/nst0}
slot 10 rait:{file:/amandatapes/daily/vtape10,tape:/dev/nst0}

This assumes we have only 1 tapedrive (/dev/nst0), which we need to manually change the tape each day.

We create the 10 virtual tapes:

$ for i in 1 2 3 4 5 6 7 8 9 10; do mkdir -p /amandatapes/daily/vtape$i/data; done

We insert a tape in the physical tapedrive, and label the first tape. This will label both the virtual tape vtape1 and the tape in the physical drive identically:

$ amlabel daily DAILY-01 slot 1
$ amcheck daily

Last check, and we are ready for the normal amdump run.

Using amrecover with half of the mirror

When needing to restore, you can use the specific vtape, or the physical tape. Just specify the correct device to amrecover (or amrestore) on the command line, or with the settape subcommand from amrecover.

# amrecover daily -d amandaserver.example.com:file:/amandatapes/daily/vtape5

Note that, when you specify the driver, you must specify the hostname too.

Alternatively, you could set up a special configuration for restoring, named "halfmirror" that is almost identical to the normal config, except for the chg-multi.conf: fill that file without rait option and only the virtual tape half of the mirror:

In the amanda.conf from the configuration named halfmirror:

tpchanger "chg-multi"
changerfile "chg-multi-halfmirror.conf"
amrecover_changer "halfmirror"
amrecover_do_fsf true
amrecover_check_label true

chg-multi-halfmirror.conf:

multieject 0
needeject 0
gravity 0
ejectdelay 0
statefile /home/amanda/daily/restorchgstatus   # different name than the normal mirror
firstslot 1
lastslot 10
slot 1 file:/amandatapes/daily/vtape1
slot 2 file:/amandatapes/daily/vtape2
slot 3 file:/amandatapes/daily/vtape3
slot 4 file:/amandatapes/daily/vtape4
slot 5 file:/amandatapes/daily/vtape5
slot 6 file:/amandatapes/daily/vtape6
slot 7 file:/amandatapes/daily/vtape7
slot 8 file:/amandatapes/daily/vtape8
slot 9 file:/amandatapes/daily/vtape9
slot 10 file:/amandatapes/daily/vtape10

By using a different name for statefile above, we do not interfere with the notion of current tape etc from the normal daily configuration. (This would not work with chg-disk.)

And recovering from this set of vtapes goes like:

# amrecover halfmirror -d halfmirror
Tip

If you are short in diskspace, you can get fancy, and keep only the last 5 (runspercycle) vtapes online, by removing the contents of the data directory in each vtape that is older than runspercycle, except the first pair of files (the label). Even the "TAPEEND" pair of files is not strictly needed for Amanda. An erased vtape, with correct label then looks like:

$ ls -l /amandatapes/daily/vtape4
-rw-------  1 amanda disk      10 Mar 29 00:04 00000-DAILY-04
-rw-------  1 amanda disk   32768 Mar 29 00:04 00000.DAILY-04