Backup server (old): Difference between revisions

From wiki.zmanda.com
Jump to navigation Jump to search
(Using a RAIT moved to a separate page)
(deprecate)
 
(22 intermediate revisions by 8 users not shown)
Line 1: Line 1:
==amanda.conf==
{{Deprecated|See [[Getting Started with Amanda]] instead}}
== Server configuration file - amanda.conf==
 
Please see {{man|5|amanda.conf}} for information on amanda.conf parameters.
 
* dumptypes
* dumptypes
* dumpcycle
* [[dumpcycle]]
* [[tapecycle]]
* [[runspercycle]]
* estimate timeouts
* estimate timeouts
* [[Tapetype definitions]]
* [[Tapetype definitions]]


==disklist==
==Client configuration file - disklist==
==tapelist==
The [[disklist]] file on the Amanda server describes the list of disk list entries ([[DLE]]s, usually directories or filesystem devices) to be backed up by the Amanda configuration.  A number of parameters can be specified for each DLE either directly in the disklist or in a dumptype definition in amanda.conf.  For more information on the disklist, see {{man|8|amanda}}.
==Exclude lists==
 
==[[Exclude and include lists]]==
 
==POSIX filenames in configuration files==
 
Amanda 2.5.1 supports the POSIX path naming standard for all filenames in dumptype configuration in {{man|5|amanda.conf}} and disklist configuration file.
 
This feature makes configuration of Amanda clients running Windows and Amanda clients that use international character sets possible.
 
Any filename referenced, including include file names, may contain any character except a NULL ('\0') character. Actual path interpretation is system dependent. POSIX file name rules are also allowed when specifying configuration file tags.


This part has been moved to a separate page.
For example we use spaces in a dumptype name in the examples below.
See [[Exclude and include lists]].
 
===Specifying Special characters===
Pathnames with special characters must be enclosed in double quotes ("). Some unprintable charactes are given special escape sequences.
 
{| style="text-align:left;" border="1"
|+'''Escape sequences'''
! align=center width="25%" | Sequence
! width="30" | Character
|-
|align=center width="25%" | \n || Newline
|-
|align=center width="25%" | \r || Carrage Return
|-
|align=center width="25%" | \t || Tab
|-
|align=center width="25%" | \f || Formfeed
|-
|align=center width="25%" | \" || Double Quote
|}
 
 
'''Example: Spaces in a dumptype name, exclude list, include list in amanda.conf'''
  define dumptype "user-tar (a1)" {
    root-tar
    comment "User partition dumped with tar and a funny dumptype name"
    priority medium
    exclude "diskfile b*"
    include list "/tmp/include list with spaces in file name"
  }
 
'''Example: Disklist file entries with spaces in backup directory and dumptype names'''
  localhost "/tmp/disk name with spaces" user-tar
  localhost "/tmp/disk name with\nnewline" "user-tar (a1)"
 
'''Example: Using file names with spaces in Amrecover sub-commands'''
  setdisk "/tmp/disk name with\nnewline"
  add "diskfile *"
  delete "diskfile a1"


== Device configuration==
== Device configuration==
Line 23: Line 75:
The speed is currently unused.
The speed is currently unused.


AMANDA provides the [[amtapetype]] utility to calculate the size of a tape, to generate a "tapetype" entry for your amanda.conf.
AMANDA provides the {{man|8|amtapetype}} utility to calculate the size of a tape, to generate a "tapetype" entry for your amanda.conf.


Specifying the appropriate tape device, but beware that it may take many hours to run (it fills the tape twice ...). Make sure you do not use hardware compression, even if you plan to use hardware compression in the future. amtapetype writes random data to tape, and random data will expand instead of compressing, therefore you'll get an estimate that's smaller than expected.
Specifying the appropriate tape device, but beware that it may take many hours to run (it fills the tape twice ...). Make sure you do not use hardware compression, even if you plan to use hardware compression in the future. amtapetype writes random data to tape, and random data will expand instead of compressing, therefore you'll get an estimate that's smaller than expected.
Line 36: Line 88:


===RAIT===
===RAIT===
Currently it is only integrated with the chg-manual changer script


RAIT is an acronym for "Redundant Array of Inexpensive Tapes", where data is striped over several tape drives, with one drive writing an exclusive-or-sum of the others which can be used for error recovery. Any one of the data streams can be lost, and the data can still be recovered.
RAIT is an acronym for "Redundant Array of Inexpensive Tapes", where data is striped over several tape drives, with one drive writing an exclusive-or-sum of the others which can be used for error recovery. Any one of the data streams can be lost, and the data can still be recovered.
Line 43: Line 93:
This means that a 3-drive RAIT set will write 2 "data" streams and one "parity" stream, and give you twice the capacity, twice the throughput, and the square of the failure rate (i.e. a 1/100 failure rate becomes 1/10,000, since a double-tape failure is required to lose data).
This means that a 3-drive RAIT set will write 2 "data" streams and one "parity" stream, and give you twice the capacity, twice the throughput, and the square of the failure rate (i.e. a 1/100 failure rate becomes 1/10,000, since a double-tape failure is required to lose data).


Similarly, a 5-drive RAIT set will give you 4 times the capacity, 4 times the throughput (with sufficient bus bandwidth), and the square of the failure rate.
This means you can back up partitions as large as twice or four times your tape size with Amanda, with higher reliability and speed.
 
This means you can back up partitions as large as four times your tape size with AMANDA, with higher reliability and speed.
 
====Using a RAIT====
 
This section has been moved to a separate section.
 
See: [[Rait]].
 
====Disaster Recovery====
 
To assist in disaster recovery (as well as changer scripts) the AMANDA package now also includes amdd, which is a simple dd(1) replacement which supports (only) the "if=xxx", "of=xxx", "bs=nnn[kMb]" "skip=nnn" and "count=nnn" options, but which can read and write RAIT tapesets.
 
Using [[amdd]] and your usual AMANDA unpack instructions will suffice for disaster recovery from RAIT tape-sets.


RAIT can also be used to mirror a backup to two drives, even when one of them is a virtual tape and the other is a real tape.


See [[Rait]] for a more complete description.


===File driver/Disk backups===
===File driver/Disk backups===
Stefan G. Weichinger, November - December, 2003 ; minor updates in April, 2005.
====Introduction====


Since release 2.4.3 AMANDA supports the usage of a output driver called "file". See man amanda, section OUTPUT DRIVERS, for more information on its implementation. As the name suggests, this driver uses files as virtual (or file) tapes. Once created and labeled, these file tapes can be selected and changed with the standard tape-changer-interface of the AMANDA server.
This driver uses files on disk as virtual tapes. Amanda can write to and read from virtual tapes, just like real tapes. A bunch of virtual tapes can even be manipulated with a changer.


Possible Uses
Possible Uses
* Test installations: You can easily explore the rich features of AMANDA on systems without tape drives.
* Inexpensive installations: Without buying a tape drive you can enjoy the benefits of AMANDA and backup to a bunch of harddisks. You can create CD/DVD-sized backups which you can burn onto optical disks later.
* disk-based installations: You can use the file-driver to backup onto a set of file tapes hosted on a bunch of hard-disks or a RAID-system. Combined with another AMANDA-configuration that dumps the file tapes to real tapes, you can provide reliable backup with faster tapeless recovery. This is called "disk-to-disk-to-tape"-backup by some people today.
====Setup====
This guide assumes you have setup the basic AMANDA-services as described in Amanda Installation Notes
The configuration in this HOWTO is called "daily". The file tapes are also called vtapes in this document, which stands for "virtual tapes".


Please be sure to understand the differences between holding disks and file tapes. The two serve different purposes; holding disks allow for parallelism of multiple DLE's being backed up while file tapes are a replacement for physical tapes.
* Test installations: You can easily explore the rich features of Amanda on systems without tape drives.
* Inexpensive installations: Without buying a tape drive you can enjoy the benefits of Amanda and backup to a bunch of internal or external harddisks connected with Firewire or USB. You can create CD/DVD-sized backups which you can burn onto optical disks later.
* Disk-based installations: You can use the file driver to backup onto a set of virtual tapes hosted on a bunch of hard-disks or a RAID-system. Combined with another Amanda configuration that dumps the virtual tapes to real tapes, you can provide reliable backup with faster tapeless recovery. This is called "disk-to-disk-to-tape" backup by some people today.  


Before beginning you will need to decide on (a) dedicated part(s) of your hard disk(s) for your file tape storage. While this space could be spread among several file systems and hard disks, I recommend to dedicate at least a specific partition, better a specific physical harddisk to the task of keeping your vtapes. The use of a dedicated disk will speed things up definitely.
See [[File driver]] for a more complete description of virtual tapes, and their use.
 
The disk space you dedicate for your vtapes should NOT be backed up by AMANDA. Also, for performance reasons there should be NO holding disks on the same partition as the vtapes, preferably not even on the same physical drive.
 
If you only have one harddisk, it will work out, too, but you will suffer low performance due to massive head-moving in your harddisk, resulting from copying data between the filesystems.
 
* Prepare the filesystem(s) used for the tapes. Decide on where to put your files, create the appropriate partition(s) and
filesystem(s) and mount them. In our example we have the dedicated partition hdc1, mounted on /amandatapes for vtape storage.
 
$ mount
[...]
/dev/hdc1 on /amandatapes type reiserfs (rw)
[...]  
Make sure there is space left. Determine the amount of space you will use.
 
$ df -h /amandatapes
Filesystem      Size  Used  Avail  Use%  Mounted on
/dev/hdc1        20G    0G    20G    0%  /amandatapes
 
In our example we have 20GB diskspace left on /amandatapes.
 
* Determine length and number of tapes. After deciding on the number of vtapes you want to create, evenly allocate the
available space among them.
 
Look at the following rule of thumb: As many filesystems exhibit dramatically reduced performance when they are nearly full I have chosen to allocate only 90% of the available space. So we have:
 
      (Available Space * 0.9) >= tapelength * tapecycle
 
This is a very conservative approach to make sure you don´t suffer any performance drop due to a nearly-full-filesystem. As it is
uncommon for AMANDA to fill, or almost fill an entire tape you may also wish to use more space than that.  So you could determine
possible combinations of tapelength/tapecycle with the more general formula:
 
      Available Space >= tapelength * tapecycle
 
In our example we take the conservative approach:
:* 20 GB * 0.9 = 18 GB to use
and so we could create the following combinations:
:* 18 GB = 18 GB * 1
:* 18 GB = 9 GB * 2
:* 18 GB = 6 GB * 3
:* 18 GB = 3 GB * 6
:* 18 GB = ......... you get the picture.
 
Using only one tape is generally considered a bad idea when it comes to backup, so we should use at least 3 tapes (for testing purposes), better 6 or more tapes.
:* 18 GB = 3 GB * 6
so we get the value 3 GB for the tapelength if we want to use 6 tapes.
 
* Create a tapetype definition. Add a new tapetype definition similar to the following to your amanda.conf. I named my definition "HARD-DISK". Choose whatever name you consider appropriate.
 
define tapetype HARD-DISK {
      comment "Dump onto hard disk"
      length 3072 mbytes # specified in mbytes to get the exact size of 3GB
}
You don´t have to specify the parameter speed (as it is commonly listed in tapetype definitions and reported by the program amtapetype). AMANDA does not use this parameter right now.
 
There is also an optional parameter filemark, which indicates the amount of space "wasted" after each tape-listitem. Leave it blank and AMANDA uses the default of 1KB.
 
* Think about tapechangers.
 
As you will use a set of vtapes, you have to also use a kind of vtape-changer. There are several tape-changer-scripts included in the AMANDA. Read more about tape-changer-scripts in AMANDA Tape Changer Support. Right now there are two scripts that can be used with vtapes. These scripts take different approaches to the handling of tapes.
:*The script chg-multi handles many drives with a tape in each drive.
:*The script chg-disk handles a library with one drive and multiple tapes.
 
So with vtapes you could look at it this way: chg-multi simulates multiple tape drives with one tape in each drive. chg-disk simulates one tape-library with multiple tapes in. As chg-multi exists for a much longer time than chg-disk, it is still used in many AMANDA-vtape-installations.
 
Contrary to chg-multi, which is a generic changer-script that must be somewhat adjusted to the use of the file-driver, chg-disk offers exactly the behavior needed for handling vtapes. IMHO the approach is much more logical, so I recommend to use chg-disk in new AMANDA-vtape-installations.
   
      To use chg-disk you need to have at least amanda-2.4.4p1-20031202.
 
Choose the one that fits your way of vtape-handling and -maintenance.  In this HOWTO I only cover the use of chg-disk. Usage of
chg-multi is pretty similar and will maybe covered in a later version of this document.
 
* Set up your tape-config. In the general section of amanda.conf, you have to set the parameters tapecycle , tapetype , tpchanger , changerfile , tapedev , rawtapedev and changerdev.
 
$ vi /usr/local/etc/amanda/daily/amanda.conf
...
 
tapecycle 6
tapetype HARD-DISK
tpchanger "chg-disk"
changerfile "/usr/local/etc/amanda/daily/changer"
tapedev  "file:/amandatapes/daily"
This reflects the use of your defined tapetype.  The parameter tapecycle tells AMANDA how much tapes can be used, Set this value according to the number of tapes you want to use. The parameter tapetype , points to the tapetype definition you have created before.  The parameter tpchanger tells AMANDA to use the generic tape-changer-script to handle the vtapes. You can think of it as a virtual tape-changer-device.
 
The parameter changerfile is used to give chg-disk the "prefix" for the "%s-changer, %s-clean, %s-slot" files it needs. Use something like "changer" in your config-dir. Please note that this file does NOT have to exist, but it won't hurt anyway.  The parameter tapedev tells the chg-disk-script where the root-dir for your vtapes is.
 
In our example the vtape-files go to /amandatapes.  To separate multiple configurations, we decided to use subdirectories according to the configuration name "daily".
      The parameter changerdev is NOT needed with chg-disk as it is not parsed by chg-disk.
 
* Create the virtual tapes.
 
      Gene Heskett has committed a shell-script which creates and labels the vtapes in one step. Stefan G. Weichinger will
      generalize this script and contribute it, this script will just read your settings in amanda.conf and create the
      appropriate vtape-directories.
 
Now you have to create the tape-directories. chg-disk needs a directory structure like:
 
slot_root_dir -|
                |- info
                |- data -> slot1/
                |- slot1/
                |- slot2/
                |- ...
                |- slotn/
where 'slot_root_dir' is the tapedev 'file:xxx' parameter and 'n' is the tapecycle parameter.
 
So in our example we do:
$ mkdir /amandatapes/daily
for the 'slot_root_dir' and
    $ mkdir /amandatapes/daily/slot1
    $ mkdir /amandatapes/daily/slot2
      .... 
 
for the virtual slots that will later contain the vtapes. If you have many vtapes to create and their names follow a pattern you may be able to do them all with a single loop such as:
 
$ for n in 1 2 3 4 5 6 7 8 9 10 11 12
> do
>    mkdir /amandatapes/daily/slot${n}
> done
Create the info-file:
$ touch /amandatapes/daily/info
 
and link the first slot to the data-file (to "load" the vtape into the first slot):
$ ln -s /amandatapes/daily/slot1 /amandatapes/daily/data
 
Make sure the AMANDA-user has write-permissions on these directories:
$ chown -R amanda_user /amandatapes
$ chgrp -R amanda_group /amandatapes
$ chmod -R 750 /amandatapes
* Label the virtual tapes. As the virtual tapes are handled just like physical tapes by the AMANDA-Server they have to be
labeled before use.
 
$ amlabel daily daily1 slot 1
  ....
$ amlabel daily daily2 slot 2
  .... 
 
If you have many vtapes to label and their names follow a pattern you may be able to do them all with a single loop such as:
 
$ for n in 1 2 3 4 5 6 7 8 9 10 11 12
> do
>    amlabel daily daily${n} slot ${n}
> done
Label all your created tapes according to the "labelstr"-parameter in your amanda.conf. Consult the amlabel-man-page for details.
 
* Test your setup with amcheck. Run amcheck daily (or, more general, amcheck <config>) and look for anything AMANDA complains about.
 
A proper output looks like:
 
$ amcheck daily
Amanda Tape Server Host Check
--
Holding disk /amhold: 6924940 KB disk space available,
that's plenty
amcheck-server: slot 2: date 20031115 label daily02
(exact label match)
NOTE: skipping tape-writable test
Tape daily02 label ok
Server check took 0.377 seconds
 
Recheck your files if errors occur.
 
====Recovery====
 
Recovering files from vtapes is very similar to recovering files from a "real" tapechanger. Make sure you read the chapter Restore.
I will simply paste a amrecover-session here (provided by JC Simonetti, author of chg-disk):
 
# /usr/local/amanda/sbin/amrecover woo
AMRECOVER Version 2.4.4p3. Contacting server on backupserver.local ...
220 backupserver AMANDA index server (2.4.4p3) ready.
200 Access OK
Setting restore date to today (2004-10-08)
200 Working date set to 2004-10-08.
Scanning /BACKUP2/holding...
Scanning /BACKUP/holding...
200 Config set to woo.
200 Dump host set to backupserver.local.
Trying disk /tmp ...
$CWD '/tmp/RECOVER' is on disk '/tmp' mounted at '/tmp'.
200 Disk set to /tmp.
Invalid directory - /tmp/RECOVER
amrecover> sethost backupserver.local
200 Dump host set to backupserver.local.
amrecover> setdisk /
200 Disk set to /.
amrecover> cd /etc
/etc
amrecover> add passwd
Added /etc/passwd
amrecover> list
TAPE B3_14 LEVEL 0 DATE 2004-09-26
        /etc/passwd
amrecover> extract
Extracting files using tape drive file:/BACKUP2/slots/ on host
backupserver.local. The following tapes are needed: B3_14
Restoring files into directory /tmp/RECOVER
Continue [?/Y/n]? Y
Extracting files using tape drive file:/BACKUP2/slots/ on host
backupserver.local. Load tape B3_14 now
Continue [?/Y/n/s/t]? Y
. /etc/passwd
amrecover> quit
200 Good bye. 
 
Nothing spectacular? The trick is this: When AMANDA asks you
 
Load tape B3_14 now
Continue [?/Y/n/s/t]? 
 
you have to run the following in a second terminal:
 
$ amtape woo slot 14
amtape: changed to slot 14 on file:/BACKUP2/slots/
 
This step is necessary to load the proper tape into your virtual changer. Let me express this in a more general way:
 
When amrecover prompts for the tape it needs to restore the files you requested, you have to "load" the tape it requests. The recommended way to do this is to use amtape. The options that make sense in this context are:
 
# amtape
Usage: amtape <conf> <command>
        Valid commands are:
                [...]
                slot <slot #>        load tape from slot <slot #>
                [...]
                label <label>        find and load labeled tape
                [...]
If you know which slot contains the requested tape (for example, if you have tape daily01 in slot 1, tape daily02 in slot 2, and so on) you may use the first option. If you just know the label of the tape you need, use the second option.
 
To continue the upper example:
 
amtape woo slot 14         # option 1 OR
amtape woo label B3_14 # option 2
 
amtape will return something like:
 
amtape: label B3_14 is now loaded. 
 
After this you can return to your amrecover-session and continue restoring your files.
 
Please be aware of the fact reported by JC Simonetti: " I have never never used the "settape" command of amrecover [with chg-disk] since there's some problems with it (tape not loaded correctly, or impossible to change from tape to tape when restoring data shared accross multiple tapes...) "
 
!!NEW!!
 
Since Amanda 2.4.3 you can let amrecover use the complete changer instead of the currently loaded tape too.
No need to open a second window to load the correct tape.
 
For this add in [[amanda.conf]]:
 
amrecover_do_fsf  yes
amrecover_check_label  yes
amrecover_changer  "changer"
 
With the last line we give the changer a name, which we can use instead of the tape device in amrecover,
when starting the command:
 
# /usr/local/amanda/sbin/amrecover woo -d changer
 
or from inside with the '''settape''' command or even:
 
Extracting files using tape drive file:/BACKUP2/slots/ on host
backupserver.local. Load tape B3_14 now
Continue [?/Y/n/s/t]? '''t'''
New tape device [?]: '''changer'''
Using tape "changer" from server backupserver.local.
Continue [?/Y/n/s/t]? '''y'''


==Server-side and Client-side encryption==
==Server-side and Client-side encryption==
*a new dumptype option, encrypt is added.
*specify either client or server side in the dumptype (not both):
**encrypt client or encrypt server
*specify client side encryption program:
**client_encrypt  "your encryption program"
***a sample encryption/decryption program amcrypt is provided. amcrypt is a wrapper of aespipe.
***espipe supports AES128, AES192 and AES256 and it uses SHA-256, SHA-384 and SHA-512 respectively.
***any encryption/decryption program can be used as long as it reads from stdin and writes to stdout.
**client_decrypt_option "decrypt parameter" #default to -d
*specify server side encryption program:
**server_encrypt "your encryption program"
***can use amcrypt as in the case of client encryption.
**server_decrypt_option "decrypt parameter" #default to -d
* The logic assumes compression then encryption during backup(thus decrypt then uncompress during restore). Specifying client-encryption and  server-compression is not supported
* dumptype sample:
define dumptype custom-tar {
      global
      program "GNUTAR"
      comment "root partitions dumped with encryption"
      compress client fast
      encrypt  server
      server_encrypt "/usr/local/sbin/amcrypt"
      server_decrypt_option "-d"
      index
      priority low
}
* The code is partially based on Matthieu Lochegnies's custom compress patch and Stefan G. Weichinger's amgtar script.
* Code has been commited to the sourceforge CVS, rpm can be downloaded from http://www.zmanda.com/downloads.html
===Additional packages needed===
* aespipe http://loop-aes.sourceforge.net/aespipe/aespipe-v2.3b.tar.bz2 and the bz2aespipe-wrapper that comes with it. It gets patched as described later.
* the wrapper-script amcrypt, as listed below,
* GNU-PG http://www.gnupg.org/(en)/download/index.html. This should be part of most current operating systems already.
===Setup===
* Configure and compile aespipe:
tar -xjf aespipe-v2.3b.tar.bz2
cd aespipe-v2.3b
./configure
make
make install
* Generate and store the gpg-key for the AMANDA-user:
# taken from the aespipe-README
head -c 2925 /dev/random | uuencode -m - | head -n 66 | tail -n 65 | \
gpg --symmetric -a > ~amanda/.gnupg/am_key.gpg
*This will ask for a passphrase. Remember this passphrase as you will need it in the next step.
Store the passphrase inside the home-directory of the AMANDA-user and protect it with proper permissions:
echo my_secret_passphrase > ~amanda/.am_passphrase
chown amanda:disk ~amanda/.am_passphrase
chmod 700 ~amanda/.am_passphrase
*We need this file because we don't want to have to enter the passphrase manually everytime we run amdump. We have to patch bz2aespipe to read the passphrase from a file. I have called that file ~amanda/.am_passphrase.
*Store the key and the passphrase in some other place as well, without these information you can't access any tapes that have been encrypted with it (this is exactly why we are doing all this, isn't it? ;) ).
* create amcrypt(or it will available in sourceforge and the rpms) as below:
#!/bin/sh
#
# Original wrapper by Paul Bijnens
#
# worked by Stefan G. Weichinger
# to enable gpg-encrypted dumps via aespipe
# also worked by Matthieu Lochegnies for server-side encryption
prefix=/usr/local
exec_prefix=${prefix}
sbindir=${exec_prefix}/sbin
AMANDA_HOME=~amanda
AM_AESPIPE=${exec_prefix}/sbin/amaespipe
AM_PASSPHRASE=$AMANDA_HOME/.am_passphrase
$AM_AESPIPE "$@" 3< $AM_PASSPHRASE
rc=$?
exit $rc
* create amaespipe(or it will available in sourceforge and the rpms) which is based on wrapper-script bz2aespipe, which comes with the aespipe-tarball:
#! /bin/sh
# FILE FORMAT
# 10 bytes: constant string 'bz2aespipe'
# 10 bytes: itercountk digits
# 1 byte: '0' = AES128, '1' = AES192, '2' = AES256
# 1 byte: '0' = SHA256, '1' = SHA384, '2' = SHA512, '3' = RMD160
# 24 bytes: random seed string
# remaining bytes are bzip2 compressed and aespipe encrypted
# These definitions are only used when encrypting.
# Decryption will autodetect these definitions from archive.
ENCRYPTION=AES256
HASHFUNC=SHA256
ITERCOUNTK=100
AMANDA_HOME=~amanda
WAITSECONDS=1
GPGKEY=""$AMANDA_HOME/.gnupg/am_key.gpg"
FDNUMBER=3
PATH=/usr/bin:/usr/local/bin
export PATH
if test x$1 = x-d ; then
    # decrypt
    n=`head -c 10 - | tr -d -c 0-9a-zA-Z`
    if test x${n} != xbz2aespipe ; then
        echo "bz2aespipe: wrong magic - aborted" >/dev/tty
        exit 1
    fi
    itercountk=`head -c 10 - | tr -d -c 0-9`
    if test x${itercountk} = x ; then itercountk=0; fi
    n=`head -c 1 - | tr -d -c 0-9`
    encryption=AES128
    if test x${n} = x1 ; then encryption=AES192; fi
    if test x${n} = x2 ; then encryption=AES256; fi
    n=`head -c 1 - | tr -d -c 0-9`
    hashfunc=SHA256
    if test x${n} = x1 ; then hashfunc=SHA384; fi
    if test x${n} = x2 ; then hashfunc=SHA512; fi
    if test x${n} = x3 ; then hashfunc=RMD160; fi
    seedstr=`head -c 24 - | tr -d -c 0-9a-zA-Z+/`
    aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${encryption} -H ${hashfunc} -S ${seedstr} -C ${itercountk} -d
else
    # encrypt
    echo -n bz2aespipe
    echo ${ITERCOUNTK} | awk '{printf "%10u", $1;}'
    n=`echo ${ENCRYPTION} | tr -d -c 0-9`
    aesstr=0
    if test x${n} = x192 ; then aesstr=1; fi
    if test x${n} = x256 ; then aesstr=2; fi
    n=`echo ${HASHFUNC} | tr -d -c 0-9`
    hashstr=0
    if test x${n} = x384 ; then hashstr=1; fi
    if test x${n} = x512 ; then hashstr=2; fi
    if test x${n} = x160 ; then hashstr=3; fi
    seedstr=`head -c 18 /dev/urandom | uuencode -m - | head -n 2 | tail -n 1`
    echo -n ${aesstr}${hashstr}${seedstr}
    aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${ENCRYPTION} -H ${HASHFUNC} -S ${seedstr} -C ${ITERCOUNTK} -w ${WAITSECONDS}
fi
exit 0
Changes from bz2aespipe:
* Decreased WAITSECONDS: No need to wait for 10 seconds to read the passphrase.
* Removed bzip2 from the pipes: AMANDA triggers GNU-zip-compression by itself, no need to do this twice (slows down things, blows up size).
* Added options -K and -p: This enables aespipe to use the generated gpg-key and tells it the number of the file-descriptor to read the passphrase from.
   
You may set various parameters inside bz2aespipe. You may also call bz2aespipe with various command-line-parameter to choose
the encryption-algorithm, hash-function etc. . For a start I have chosen to call bz2aespipe without command-line-options.


===Plans===
This section has been moved to a separate page.


There are several TODO:
See [[Encryption]].
 
*test to see if aespipe can be replaced by gpg.
*test to see if public-key encryption works.


==Custom Compression==
==Custom Compression==
Line 544: Line 143:
==Tape hardware compression==
==Tape hardware compression==


There are multiple methods to set (turn off) hardware compression.
This section has been moved to a separate page.
 
* Using stinit command: stinit(8) command initializes the SCSI tape drives at the system startup by sending driver ioctl commands. Use "comp" field in /etc/stinit.def configuration file to configure hardware compression for each type of tape.
 
* Use mt(1) command to turn off hardware compression at boot time.  For example:
# mt -f /dev/st0 compression 0
 
Please note that compression information is not stored on the tapes ident header block until the tape has been written to.


===Procedure for turning off compression and labelling tapes===
See:  [[Hardware compression]].
* Label the tape
* Rewind the tape
* Read the label to a file using dd command
* Turn off tape compression using mt(1) command. See above.
* Re-write the label block and write more /dev/zero blocks to flush its buffers.

Latest revision as of 23:48, 1 January 2011

Deprecated
See Getting Started with Amanda instead

Server configuration file - amanda.conf

Please see amanda.conf(5) for information on amanda.conf parameters.

Client configuration file - disklist

The disklist file on the Amanda server describes the list of disk list entries (DLEs, usually directories or filesystem devices) to be backed up by the Amanda configuration. A number of parameters can be specified for each DLE either directly in the disklist or in a dumptype definition in amanda.conf. For more information on the disklist, see amanda(8).

Exclude and include lists

POSIX filenames in configuration files

Amanda 2.5.1 supports the POSIX path naming standard for all filenames in dumptype configuration in amanda.conf(5) and disklist configuration file.

This feature makes configuration of Amanda clients running Windows and Amanda clients that use international character sets possible.

Any filename referenced, including include file names, may contain any character except a NULL ('\0') character. Actual path interpretation is system dependent. POSIX file name rules are also allowed when specifying configuration file tags.

For example we use spaces in a dumptype name in the examples below.

Specifying Special characters

Pathnames with special characters must be enclosed in double quotes ("). Some unprintable charactes are given special escape sequences.

Escape sequences
Sequence Character
\n Newline
\r Carrage Return
\t Tab
\f Formfeed
\" Double Quote


Example: Spaces in a dumptype name, exclude list, include list in amanda.conf 

 define dumptype "user-tar (a1)" {
    root-tar
    comment "User partition dumped with tar and a funny dumptype name"
    priority medium
    exclude "diskfile b*"
    include list "/tmp/include list with spaces in file name"
 }

Example: Disklist file entries with spaces in backup directory and dumptype names 

 localhost "/tmp/disk name with spaces" user-tar
 localhost "/tmp/disk name with\nnewline" "user-tar (a1)"

Example: Using file names with spaces in Amrecover sub-commands 

 setdisk "/tmp/disk name with\nnewline"
 add "diskfile *"
 delete "diskfile a1"

Device configuration

Tapetypes

Tapetype definitions are specified in amanda.conf configuration file. The tapetype definition provides AMANDA how much it is supposed to be able to store in a tape (length), how much space is wasted at the end of a dump image with the EOF mark (filemark) and how fast the tape unit is (speed).

The most important parameter is length, since AMANDA may decide to delay a backup if length is too small, but, if it is too large, AMANDA may end up leaving dumps in the holding disk or having to abort some dump.

Filemark is important if you have many disks, particularly with small incremental backups. The space wasted by so many filemarks may add up and considerably modify the available tape space.

The speed is currently unused.

AMANDA provides the amtapetype(8) utility to calculate the size of a tape, to generate a "tapetype" entry for your amanda.conf.

Specifying the appropriate tape device, but beware that it may take many hours to run (it fills the tape twice ...). Make sure you do not use hardware compression, even if you plan to use hardware compression in the future. amtapetype writes random data to tape, and random data will expand instead of compressing, therefore you'll get an estimate that's smaller than expected.

Some tapetype definitions are available here.

Changers

This part has been moved to a separate page.

See Changers.

RAIT

RAIT is an acronym for "Redundant Array of Inexpensive Tapes", where data is striped over several tape drives, with one drive writing an exclusive-or-sum of the others which can be used for error recovery. Any one of the data streams can be lost, and the data can still be recovered.

This means that a 3-drive RAIT set will write 2 "data" streams and one "parity" stream, and give you twice the capacity, twice the throughput, and the square of the failure rate (i.e. a 1/100 failure rate becomes 1/10,000, since a double-tape failure is required to lose data).

This means you can back up partitions as large as twice or four times your tape size with Amanda, with higher reliability and speed.

RAIT can also be used to mirror a backup to two drives, even when one of them is a virtual tape and the other is a real tape.

See Rait for a more complete description.

File driver/Disk backups

This driver uses files on disk as virtual tapes. Amanda can write to and read from virtual tapes, just like real tapes. A bunch of virtual tapes can even be manipulated with a changer.

Possible Uses

  • Test installations: You can easily explore the rich features of Amanda on systems without tape drives.
  • Inexpensive installations: Without buying a tape drive you can enjoy the benefits of Amanda and backup to a bunch of internal or external harddisks connected with Firewire or USB. You can create CD/DVD-sized backups which you can burn onto optical disks later.
  • Disk-based installations: You can use the file driver to backup onto a set of virtual tapes hosted on a bunch of hard-disks or a RAID-system. Combined with another Amanda configuration that dumps the virtual tapes to real tapes, you can provide reliable backup with faster tapeless recovery. This is called "disk-to-disk-to-tape" backup by some people today.

See File driver for a more complete description of virtual tapes, and their use.

Server-side and Client-side encryption

This section has been moved to a separate page.

See Encryption.

Custom Compression

  • compress client custom
    • Specify client_custom_compress "PROG"
    • PROG must not contain white space and it must accept -d for uncompress.
  • compress server custom
    • Specify server_custom_compress "PROG"
    • PROG must not contain white space and it must accept -d for uncompress.
  • sample dumptype:
 define dumptype custom-tar {
 global
 program "GNUTAR"
 comment "root partitions dumped with custom compression"
 compress server custom
 server_custom_compress "/usr/bin/my_gzip"
 priority low
}
  • I have tested custom compression using bzip2. Dumps works fine. Amrestore has a glitch on which

the image gets uncompressed correctly and written to a temp file but gets a broken-pipe error. I am investigating the problem.


Tape hardware compression

This section has been moved to a separate page.

See: Hardware compression.

Retrieved from ‘http://wiki.zmanda.com/index.php?title=Backup_server_(old)&oldid=7972’