Backup server (old): Difference between revisions
No edit summary |
|||
Line 112: | Line 112: | ||
** Disks | ** Disks | ||
==Server side encryption== | ==Server side encryption== | ||
This is based on a AMANDA-vtape-setup with the AMANDA-release 2.4.5. As this is still in the testing-stage, I have | |||
coded the home-dir of the AMANDA-user into my scripts (/var/lib/amanda). | |||
***This page needs updating*** | |||
===Additional packages needed=== | |||
* aespipe http://loop-aes.sourceforge.net/aespipe/aespipe-v2.3b.tar.bz2 and the bz2aespipe-wrapper that comes with it. It gets patched as described later. | |||
* the wrapper-script /usr/local/libexec/amgtar, as listed down below, | |||
* GNU-PG http://www.gnupg.org/(en)/download/index.html. This should be part of most current operating systems already. | |||
===Setup=== | |||
* Configure and compile aespipe: | |||
tar -xjf aespipe-v2.3b.tar.bz2 | |||
cd aespipe-v2.3b | |||
./configure | |||
make | |||
make install | |||
* Generate and store the gpg-key for the AMANDA-user: | |||
# taken from the aespipe-README | |||
head -c 2925 /dev/random | uuencode -m - | head -n 66 | tail -n 65 | \ | |||
gpg --symmetric -a > /var/lib/amanda/.gnupg/am_key.gpg | |||
This will ask for a passphrase. Remember this passphrase as you will need it in the next step. | |||
Store the passphrase inside the home-directory of the AMANDA-user and protect it with proper permissions: | |||
echo my_secret_passphrase > ~amanda/.am_passphrase | |||
chown amanda:disk ~amanda/.am_passphrase | |||
chmod 700 ~amanda/.am_passphrase | |||
We need this file because we don't want to have to enter the passphrase manually everytime we run amdump. We have to patch bz2aespipe to read the passphrase from a file. I have called that file ~amanda/.am_passphrase. | |||
Store the key and the passphrase in some other place as well, without these information you can't access any tapes that | |||
have been encrypted with it (this is exactly why we are doing all this, isn't it? ;) ). | |||
* Create the wrapper for GNU-tar: /usr/local/libexec/amgtar | |||
#!/bin/sh | |||
# | |||
# Original wrapper by Paul Bijnens | |||
# | |||
# crippled by Stefan G. Weichinger | |||
# to enable gpg-encrypted dumps via aespipe | |||
GTAR=/bin/tar | |||
AM_AESPIPE=/usr/local/bin/amaespipe | |||
AM_PASSPHRASE=/var/lib/amanda/.am_passphrase | |||
LOG=/dev/null | |||
LOG_ENABLED=1 | |||
if [ "$LOG_ENABLED" = "1" ] | |||
then | |||
LOG=/var/log/amanda/amgtar.debug | |||
date >> $LOG | |||
echo "$@" >> $LOG | |||
fi | |||
if [ "$3" = "/dev/null" ] | |||
then | |||
echo "Estimate only" >> $LOG | |||
$GTAR "$@" | |||
else | |||
echo "Real backup" >> $LOG | |||
$GTAR --use-compress-program="$AM_AESPIPE" "$@" 3< $AM_PASSPHRASE | |||
fi | |||
rc=$? | |||
exit $rc | |||
* Copy the wrapper-script bz2aespipe, which comes with the aespipe-tarball, to /usr/local/bin/amaespipe and edit it this way: | |||
#! /bin/sh | |||
# FILE FORMAT | |||
# 10 bytes: constant string 'bz2aespipe' | |||
# 10 bytes: itercountk digits | |||
# 1 byte: '0' = AES128, '1' = AES192, '2' = AES256 | |||
# 1 byte: '0' = SHA256, '1' = SHA384, '2' = SHA512, '3' = RMD160 | |||
# 24 bytes: random seed string | |||
# remaining bytes are bzip2 compressed and aespipe encrypted | |||
# These definitions are only used when encrypting. | |||
# Decryption will autodetect these definitions from archive. | |||
ENCRYPTION=AES256 | |||
HASHFUNC=SHA256 | |||
ITERCOUNTK=100 | |||
WAITSECONDS=1 | |||
GPGKEY="/var/lib/amanda/.gnupg/am_key.gpg" | |||
FDNUMBER=3 | |||
if test x$1 = x-d ; then | |||
# decrypt | |||
n=`head -c 10 - | tr -d -c 0-9a-zA-Z` | |||
if test x${n} != xbz2aespipe ; then | |||
echo "bz2aespipe: wrong magic - aborted" >/dev/tty | |||
exit 1 | |||
fi | |||
itercountk=`head -c 10 - | tr -d -c 0-9` | |||
if test x${itercountk} = x ; then itercountk=0; fi | |||
n=`head -c 1 - | tr -d -c 0-9` | |||
encryption=AES128 | |||
if test x${n} = x1 ; then encryption=AES192; fi | |||
if test x${n} = x2 ; then encryption=AES256; fi | |||
n=`head -c 1 - | tr -d -c 0-9` | |||
hashfunc=SHA256 | |||
if test x${n} = x1 ; then hashfunc=SHA384; fi | |||
if test x${n} = x2 ; then hashfunc=SHA512; fi | |||
if test x${n} = x3 ; then hashfunc=RMD160; fi | |||
seedstr=`head -c 24 - | tr -d -c 0-9a-zA-Z+/` | |||
#aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${encryption} -H ${hashfunc} -S "${seedstr}" -C ${itercountk} -d | bzip2 -d -q | |||
aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${encryption} -H ${hashfunc} -S "${seedstr}" -C ${itercountk} -d | |||
else | |||
# encrypt | |||
echo -n bz2aespipe | |||
echo ${ITERCOUNTK} | awk '{printf "%10u", $1;}' | |||
n=`echo ${ENCRYPTION} | tr -d -c 0-9` | |||
aesstr=0 | |||
if test x${n} = x192 ; then aesstr=1; fi | |||
if test x${n} = x256 ; then aesstr=2; fi | |||
n=`echo ${HASHFUNC} | tr -d -c 0-9` | |||
hashstr=0 | |||
if test x${n} = x384 ; then hashstr=1; fi | |||
if test x${n} = x512 ; then hashstr=2; fi | |||
if test x${n} = x160 ; then hashstr=3; fi | |||
seedstr=`head -c 18 /dev/urandom | uuencode -m - | head -n 2 | tail -n 1` | |||
echo -n ${aesstr}${hashstr}${seedstr} | |||
#bzip2 | aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${ENCRYPTION} -H ${HASHFUNC} -S ${seedstr} -C ${ITERCOUNTK} -w ${WAITSECONDS} | |||
aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${ENCRYPTION} -H ${HASHFUNC} -S ${seedstr} -C ${ITERCOUNTK} -w ${WAITSECONDS} | |||
fi | |||
exit 0 | |||
or apply this small patch (bz2aespipe.patch) | |||
@@ -15,3 +15,5 @@ | |||
ITERCOUNTK=100 | |||
-WAITSECONDS=10 | |||
+WAITSECONDS=1 | |||
+GPGKEY="/var/lib/amanda/.gnupg/am_key.gpg" | |||
+FDNUMBER=3 | |||
@@ -36,3 +38,4 @@ | |||
seedstr=`head -c 24 - | tr -d -c 0-9a-zA-Z+/` | |||
- aespipe -e ${encryption} -H ${hashfunc} -S "${seedstr}" -C ${itercountk} -d | bzip2 -d -q | |||
+ #aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${encryption} -H ${hashfunc} -S "${seedstr}" -C ${itercountk} -d | bzip2 -d -q | |||
+ aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${encryption} -H ${hashfunc} -S "${seedstr}" -C ${itercountk} -d | |||
else | |||
@@ -52,3 +55,4 @@ | |||
echo -n ${aesstr}${hashstr}${seedstr} | |||
- bzip2 | aespipe -e ${ENCRYPTION} -H ${HASHFUNC} -S ${seedstr} -C ${ITERCOUNTK} -T -w ${WAITSECONDS} | |||
+ #bzip2 | aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${ENCRYPTION} -H ${HASHFUNC} -S ${seedstr} -C ${ITERCOUNTK} -w ${WAITSECONDS} | |||
+ aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${ENCRYPTION} -H ${HASHFUNC} -S ${seedstr} -C ${ITERCOUNTK} -w ${WAITSECONDS} | |||
fi | |||
Patch changes: | |||
* Decreased WAITSECONDS: No need to wait for 10 seconds to read the passphrase. | |||
* Removed bzip2 from the pipes: AMANDA triggers GNU-zip-compression by itself, no need to do this twice (slows down things, blows up size). | |||
* Added options -K and -p: This enables aespipe to use the generated gpg-key and tells it the number of the file-descriptor to read the passphrase from. | |||
You may set various parameters inside bz2aespipe. You may also call bz2aespipe with various command-line-parameters | |||
to choose the encryption-algorithm, hash-function etc. . For a start I have chosen to call bz2aespipe without | |||
command-line-options. | |||
Reconfigure and recompile AMANDA (yes, I'm sorry ...): | |||
As described in How to use a wrapper you have to run configure again with the option --with-gnutar=/usr/local/libexec/amgtar, after that recompile and reinstall AMANDA. These steps are described in the mentioned document. | |||
===Plans=== | |||
There are several wishes: | |||
* Ability to switch encryption inside a dumptype. This HOWTO describes a method that enables/disables encryption for the whole installation. You might remove the amgtar-wrapper and simply link to plain GNU-tar again to disable encryption, but be aware that you also disable decryption with this step. You will hit problems when you then try to restore encrypted tapes. | |||
* Ability to switch encryption-parameters inside a dumptype. Choice of algorithm, hash-functions etc. I don't know if it makes sense to put it into a dumptype or if it would be enough to configure it once inside amaespipe (I assume the latter). | |||
* All this leads to the need to code this into AMANDA itself: new dumptype-options and corresponding calls to GNU-tar etc. inside client-src/sendbackup-gnutar.c. | |||
==Compression== | ==Compression== |
Revision as of 23:01, 10 November 2005
Simple Backup server configuration
- Create the config directory (eg. /usr/local/etc/amanda/confname) and copy the example/ files into that directory. Edit these files to be correct for your site, consulting the amanda(8) man page if necessary. You will also need to create the directory for the log and database files for the configuration to use (eg /usr/local/var/amanda/confname), and the work directory on the holding disk. These directories need to agree with the parameters in amanda.conf. Don't forget to make all these directories writable by the dump user!
Make sure that you specify the no-rewind version of the tape device in your amanda.conf file. This is a frequently encountered problem for new sites.
Note that you might want to temporarily set the option "no-record" in all your dumptypes when first installing AMANDA if you'd like to run tests of AMANDA in parallel with your existing dump scheme. AMANDA will then run but will not interfere with your current dumpdates. However, you don't want to run with "no-record" under normal operations.
- Put AMANDA into your crontab. Here's a sample:
- /etc/crontab
0 16 * * 1-5 /usr/local/sbin/amcheck -m confname 45 0 * * 2-6 /usr/local/sbin/amdump confname
This is for SunOS 4.x, which has a per-user crontab; most other systems also require a userid on each cron line. See your cron(8) for details. With these cron lines, AMANDA will check that the correct tape is in the drive every weekday afternoon at 4pm (if it isn't, all the operators will get mail). At 12:45am that night the dumps will be run.
- Put the AMANDA services into your /etc/services file. Add entries like:
- /etc/services
amanda 10080/udp amandaidx 10082/tcp amidxtape 10083/tcp
- You may choose a different port number if you like, but it must match that in the services file on the client hosts too.
- If you are running NIS (aka YP), you have to enter the AMANDA service into your NIS services database. Consult your NIS documentation for details.
- You may use the `patch-system' script, from client-src, in order to modify this file. Run it with a `-h' argument for usage.
- If you are going to use the indexing capabilities of AMANDA, follow one of the following steps:
- If your server uses inetd, then add these lines to your inetd.conf on the tape server host:
- /etc/inetd.conf
amandaidx stream tcp nowait $USER $AMINDEXD_PATH amindexd amidxtape stream tcp nowait $USER $AMIDXTAPED_PATH amidxtaped
- where $AMINDEXD_PATH and $AMIDXTAPED_PATH are the complete paths to where the amindexd and amidxtaped executables (usually :libexec_dir/amindexd and libexec_dir/amidxtaped), and USER is the AMANDA user.
- You may use the `patch-system' script, from client-src, in order to modify this file. Run it with a `-h' argument for usage.
- If your tape server uses xinetd instead of inetd, then you have to add the following two files to your xinetd-configuration (usually :/etc/xinetd.d) and edit the paths:
- /etc/xinetd.d/amandaidx
service amandaidx { socket_type = stream protocol = tcp wait = no user = $USER group = $GROUP groups = yes server = $AMINDEXD_PATH/amindexd }
- /etc/xinetd.d/amidxtape
service amidxtape { socket_type = stream protocol = tcp wait = no user = $USER group = $GROUP groups = yes server = $AMIDXTAPED_PATH/amidxtaped }
- If your tape server uses Dan Bernstein's daemontools http://cr.yp.to/daemontools.html) instead of (x)inetd, you have to create amandaidx and amidxtape services by hand.
- Create service directories:
mkdir -p $prefix/etc/amanda/supervise/amandaidx mkdir -p $prefix/etc/amanda/supervise/amidxtape
- Create service startup files and make them executable:
- /etc/amanda/supervise/amandaidx/run
#!/bin/sh exec /usr/local/bin/setuidgid amanda \ /usr/local/bin/tcpserver -DHRl0 0 10082 \ /usr/local/libexec/amindexd >/dev/null 2>/dev/null
- /etc/amanda/supervise/amidxtape/run
#!/bin/sh exec /usr/local/bin/setuidgid amanda \ /usr/local/bin/tcpserver -DHRl0 0 10083 \ /usr/local/libexec/amidxtaped >/dev/null 2>/dev/null
- Link service directories into your svscan directory:
cd /service ln -s $prefix/etc/amanda/supervise/amandaidx . ln -s $prefix/etc/amanda/supervise/amidxtape .
- If the tape server host is itself going to be backed up (as is usually the case), you must also follow the client-side install instructions below on the server host, INCLUDING setting up the file .amandahosts so that the server host lets itself in. This is a frequently encountered problem for new sites.
amanda.conf
- dumptypes
- backup cycle
- exclude lists
- estimate timeouts
- Tapetype definitions
disklist
tapelist
Device configuration
- Changers
- RAIT
- Disks
Server side encryption
This is based on a AMANDA-vtape-setup with the AMANDA-release 2.4.5. As this is still in the testing-stage, I have coded the home-dir of the AMANDA-user into my scripts (/var/lib/amanda).
***This page needs updating***
Additional packages needed
- aespipe http://loop-aes.sourceforge.net/aespipe/aespipe-v2.3b.tar.bz2 and the bz2aespipe-wrapper that comes with it. It gets patched as described later.
- the wrapper-script /usr/local/libexec/amgtar, as listed down below,
- GNU-PG http://www.gnupg.org/(en)/download/index.html. This should be part of most current operating systems already.
Setup
- Configure and compile aespipe:
tar -xjf aespipe-v2.3b.tar.bz2 cd aespipe-v2.3b ./configure make make install
- Generate and store the gpg-key for the AMANDA-user:
# taken from the aespipe-README head -c 2925 /dev/random | uuencode -m - | head -n 66 | tail -n 65 | \ gpg --symmetric -a > /var/lib/amanda/.gnupg/am_key.gpg
This will ask for a passphrase. Remember this passphrase as you will need it in the next step.
Store the passphrase inside the home-directory of the AMANDA-user and protect it with proper permissions:
echo my_secret_passphrase > ~amanda/.am_passphrase chown amanda:disk ~amanda/.am_passphrase chmod 700 ~amanda/.am_passphrase
We need this file because we don't want to have to enter the passphrase manually everytime we run amdump. We have to patch bz2aespipe to read the passphrase from a file. I have called that file ~amanda/.am_passphrase.
Store the key and the passphrase in some other place as well, without these information you can't access any tapes that have been encrypted with it (this is exactly why we are doing all this, isn't it? ;) ).
- Create the wrapper for GNU-tar: /usr/local/libexec/amgtar
#!/bin/sh # # Original wrapper by Paul Bijnens # # crippled by Stefan G. Weichinger # to enable gpg-encrypted dumps via aespipe
GTAR=/bin/tar AM_AESPIPE=/usr/local/bin/amaespipe AM_PASSPHRASE=/var/lib/amanda/.am_passphrase LOG=/dev/null LOG_ENABLED=1
if [ "$LOG_ENABLED" = "1" ] then LOG=/var/log/amanda/amgtar.debug date >> $LOG echo "$@" >> $LOG fi
if [ "$3" = "/dev/null" ] then
echo "Estimate only" >> $LOG $GTAR "$@"
else
echo "Real backup" >> $LOG $GTAR --use-compress-program="$AM_AESPIPE" "$@" 3< $AM_PASSPHRASE
fi
rc=$? exit $rc
- Copy the wrapper-script bz2aespipe, which comes with the aespipe-tarball, to /usr/local/bin/amaespipe and edit it this way:
#! /bin/sh
# FILE FORMAT # 10 bytes: constant string 'bz2aespipe' # 10 bytes: itercountk digits # 1 byte: '0' = AES128, '1' = AES192, '2' = AES256 # 1 byte: '0' = SHA256, '1' = SHA384, '2' = SHA512, '3' = RMD160 # 24 bytes: random seed string # remaining bytes are bzip2 compressed and aespipe encrypted
# These definitions are only used when encrypting. # Decryption will autodetect these definitions from archive. ENCRYPTION=AES256 HASHFUNC=SHA256 ITERCOUNTK=100 WAITSECONDS=1 GPGKEY="/var/lib/amanda/.gnupg/am_key.gpg" FDNUMBER=3
if test x$1 = x-d ; then # decrypt n=`head -c 10 - | tr -d -c 0-9a-zA-Z` if test x${n} != xbz2aespipe ; then echo "bz2aespipe: wrong magic - aborted" >/dev/tty exit 1 fi itercountk=`head -c 10 - | tr -d -c 0-9` if test x${itercountk} = x ; then itercountk=0; fi n=`head -c 1 - | tr -d -c 0-9` encryption=AES128 if test x${n} = x1 ; then encryption=AES192; fi if test x${n} = x2 ; then encryption=AES256; fi n=`head -c 1 - | tr -d -c 0-9` hashfunc=SHA256 if test x${n} = x1 ; then hashfunc=SHA384; fi if test x${n} = x2 ; then hashfunc=SHA512; fi if test x${n} = x3 ; then hashfunc=RMD160; fi seedstr=`head -c 24 - | tr -d -c 0-9a-zA-Z+/` #aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${encryption} -H ${hashfunc} -S "${seedstr}" -C ${itercountk} -d | bzip2 -d -q aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${encryption} -H ${hashfunc} -S "${seedstr}" -C ${itercountk} -d else # encrypt echo -n bz2aespipe echo ${ITERCOUNTK} | awk '{printf "%10u", $1;}' n=`echo ${ENCRYPTION} | tr -d -c 0-9` aesstr=0 if test x${n} = x192 ; then aesstr=1; fi if test x${n} = x256 ; then aesstr=2; fi n=`echo ${HASHFUNC} | tr -d -c 0-9` hashstr=0 if test x${n} = x384 ; then hashstr=1; fi if test x${n} = x512 ; then hashstr=2; fi if test x${n} = x160 ; then hashstr=3; fi seedstr=`head -c 18 /dev/urandom | uuencode -m - | head -n 2 | tail -n 1` echo -n ${aesstr}${hashstr}${seedstr} #bzip2 | aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${ENCRYPTION} -H ${HASHFUNC} -S ${seedstr} -C ${ITERCOUNTK} -w ${WAITSECONDS} aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${ENCRYPTION} -H ${HASHFUNC} -S ${seedstr} -C ${ITERCOUNTK} -w ${WAITSECONDS} fi exit 0
or apply this small patch (bz2aespipe.patch)
@@ -15,3 +15,5 @@ ITERCOUNTK=100 -WAITSECONDS=10 +WAITSECONDS=1 +GPGKEY="/var/lib/amanda/.gnupg/am_key.gpg" +FDNUMBER=3 @@ -36,3 +38,4 @@ seedstr=`head -c 24 - | tr -d -c 0-9a-zA-Z+/` - aespipe -e ${encryption} -H ${hashfunc} -S "${seedstr}" -C ${itercountk} -d | bzip2 -d -q + #aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${encryption} -H ${hashfunc} -S "${seedstr}" -C ${itercountk} -d | bzip2 -d -q + aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${encryption} -H ${hashfunc} -S "${seedstr}" -C ${itercountk} -d else @@ -52,3 +55,4 @@ echo -n ${aesstr}${hashstr}${seedstr} - bzip2 | aespipe -e ${ENCRYPTION} -H ${HASHFUNC} -S ${seedstr} -C ${ITERCOUNTK} -T -w ${WAITSECONDS} + #bzip2 | aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${ENCRYPTION} -H ${HASHFUNC} -S ${seedstr} -C ${ITERCOUNTK} -w ${WAITSECONDS} + aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${ENCRYPTION} -H ${HASHFUNC} -S ${seedstr} -C ${ITERCOUNTK} -w ${WAITSECONDS} fi
Patch changes:
- Decreased WAITSECONDS: No need to wait for 10 seconds to read the passphrase.
- Removed bzip2 from the pipes: AMANDA triggers GNU-zip-compression by itself, no need to do this twice (slows down things, blows up size).
- Added options -K and -p: This enables aespipe to use the generated gpg-key and tells it the number of the file-descriptor to read the passphrase from.
You may set various parameters inside bz2aespipe. You may also call bz2aespipe with various command-line-parameters to choose the encryption-algorithm, hash-function etc. . For a start I have chosen to call bz2aespipe without command-line-options.
Reconfigure and recompile AMANDA (yes, I'm sorry ...):
As described in How to use a wrapper you have to run configure again with the option --with-gnutar=/usr/local/libexec/amgtar, after that recompile and reinstall AMANDA. These steps are described in the mentioned document.
Plans
There are several wishes:
- Ability to switch encryption inside a dumptype. This HOWTO describes a method that enables/disables encryption for the whole installation. You might remove the amgtar-wrapper and simply link to plain GNU-tar again to disable encryption, but be aware that you also disable decryption with this step. You will hit problems when you then try to restore encrypted tapes.
- Ability to switch encryption-parameters inside a dumptype. Choice of algorithm, hash-functions etc. I don't know if it makes sense to put it into a dumptype or if it would be enough to configure it once inside amaespipe (I assume the latter).
- All this leads to the need to code this into AMANDA itself: new dumptype-options and corresponding calls to GNU-tar etc. inside client-src/sendbackup-gnutar.c.