How To:Set up data encryption
This article is a part of the How Tos collection.
What is needed to recover encrypted tapes
To properly retrieve any encrypted data, the following are needed:
- the key (the private key in the public-key encryption case)
- the passphrase
- the "crypt" program used. Amanda dump file header indicates what crypt program was used. For example:
AMANDA: FILE 20051215 boston.zmanda.com /usr/tmp/gpa2 lev 0 comp .gz program /bin/gtar crypt enc client_encrypt /usr/local/sbin/amcrypt client_decrypt_option -d To restore, position tape at start of file and run: dd if=<tape> bs=32k skip=1 | /usr/local/sbin/amcrypt -d | /usr/bin/gzip -dc | /bin/gtar -f...
- If the key or passphrase is lost or misplaced, the data cannot be recovered.
- There is no back-door to the encryption algorithm.
- Proper key management strategy should be in your plan before using data encryption for backup.
amcrypt 'versus' amcrypt-ossl
The amcrypt script will use a combination of aespipe and gpg (GnuPG) to perform symmetric encryption of data. This method is somewhat more difficult to implement than using openssl (through the amcrypt-ossl script), especially in non-Linux systems. First, the need to have a 65 lines key file (generated from /dev/random, piped through uuencode and then head) has a generation procedure (documented here and in the manual page) that is implementation-dependent and will not work smoothly on BSD-derived systems and possibly others. Second, because it seems that aespipe has itself issues with kernels other than Linux. Be sure to use the amaespipe script included in 3.1.1 and later if you are trying to run it amcrypt in non-linux systems. Should be safe to use that script in older amanda deployments, as a drop-in replacement, but remember to always check the resulting encrypted file, and its decryption, after any changes to the script, and use it at your own risk.
The amcrypt-ossl implementation will only require a random passphrase .am_passphrase file and will work out-of-the-box at least also in NetBSD.
Server-side and client side encryption
- a new dumptype option, encrypt is added.
- specify either client or server side in the dumptype (not both):
- encrypt client or encrypt server
- specify client side encryption program:
- client_encrypt "your encryption program"
- a sample encryption/decryption program amcrypt is provided. amcrypt is a wrapper of aespipe.
- aespipe supports AES128, AES192 and AES256 and it uses SHA-256, SHA-384 and SHA-512 respectively.
- any encryption/decryption program can be used as long as it reads from stdin and writes to stdout.
- client_decrypt_option "decrypt parameter" #default to -d
- client_encrypt "your encryption program"
- specify server side encryption program:
- server_encrypt "your encryption program"
- can use amcrypt as in the case of client encryption.
- server_decrypt_option "decrypt parameter" #default to -d
- server_encrypt "your encryption program"
- The logic assumes compression then encryption during backup(thus decrypt then uncompress during restore). Specifying client-encryption and server-compression is not supported
- dumptype sample:
define dumptype server-encrypt-fast { global program "GNUTAR" comment "dump with fast client compression and server symmetric encryption" compress client fast encrypt server server_encrypt "/usr/local/sbin/amcrypt" server_decrypt_option "-d" }
define dumptype client-encrypt-nocomp { global program "GNUTAR" comment "dump with no ompression and client symmetric encryption" compress none encrypt client client_encrypt "/usr/local/sbin/amcrypt" client_decrypt_option "-d" }
- To restore client encrypted tape. Do either:
1. take the physical tape to the client machine and do the restore on the client machine where it has the key( am_key.gpg) and passphrase(.am_passphrase).
or
2. take the key and passphrase to the server machine where the tape is located.
amcrypt - GnuPG/aespipe method
Additional packages needed
- aespipe http://loop-aes.sourceforge.net/aespipe/aespipe-v2.3b.tar.bz2 and the bz2aespipe-wrapper that comes with it. It gets patched as described later.
- the wrapper-script amcrypt, as listed below,
- GNU-PG http://www.gnupg.org/(en)/download/index.html. This should be part of most current operating systems already.
- uuencode ( sharutils*.rpm in linux distro).
Setup
- Configure and compile aespipe:
tar -xjf aespipe-v2.3b.tar.bz2 cd aespipe-v2.3b ./configure make make install
- Generate and store the gpg-key for the Amanda user:
# taken from the aespipe-README head -c 2925 /dev/random | uuencode -m - | head -n 66 | tail -n 65 | \ gpg --symmetric -a > ~amanda/.gnupg/am_key.gpg
Reading from /dev/random may take indefinitely long if kernel's random entropy pool is empty. If that happens, do some other work on some other console (use keyboard, mouse and disks).
# Additional steps to force correct ownership chown amanda:disk ~amanda/.gnupg/am_key.gpg chmod 400 ~amanda/.gnupg/am_key.gpg
Note that the options here are very linux-specific. The idea is to read from /dev/random, uuencode it, and get 65 lines of gibberish. On FreeBSD you have to use something like head -c 4000 /dev/random ... because head -c xxxx /dev/random | uuencode -m - needs to generate 65 full lines of encoded text.
- This will ask for a passphrase. Remember this passphrase as you will need it in the next step.
Store the passphrase inside the home-directory of the Amanda user and protect it with proper permissions:
echo my_secret_passphrase > ~amanda/.am_passphrase chown amanda:disk ~amanda/.am_passphrase chmod 700 ~amanda/.am_passphrase
- We need this file because we don't want to have to enter the passphrase manually everytime we run amdump. We have to patch bz2aespipe to read the passphrase from a file. I have called that file ~amanda/.am_passphrase.
- Store the key and the passphrase in some other place as well, without these information you can't access any tapes that have been encrypted with it (this is exactly why we are doing all this, isn't it? ;) ).
- create amcrypt(or it will available in sourceforge and the rpms) as below:
#!/bin/sh # # Original wrapper by Paul Bijnens # # adapted by Stefan G. Weichinger # to enable gpg-encrypted dumps via aespipe # also adapted by Matthieu Lochegnies for server-side encryption prefix=/usr/local exec_prefix=${prefix} sbindir=${exec_prefix}/sbin AMANDA_HOME=~amanda AM_AESPIPE=${exec_prefix}/sbin/amaespipe AM_PASSPHRASE=$AMANDA_HOME/.am_passphrase $AM_AESPIPE "$@" 3< $AM_PASSPHRASE rc=$? exit $rc
- create amaespipe(or it will available in sourceforge and the rpms) which is based on wrapper-script bz2aespipe, which comes with the aespipe-tarball:
#! /bin/sh # FILE FORMAT # 10 bytes: constant string 'bz2aespipe' # 10 bytes: itercountk digits # 1 byte: '0' = AES128, '1' = AES192, '2' = AES256 # 1 byte: '0' = SHA256, '1' = SHA384, '2' = SHA512, '3' = RMD160 # 24 bytes: random seed string # remaining bytes are bzip2 compressed and aespipe encrypted # These definitions are only used when encrypting. # Decryption will autodetect these definitions from archive. ENCRYPTION=AES256 HASHFUNC=SHA256 ITERCOUNTK=100 AMANDA_HOME=~amanda WAITSECONDS=1 GPGKEY=""$AMANDA_HOME/.gnupg/am_key.gpg" FDNUMBER=3 PATH=/usr/bin:/usr/local/bin export PATH if test x$1 = x-d ; then # decrypt n=`head -c 10 - | tr -d -c 0-9a-zA-Z` if test x${n} != xbz2aespipe ; then echo "bz2aespipe: wrong magic - aborted" >/dev/tty exit 1 fi itercountk=`head -c 10 - | tr -d -c 0-9` if test x${itercountk} = x ; then itercountk=0; fi n=`head -c 1 - | tr -d -c 0-9` encryption=AES128 if test x${n} = x1 ; then encryption=AES192; fi if test x${n} = x2 ; then encryption=AES256; fi n=`head -c 1 - | tr -d -c 0-9` hashfunc=SHA256 if test x${n} = x1 ; then hashfunc=SHA384; fi if test x${n} = x2 ; then hashfunc=SHA512; fi if test x${n} = x3 ; then hashfunc=RMD160; fi seedstr=`head -c 24 - | tr -d -c 0-9a-zA-Z+/` aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${encryption} -H ${hashfunc} -S ${seedstr} -C ${itercountk} -d else # encrypt echo -n bz2aespipe echo ${ITERCOUNTK} | awk '{printf "%10u", $1;}' n=`echo ${ENCRYPTION} | tr -d -c 0-9` aesstr=0 if test x${n} = x192 ; then aesstr=1; fi if test x${n} = x256 ; then aesstr=2; fi n=`echo ${HASHFUNC} | tr -d -c 0-9` hashstr=0 if test x${n} = x384 ; then hashstr=1; fi if test x${n} = x512 ; then hashstr=2; fi if test x${n} = x160 ; then hashstr=3; fi seedstr=`head -c 18 /dev/urandom | uuencode -m - | head -n 2 | tail -n 1` echo -n ${aesstr}${hashstr}${seedstr} aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${ENCRYPTION} -H ${HASHFUNC} -S ${seedstr} -C ${ITERCOUNTK} -w ${WAITSECONDS} fi exit 0
Changes from bz2aespipe:
- Decreased WAITSECONDS: No need to wait for 10 seconds to read the passphrase.
- Removed bzip2 from the pipes: AMANDA triggers GNU-zip-compression by itself, no need to do this twice (slows down things, blows up size).
- Added options -K and -p: This enables aespipe to use the generated gpg-key and tells it the number of the file-descriptor to read the passphrase from.
You may set various parameters inside bz2aespipe. You may also call bz2aespipe with various command-line-parameter to choose the encryption-algorithm, hash-function etc. . For a start I have chosen to call bz2aespipe without command-line-options.
amcrypt-ossl - OpenSSL method
Additional Packages Needed
- OpenSSL
Setup
The OpenSSL method uses only one passphrase to create a symmetric encryption of the data. Just place a (preferably random) strong key in ~amanda/.am_passphrase and add the dumptypes in amanda.conf. The security of your data depends directly on how hard to guess this passphrase is. Be sure to pick a long, random one, and that uses numbers, letters and symbols. You should use programs like pwgen or makepasswd to generate a long (more than 30 chars) random passphrase. Store it in a safe place, besides the .am_passphrase file. There is no way of recovering your data if this passphrase is lost.
Store the passphrase inside the home-directory of the Amanda user and protect it with proper permissions:
echo my_secret_passphrase > ~amanda/.am_passphrase chown amanda:disk ~amanda/.am_passphrase chmod 700 ~amanda/.am_passphrase
In amanda.conf:
define dumptype server-encrypt-fast { global program "GNUTAR" comment "dump with fast client compression and server symmetric encryption" compress client fast encrypt server server_encrypt "/usr/local/sbin/amcrypt-ossl" server_decrypt_option "-d" } define dumptype client-encrypt-nocomp { global program "GNUTAR" comment "dump with no compression and client symmetric encryption" compress none encrypt client client_encrypt "/usr/local/sbin/amcrypt-ossl" client_decrypt_option "-d" }
Now be sure to use one of these defined dumptypes in the disklist file.
Encryption Verification
(Based on information from http://www.eecs.iu-bremen.de/wiki/index.php/Amanda_%28encryption%29)
To verify backups are encrypted, inspect the dump file header. The restore line should include a reference to the decryption method specified in the dumptype.
fruit:/root$ less /amanda/daily/slot16/00001.kiwi.netlab.eecs.jacobs-university.de._.1 AMANDA: FILE 20090613000352 kiwi.netlab.eecs.jacobs-university.de / lev 1 comp N program /bin/tar crypt enc client_encrypt /usr/sbin/amcrypt-ossl client_decrypt_option -d To restore, position tape at start of file and run: dd if=<tape> bs=32k skip=1 | /usr/sbin/amcrypt-ossl -d | /bin/tar -xpGf - ... [...]
Decryption can be verified by running the restore command as specified in the dump file header. Encryption can be verified by running the same command without piping through the decryption mechanism, and verifying the result is indecipherable.
Solaris Notes
Below are notes about setting up amcrypt under Solaris 5.10 with Amanda 2.6.0p2
- Download gnupg-1.4.8 and build from sources. The sunfreeware version of gnupg did not pass the aespipe tests.
- Install aespipe: It is important to run the aespipe tests. If the aespipe tests fail, then running amcrypt might fail with "Error: gpg key file decryption failed". As root, run:
wget http://loop-aes.sourceforge.net/aespipe-latest.tar.bz2 gtar -jxf downloads/aespipe-latest.tar.bz2 cd aespipe-v2.3e/ ./configure gmake gmake tests gmake install
- Solaris: As root, install coreutils from http://www.sunfreeware.com so that head and tail have the -c argument. If this head and tail are not updated, then the amdump.1 log file might contain:
yourhost.yourdomain.com /dev/dsk/c1t0d0s0 lev 0 FAILED [data write: Broken pipe]
- Solaris: As root, edit ~dumper/amanda/sbin/amaespipe so that it uses /usr/local/bin/head and /usr/local/bin/tail. Also, change the tr command to use [:digit:] and [:alnum:]:
#! /bin/bash # # Copyright (c) 2005 Zmanda Inc. All Rights Reserved. # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License version 2 as published # by the Free Software Foundation. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY # or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License # for more details. # # You should have received a copy of the GNU General Public License along # with this program; if not, write to the Free Software Foundation, Inc., # 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # # Contact information: Zmanda Inc, 505 N Mathlida Ave, Suite 120 # Sunnyvale, CA 94085, USA, or: http://www.zmanda.com # prefix="/usr/local/amanda-2.6.0p2" exec_prefix="${prefix}" sbindir="/usr/local/amanda-2.6.0p2/sbin" amlibexecdir="/usr/local/amanda-2.6.0p2/libexec/amanda" . "${amlibexecdir}/amanda-sh-lib.sh" # add sbin and ucb dirs PATH="$PATH:/usr/sbin:/sbin:/usr/ucb" export PATH # wrapper script to use aespipe # based on bz2aespipe distributed by aespipe from # http://loop-aes.sourceforge.net/ # FILE FORMAT # 10 bytes: constant string 'bz2aespipe' # 10 bytes: itercountk digits # 1 byte: '0' = AES128, '1' = AES192, '2' = AES256 # 1 byte: '0' = SHA256, '1' = SHA384, '2' = SHA512, '3' = RMD160 # 24 bytes: random seed string # remaining bytes are aespipe encrypted # These definitions are only used when encrypting. # Decryption will autodetect these definitions from archive. ENCRYPTION=AES256 HASHFUNC=SHA256 ITERCOUNTK=100 WAITSECONDS=1 AMANDA_HOME=/users/dumper GPGKEY="$AMANDA_HOME/.gnupg/am_key.gpg" FDNUMBER=3 if test x$1 = x-d ; then # decrypt #n=`/usr/local/bin/head -c 10 - | tr -d -c 0-9a-zA-Z` n=`/usr/local/bin/head -c 10 - | /usr/bin/tr -d -c '[:alnum:]'` if test x${n} != xbz2aespipe ; then echo `_ 'bz2aespipe: wrong magic - aborted'` >/dev/tty exit 1 fi #itercountk=`/usr/local/bin/head -c 10 - | tr -d -c 0-9` itercountk=`/usr/local/bin/head -c 10 - | /usr/bin/tr -d -c '[:digit:]'` if test "x${itercountk}" = x ; then itercountk=0; fi #n=`/usr/local/bin/head -c 1 - | tr -d -c 0-9` n=`/usr/local/bin/head -c 1 - | /usr/bin/tr -d -c '[:digit:]'` encryption=AES128 if test x${n} = x1 ; then encryption=AES192; fi if test x${n} = x2 ; then encryption=AES256; fi #n=`/usr/local/bin/head -c 1 - | tr -d -c 0-9` n=`/usr/local/bin/head -c 1 - | /usr/bin/tr -d -c '[:digit:]'` hashfunc=SHA256 if test x${n} = x1 ; then hashfunc=SHA384; fi if test x${n} = x2 ; then hashfunc=SHA512; fi if test x${n} = x3 ; then hashfunc=RMD160; fi #seedstr=`/usr/local/bin/head -c 24 - | tr -d -c 0-9a-zA-Z+/` seedstr=`/usr/local/bin/head -c 24 - | /usr/bin/tr -d -c '[:alnum:]'/` #echo "aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${encryption} -H ${hashfunc} -S ${seedstr} -C ${itercountk} -d" >> /tmp/amaespipe aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${encryption} -H ${hashfunc} -S ${seedstr} -C ${itercountk} -d else # encrypt echo -n bz2aespipe echo ${ITERCOUNTK} | awk '{printf "%10u", $1;}' #n=`echo ${ENCRYPTION} | tr -d -c 0-9` n=`echo ${ENCRYPTION} | /usr/bin/tr -d -c '[:digit:]'` aesstr=0 if test x${n} = x192 ; then aesstr=1; fi if test x${n} = x256 ; then aesstr=2; fi n=`echo ${HASHFUNC} | /usr/bin/tr -d -c '[:digit:]'` hashstr=0 if test x${n} = x384 ; then hashstr=1; fi if test x${n} = x512 ; then hashstr=2; fi if test x${n} = x160 ; then hashstr=3; fi seedstr=`/usr/local/bin/head -c 18 /dev/urandom | uuencode -m - | /usr/local/bin/head -n 2 | /usr/local/bin/tail -n 1` echo -n ${aesstr}${hashstr}${seedstr} #echo "encrypt: aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${ENCRYPTION} -H ${HASHFUNC} -S ${seedstr} -C ${ITERCOUNTK} -w ${WAITSECONDS}" >> /tmp/amaespipe aespipe -K ${GPGKEY} -p ${FDNUMBER} -e ${ENCRYPTION} -H ${HASHFUNC} -S ${seedstr} -C ${ITERCOUNTK} -w ${WAITSECONDS} fi exit 0
- As dumper, create the keys, which are then encrypted:
mkdir ~/.gnupg chmod 0700 ~/.gnupg /usr/local/bin/head -c 5000 /dev/random | uuencode -m - | /usr/local/bin/head -n 66 | /usr/local/bin/tail -n 65 | gpg --symmetric -a > ~dumper/.gnupg/am_key.gpg
Note that the amcrypt man page says to get 2925 characters. This is not enough since when the text is uuencoded, the header will look like:
begin-base64 644 -
The problem is that the line is not 20 characters long. Instead, we get 5000 characters and then grab 65 lines that are at least 20 characters long. This problem ends up causing the "Error: gpg key file decryption failed" message.
- As dumper, stash the passphrased used above
echo "mysecretpassphrase" > ~dumper/.am_passphrase chown dumper ~dumper/.am_passphrase chmod 0700 ~dumper/.am_passphrase
- Check the test Amanda configuration. The disktype in amanda.conf should include:
encrypt server server_encrypt "/usr/local/amanda/sbin/amcrypt" server_decrypt_option "-d"
- Try the test dump:
amcheck test amdump test
Encrypting backup data using hardware accelerator cards
One easy way to encrypt the backup data is to use a PCI-X or PCI-e based encryption accelerator board. A variety of such cards are available and most also store keys securely in hardware, hence there is no need to write down a key or memorize a password.
One choice is Indra Networks' StorSecure family. These products are specifically designed for backup encryption, so they accelerate both compression and encryption. Further, Indra Networks has already integrated with Amanda, so it is a readymade solution. A white paper describing this solution is available.