Backup client (old)
GNUTAR
Amanda supports backup made using GNU tar command. To use tar as the backup program, define a dumptype definition with program name set to "GNUTAR"
dumptype tar-client {
....
program "GNUTAR"
}
GNU tar does not maintain information about the backups similar to dump command. Amanda maintains a file /etc/amandates file on the client. The structure of the file is similar to /etc/dumpdates and performs the same function. It contains filesystem/directory
name, dump level and dump date.
If /etc directory is not writable by the "amanda" user on the client, administrator will have to create an empty file /etc/amandates and must be writable by "amanda" user.
Amanda estimates the size of the dump performed by the tar command.
The tar command traverses the directory hierarchy and reads the files. In order to read the files, it must run as "super user" on the client. The amanda commands "calcsize" and "runtar" are installed as setuid "root" so that it can reads the files and directories.
Unlike dump command, GNU tar allows users to backup sub-directories. It also allows administrators to exclude and include files in the backup. Amanda uses --exclude and --exclude-from tar operations for this feature. Please see Excluding lists section for more information on excluding files from backup.
Amanda creates tar snapshot file when a dump level 0 is performed. The tar snapshot file is usually created in /var/lib/amanda/gnutar-lists/ directory on the client. It contains backup timestamp and list of files backed up. This file is used when incremental dumps are done in the future. Amanda uses --listed-incremental gnu tar option to create the file.
DUMP
Amanda uses native filesystem dump program to do filesystem backups. Dumptype definition in amanda.conf should have program set to "DUMP":
dumptype dump-client {
....
program "DUMP"
}
Amanda automatically runs the native filesystem dump program. It will run xfsdump for XFS filesystems, vxdump for Veritas filesystems, vdump for AdvFS (Tru64), dump for other filesystems.
DUMP program only works for whole filesystem backups and it is not possible to exclude files.
Amanda uses /etc/dumpdates file (created by dump program) for tracking backups. Filesystem name, Date stamp and last dump level are kept in this file.
Windows client
Installation
AMANDA is able to back up Microsoft Windows shared disks by using Samba, a package that implements a SMB client and server for Unix.
After building and installing Samba, AMANDA must be configured with support for smbclient. AMANDA will automatically find smbclient if it is in your PATH when you run configure, or you may add the following argument:
--with-smbclient=/full/path/to/smbclient
Configuration
Once AMANDA and Samba are installed, the only difference between a Unix client/disk and PC client/share is in how the backup disks are specified in the file disklist. For each PC share, the entry lists the 'samba server' host (where the patched Samba software is installed) and the disk field is the share name. The remaining fields are like any other DLE.
A user must be created on the PC with full access rights (read/write) to the share. AMANDA, via the Samba server, will connect to the PC via this user. If the user does not have full access, incremental backups will not work and the whole share will be backed up every time (the archive bits are never reset).
The file /etc/amandapass must be created by hand. It contains share name to user name, password and workgroup mapping. Each line consists of two or three fields, separated by whitespace:
- Share name followed by optional directory. You have to use forward slashes (/), not backslashes (\). This must match the disklist entry exactly (case sensitive). This may be asterisk (*) to match all remaining shares for this Samba server. The first match in the file is used, so specific entries must be listed first. The directory is appended to the share name as full MS network path. Like //thepc/c$/mydir. No blanks are allowed in directory!
- User name and password. Separated by a percent sign (%). See the description of the -U option in the manpage of smbclient. No whitespace is allowed in either the user name or password.
- Workgroup (optional).
This file must be owned by the AMANDA-user, and disallow world access privileges. Blank lines are ignored. A "#" on a line at the start of a field (including start of line) causes the rest of the line to be ignored.
Example configuration
The AMANDA client software and (patched) Samba is installed on host "pcserver". A share to be backed up called "backupc" is on PC "thepc". The share will be accessed via PC user "bozo" and password "f00bar" and does not require a workgroup.
The entry in the file disklist is:
pcserver //thepc/backupc nocomp-user-gnutar
^ samba installed unix host
^ pc host and share name
^ dumptype must include the tar option
In /etc/amandapass on the machine 'pcserver':
//thepc/backupc bozo%f00bar
If smbclient requires a workgroup specification (-W), you may add it as a third argument in the line in the file /etc/amandapass :
//thepc/backupc bozo%f00bar NTGROUP
This will cause smbclient to be invoked with -W NTGROUP.
An example dumptype in amanda.conf would be:
define dumptype nocomp-user-gnutar {
program "GNUTAR"
comment "user partitions dumped with tar and no compression"
options no-compress
priority medium
}
Essentially, the entry in disklist is a 'pseudo-disk' which contains all the relevant information needed by smbclient to backup the disk, but in a way that is compatible to AMANDA.
amcheck does a quick check to see if smbclient exists and tries to connect to the PC clients. It also checks for the existence and permissions of /etc/amandapass.
Bugs and notes
- Samba will not back up open files (such as PAGEFILE.SYS and registry files) nor Access Control List data. Furthermore, at restore time, smbclient is unable to overwrite read-only files. Hence, AMANDA+Samba is not a perfect solution for backing up (restoring, actually) system disks.
- Samba does not use the Windows Backup API, so configuring the AMANDA backup user as a member of group backup on the Windows host is useless. You will probably have to configure it as an Administrator, and make sure it can read and change permission of all files in the share.
- It seems impossible to detect when a per-user based login fails, e.g. the username doesn't have sufficient access. It connects but cannot see any files (e.g. backups do nothing). The selfcheck code isn't particularly robust in this area either, so you may get no warnings when a disk isn't being backed up. Just check to see that level 0 dumps are bigger than 64K, otherwise it means the backup was empty.
- The estimate and totals are probably a bit off since tar pads to the nearest 512 bytes after each file (I think). Not sure how much of a problem this is.
- smbclient only supports excluding a single file from the command line, not a file of patterns like GNU tar. So "exclude" is supported from a dumptype but not "exclude list".
- Also the new option "exclude append" is not yet supported with smbclient.
- Since Samba-3.0.2a smbclient supports multiple exclusion-patterns. It is one of the "Ongoing Projects" to make use of this in AMANDA. Refer to http://www.amanda.org/ongoing.php for details.
- The size estimate calculation does not use the same method as the dump, so it may be inaccurate. It also does not support any type of exclusion ("exclude" is ignored). Things are done this way because doing a simulated dump to /dev/null, like other dump programs, would take forever with current implementations of Samba.
- If you compile with support for smbclient, GNU-tar support is automatically enabled. If you aren't using the GNU-tar part, you may get warnings about the availability of /usr/local/bin/gtar (or whatever it was compiled with). These may safely be ignored, unless you enable index generation for those filesystems.
Client side encryption
pre and post- backup scripts (wrappers)
Bert de Ridder's suggestions
This is a mini-howto explaining how to control other running tasks on a server where the AMANDA software is used to backup data.
Problem : Lots of software is picky about their datafiles being backed up while the files are in use. It sometimes is even necessary to know the state of the datafiles at the moment of backup so that when restoring you know exactly *what* you are restoring. And most of the time there are dependencies between the datafiles as well (for instance, the pure datafiles and the controlfiles of an Oracle database.)
The solution is actually quite simple; you just use a custom made backupscript instead of the standard tar command. Inside this tar command, you do some necessary processing before executing the tar command and - if necessary - do some more processing. This way, you can easily stop an Oracle database, tar the files, send them to the tape server and restart the Oracle database. This of course is just an example, anything you can do in a shell script can be done.
- Create the script: This is the most important step, this script is the work horse of the solution. I've called it /bin/amandatar. You can call it whatever you want though. It's a Perl script, it may not be very pretty code, but it does the job. In the script, an example is given for the backup of a Lotus Notes Domino server.
#!/usr/bin/perl
# Tar wrapper for Amanda's tar. # use Getopt::Long qw(:config pass_through); # Obtain directory and file information from the command line.
$result = GetOptions ( 'directory=s' => \$dir, 'file=s' => \$file );
# Check whether Amanda wants to do some administrative task (eg. indexinfo # or obtain the number of bytes to be backed up) # if file = /dev/null it's an administrative task and most of the time, no extra # processing is necessary.
# What you see here is just a log of the backup start time, and – more important # the stopping of the domino server
if ( $file ne '/dev/null' )
{
if ( $dir eq '/local/notesdata' )
{
system "echo 'Start backup notes at ' >> /var/lib/amanda/runtime" ;
system "date >> /var/lib/amanda/runtime";
system ( "/etc/init.d/domino stop >> /var/lib/amanda/runtime" );
}
}
# The command line is being 'reconstructed'. Necessary because the GetOptions # call above has stripped the file and directory information. # This is what I meant with 'ugly' code ;-)
while ( $ARGV[0] ne )
{
$val = $ARGV[0] ;
unshift ( @NEWARGV, $val, ) ;
shift @ARGV;
}
while ( $NEWARGV[0] ne )
{
$val = $NEWARGV[0] ;
unshift ( @ARGV, $val ) ;
shift @NEWARGV;
}
if ( $dir ne )
{
unshift ( @ARGV, '--directory', $dir );
}
if ( $file ne )
{
unshift ( @ARGV, '--file', $file );
}
if ( $file ne '/dev/null' )
{
system "echo 'Backing up directory ' $dir >> /var/lib/amanda/runtime" ;
}
# And finally make sure tar is called :-) # (path may differ on your installation) unshift ( @ARGV , "/bin/tar" ) ;
system ( @ARGV ) ;
# Postprocessing # # If Notes backup was requested, restart the server. # Log the backup end time. #
if ( $file ne '/dev/null' )
{
if ( $dir eq '/local/notesdata' )
{
system ( "/etc/init.d/domino start >> /var/lib/amanda/runtime" );
system "echo 'End backup notes at ' >> /var/lib/amanda/runtime" ;
system "date >> /var/lib/amanda/runtime";
}
}
exit 0;
# End script
On some systems it may be necessary to setuid root the script.
- Rebuild AMANDA so that it uses your newly created script.
Download the sources, untar them to a directory. I'm sure there are lots of documents already available on how to do this, so I won't go into too much detail. (Refer to Amanda Installation Notes).
Fast Path:
/usr/local/src # tar -xvzf amanda-source.tar.gz /usr/local/src # cd amanda-version /usr/local/src/amanda-version # ./configure \ --with-user=amanda \ --prefix=/usr/local \ --exec-prefix=/usr \ --bindir=/usr/bin \ --sbindir=/usr/sbin \ --libexecdir=/usr/lib/amanda \ --with-configdir=/etc/amanda \ --with-group=disk \ --with-gnutar=/bin/amandatar \ --with-gnutar-listdir=/var/lib/amanda/gnutar-lists \ --with-tmpdir=/tmp/amanda \ --with-smbclient=/usr/bin/smbclient \ --mandir=/usr/local/man
Here, it may be necessary to adjust some paths to match your installation. This setup works on SuSE Linux (also SLES) and MacOSX although you may have to use another binary tar.
As you see, you may also "replace" the smbclient if necessary. I haven't yet tested it though. I'll leave it as an exercise for the reader <g>.
/usr/local/src/amanda-version # make /usr/local/src/amanda-version # make install
Now proceed as with a "normal" installation.
Paul Bijnens's suggestions
Currently, there is no direct support to run a program before or after a backup on a client. But there is an easy workaround by using a wrapper for GNU-tar that does the additional tasks.
Let's suppose you want to stop a database before the backup, and start it up again when the backup is finished. You have already two scripts "shutdb" and "startdb" to shutdown and startup the database.
First you have to configure AMANDA on the client to use the gnutar-wrapper instead of the real GNU-tar:
./configure ... --with-gnutar=/usr/local/bin/amgtar ...
and re-compile AMANDA. The program "amgtar" can be a simple link to the real GNU-tar-binary on clients that don't need special handling, or it can be a script.
AMANDA expects that the bytestream on stdout is the backup image, and the bytestream on stderr are messages. The stderr messages are filtered against a known set of strings, and anything unexpected is flagged as "STRANGE" in the AMANDA report. The return-codes of the program should be the same as the return-codes of GNU-tar:
- 0 = ok (backup image will be put on tape)
- 1 = not ok (backup image will not be put on tape, same level will be tried next time).
The arguments passed to the program are pretty static (see in the sources client-src/sendbackup-gnutar.c, line 483). To decide if you need to stop/start the database you have to check if:
- this run makes a backup and not a restore: look for "--create"
- this is not an estimate run: look for "--file /dev/null" (estimate) or "--file -" (real run)
- this run is for the database directory: look for "--directory /my/data/base"
In all other cases, we just pass the args and run the real GNU-tar.
Here is an example script in Bourne shell:
#!/bin/sh # # uncomment next block to follow the flow # LOG=/tmp/amanda/mytar.debug # date >> $LOG # echo "$@" >> $LOG # if [ "$3" = "/dev/null" ] # then echo "Estimate only" >> $LOG # else echo "Real backup" >> $LOG # fi # - Avoid output to stdout! (the backup stream by tar) # - Any output to stderr is flagged as "strange" by amanda # and may be used to pass error messages into the report if [ "$1" = "--create" -a "$3" = "-" -a "$5" = "/my/dir" ] then # echo "/my/dir: want to execute some progs first" >>$LOG /usr/local/bin/shutdb thedb >&2 /usr/local/bin/gtar "$@" rc=$? # echo "Finished the real backup; some postprocessing" >>$LOG /usr/local/bin/startdb thedb >&2 exit $rc else /usr/local/bin/gtar "$@" fi
Here is an example script in perl:
#!/usr/bin/perl -w
use Getopt::Long qw(:config pass_through);
my @saveopts = @ARGV;
GetOptions (
'create' => \$create,
'directory=s' => \$dir,
'file=s' => \$file,
);
@ARGV = @saveopts;
my $postproc = 0;
if ($create && $dir eq '/my/data/base' && $file ne '/dev/null') {
system '/usr/local/bin/dbshut thedb >/tmp/amanda/dbshut.debug 2>&1';
$postproc = 1;
}
unshift(@ARGV, "/usr/local/bin/gtar");
system @ARGV;
my $rc = $? >> 8;
if ($postproc) {
system '/usr/local/bin/dbstart thedb >/tmp/amanda/dbstart.debug 2>&1';
}
exit $rc;