Restoring files (old)

From The Open Source Backup Wiki (Amanda, MySQL Backup, BackupPC)
(Redirected from Restoring files)
Jump to navigationJump to search
Deprecated
See Getting Started with Amanda instead

This section describes how to restore files backed up with Amanda either with or without Amanda tools. The section has been presented as multiple examples. Read the whole section and use the example that is most applicable to your situation

Using amrecover

When you have enabled index creation in the dumptype, the most easy tool to restore files is the amrecover(8) command.

Amrecover should be run as super user (Unix root user). It can be use recover files either on the Amanda server or Amanda client.

If you have a tape changer, following parameters must be enabled in the amanda.conf(5):

amrecover_do_fsf yes
amrecover_check_label yes
amrecover_changer "changer"

One of the most frequent errors with using amrecover is forgetting to rewind the tape: see: Amrecover: EOF, check amidxtaped.debug. The first line rewinds the tape automatically before use, and moreover, will position the tape quickly to the correct image on tape. We add the second line because we are humans, and do make mistakes. With the last line, we give our changer a name that we can use as tape device for amrecover.

Amrecover lets you browse the index of the archives, using an ftp-like interface. You can add single files or directories to the extraction list. Restoring a directory will include all its contents.

When amrecover is run on the client as superuser, Amanda client should have permission to recover from the server. The file .amandahosts on the server needs to contain entries for those Amanda clients that are trusted to do recovery:

Modern format

mathilda.example.com root amindexd amidxtaped

pre-Amanda 2.5.1 format:

mathilda.example.com root
katastrov.example.com root

The first line is just like any other .amandahosts file for clients. The second line allows user root on host katastrov to contact this host with amrecover.

Suppose the complete disk containing the /home filesystem on host katastrov crashed. The disk has been replaced, formatted and mounted. Now we just need to restore the data back.

Start amrecover as root in the top of the client filesystem:

# cd /home
# amrecover daily
AMRECOVER Version 2.4.5. Contacting server on mathilda ...
220 mathilda AMANDA index server (2.4.5) ready.
200 Access OK
Setting restore date to today (2005-10-15)
200 Working date set to 2005-10-15.
Scanning /space/amandahold/daily...
200 Config set to daily.
200 Dump host set to katastrov.
Trying disk /home ...
$CWD '/home' is on disk '/home' mounted at '/home'.
200 Disk set to /home.
/home
amrecover> add .
Added dir / at date 2005-10-14
Added dir / at date 2005-10-15
amrecover> extract

Extracting files using tape drive file:/amanda-vtapes/daily/slots on host mathilda.
The following tapes are needed: DAILY-12
                                DAILY-14

Restoring files into directory /home
Continue [?/Y/n]? y

Extracting files using tape drive file:/amanda-vtapes/daily/slots on host mathilda.
Load tape DAILY-12 now
Continue [?/Y/n/s/t]? t
New tape device [?]: changer
Using tape "changer" from server mathilda.
Continue [?/Y/n/s/t]? y
[...list of files...]
Load tape DAILY-14 now
Continue [?/Y/n/s/t]? y
[...list of files...]
amrecover> exit
200 Good bye.

Instead of instructing amrecover to use the changer from within as above, you could also do that from the command line:

# amrecover daily -d changer

There are more subcommands, like:

sethost
restore files from a different host to the current directory
setdate
file listing will return only information from that day or earlier
setdisk
specifies which DLE (disk list entry) to examine
listdisk
list all disknames
settape
specify the host and tapedrive on that host to use to restore

and many more.

Warning

Always do restoration of backed up files to a temporary directory. Verify the restored files and then move them to correct location. amrecover command can erase files. See Why does amrecover erase files? for more information.

Restricting Access

You may want to restrict which hosts can perform a recovery of particular DLEs. See How To:Limit the Hosts That Can Recover a DLE for details, noting that this requires at least Amanda-3.2.

Using amrestore

A basic restore utility is the Amanda amrestore command. This command can be used to get the backup image of a tape. You can then use your own restore/gnutar command with tailored options to manipulate this image for restoring just what and how you want it. The amrestore(8) command can be used in bare metal recovery conditions and does not use the Amanda index.

The amrestore(8) command needs to run on the Amanda server. It can be run as the amanda user or any user that has permissions to read the backup media. Also, amrestore does not work with a media changer. It uses the tape that is already loaded in the tape device.

Do not forget to rewind the tape when using amrestore either!

To restore the /home partition of the amanda server itself, we could do:

# amrestore -p /dev/rmt/0cn mathilda '^/home$'  |  restore -ivbf 2 -

The option "-b 2" avoids the "short read" errors from restore at end of file when reading from a pipe.

If the backups were created with program "GNUTAR", then use gtar to restore:

# amrestore -p /dev/rmt/0cn mathilda '^/home$'  |
gtar --numeric-owner -xpGvf - ./your/lost/file

The option "-G" in conjunction with "-x" causes tar to read the lists of directory contents previously stored in the archive, delete files in the file system that did not exist in their directories when the archive was created, and then extract the files in the archive. This behavior is convenient when restoring a damaged file system from a succession of incremental backups: it restores the entire state of the file system to that which obtained when the backup was made. If --incremental (-G) isn't specified, the file system will probably fill up with files that shouldn't exist any more.

The option "--numeric-owner" is added here, for the case that when you are restoring with a bootable CD, and the /etc/passwd and /etc/group and even the /etc/nsswitch.conf file of that CD do not match with the files of the server that you are restoring. Specifying this option makes sure that, after reboot with the server's settings active again, all the files will have the correct owner. When you leave out this option, then the username takes precedence over the uid, resulting in possibly wrong ownerships of the files after reboot.

Note also that the "host" and the "disk" in the amrestore command are actually expressions, that can match many hosts or disks.

If you omit the "-p" option, then the dump images will be written to the local directory, creating probably very large files, having names host.name._disk_name.YYYYMMDD.L (L standing for the dump level).

While scanning the tape, amrestore reports the images it encounters on the tape. This can be used to make a quick table of contents of a tape, giving a hostname that never matches:

$ mt -f /dev/nrmt9 rewind
$ amrestore -p /dev/nrmt9 no-such-host > /dev/null

The "-p >/dev/null" above is just in case we guessed wrong, and someone did name a host "no-such-host".

When needing to restore on a client without tape, we need to separate the two commands from the pipe over two hosts. We can use "rsh", "ssh", or even "netcat" for this:

On the client, e.g. with rsh (set up .rhosts on the server to allow access without passwords if needed). On the client, the restore program usually needs to be executed as root, but amrestore on the server is done as user amanda:

# rsh -n -l amanda mathilda amrestore -p /dev/rmt/0cn katastrov '/home\$' | restore -ivbf 2 -

Or reversed, on the server using e.g. ssh (which can use public key authorisation to allow access without passwords):

$ amrestore -p /dev/rmt/0cn katastrov '/home$'  |  ssh -l root katastrov gtar -xpGvf - ./the/files

Or with netcat, first on the client execute as root:

# nc -l -p 1234 | gtar -xpGvf -

then on the tape server execute as amanda:

$ amrestore -p /dev/rmt/0cn katastrov '/home$' | nc -w 1 katastrov 1234

Using amrestore with split dumps

When the dump you try to restore was created with a dumptype having the tape_splitsize parameter set, amrestore does not automatically concatenate them again. When using amrestore on such chunked files, you need to omit the "-p" option (otherwise amrestore quits after reading the first chunk). Without the "-p" option, you get several files on disk.

Here is an example, using a virtual tape from a chg-disk changer, which was split on tape into 10Mbyte chunks:

$ amtape test slot 1
$ ammt -t file:/space/amanda-vtapes/test/slots rewind
$ amrestore file:/space/amanda-vtapes/test/slots katastrov '/home$'
amrestore:   1: skipping katastrov._space_extra__stuff.20060123.0
amrestore:   2: restoring katastrov._home.20060123.0.1
amrestore:   3: restoring katastrov._home.20060123.0.2
amrestore:   4: skipping katastrov._home2.20060123.0.0000001
amrestore:   5: skipping katastrov._home2.20060123.0.0000002
amrestore:   6: reached end of information
$ ls -l
ls -l
total 14400
-rw-r-----  1 amanda   disk   10420224 Jan 24 10:03 katastrov._home.20060123.0.1
-rw-r-----  1 amanda   disk    4325376 Jan 24 10:03 katastrov._home.20060123.0.2

The format of the file names is now host.name._disk_name.YYYYMMDD.L.N, with a chunk number appended. To restore from these files, just concatenate the files in the correct order:

$ cat katastrov* | gtar -zxvf -

Here we rely on the glob to return the files in the correct order. Beware when the trailing sequence numbers are not the same length. Then do:

$ cat katastrov*.? katastrov*.?? | gtar -zxvf -

If you restored with the "-r" option, the header of each tapefile is not removed. Restoring from those files can be done with a little bit shell scripting:

$ for f in katastrov*.? ; do dd if=$f bs=32k skip=1; done | gtar -zxvf -

Using amfetchdump

When the split dumps are spread over multiple tapes, and you use amrestore, you quickly need lots of temporary diskspace to restore the pieces to disk from the different tapes, and assembling them in the correct order. (The last, incomplete, piece on one tape is rewritten again on the next tape: you have to discard the incomplete part.)

The program amfetchdump(8) does this all for you: it knows which tapes are needed, drives the changer and extracts all the necessary pieces, discarding the incomplete pieces at the end of each tape, and concatenates them all in the correct order.

An example with the same chg-disk setup:

$ amfetchdump -apl test katastrov '/home$' 20060123  | gtar -ztvf -  > my.toc

The "-a" option is just to indicate that the chg-disk changer has all necessary tapes available. The "-l" option says to just strip each header, and just concatenate the chunks without uncompressing (we do the uncompress in gtar ourselves).

Because amfetchdump needs access to the tape changer, the Amanda configuration files, and the dump logs for the tape contents, it is not the perfect tool for a disaster recovery when you have lost the Amanda setup. It is good practice to avoid tape-splitting for those DLEs that contain the Amanda configuration and log files.

When you lost the dump logs with the tape contents, amfetchdump can recreate those with the "-i" option.

Finding which tape

When needing to restore something, we can find out which tapes are needed, using different ways. Amrecover will tell it, but if you don't have indexing enabled, other roads need to be explored.

Use amadmin(8) info to find the tapes most recently used to backup the partition.

$ amadmin daily info katastrov /home

Current info for nabokov /home:
  Stats: dump rates (kps), Full:  684.0, 611.0, 391.0
                    Incremental:  250.0, 230.0, 749.0
          compressed size, Full:  20.8%, 20.7%, 20.7%
                    Incremental:  21.1%, 23.0%, 23.0%
  Dumps: lev datestmp  tape             file   origK   compK secs
          0  20051015  DAILY-14           26 7134020 1485023 2170
          1  20051014  DAILY-12           61 3750910  823825 1203
          2  20050930  DAILY-02           55 3341080  792370 1166
          3  20051001  DAILY-03           90 2951645  674173 1072

Above information shows that two tapes are needed to do a full restore (DAILY-14, DAILY-12). Note that, even if DAILY-02 and DAILY-03 are listed, they are actually older than the full backup, so they should not be used to restore the most recent data.

Or use amadmin(8) find to search for an older version:

$ amadmin daily find home katastrov /home
Scanning /space/amandahold/daily...
Scanning /space/amandahold/daily...

date       host      disk  lv tape or file file status
2005-09-21 katastrov /home  0 DAILY-15       23 OK
2005-09-22 katastrov /home  1 DAILY-17       10 OK
2005-09-23 katastrov /home  1 DAILY-18       62 OK
2005-09-24 katastrov /home  1 DAILY-18      174 OK
2005-09-27 katastrov /home  1 DAILY-19      118 OK
2005-09-28 katastrov /home  2 DAILY-20       74 OK
2005-09-29 katastrov /home  2 DAILY-01      101 OK
2005-09-30 katastrov /home  2 DAILY-02       55 OK
2005-10-01 katastrov /home  3 DAILY-03       90 OK
2005-10-04 katastrov /home  0 DAILY-05       18 OK
2005-10-05 katastrov /home  1 DAILY-06       66 OK
2005-10-06 katastrov /home  1 DAILY-06      177 OK
2005-10-07 katastrov /home  1 DAILY-08       35 OK
2005-10-08 katastrov /home  1 DAILY-08      103 OK
2005-10-11 katastrov /home  1 DAILY-09       70 OK
2005-10-12 katastrov /home  1 DAILY-10      100 OK
2005-10-13 katastrov /home  1 DAILY-11       83 OK
2005-10-14 katastrov /home  1 DAILY-12       61 OK
2005-10-15 katastrov /home  0 DAILY-14       26 OK

Here we see we can to back up four weeks back, if needed.

If you lost the Amanda databases, then you can fall back to Table of Contents generating amrestore:

$ mt -f /dev/nst0 rewind
$ amrestore -p /dev/nst0 no-such-host > /dev/null

Or, without any Amanda tools, just get the 32k header of each tape file, and print the first line:

$ mt -f /dev/nst0 rewind
$ while mt -f /dev/nst0 fsf 1
> do  dd if=/dev/nst0 bs=32k count=1 | head -1
> done

Tip: the next lines of the header block contain the command you need to restore the image.

Because losing the Amanda database is not very nice, many experienced sysadmins make a copy of the current Amanda configuration, including the database files to a second computer using rsync.

The info database is not too large, and many experienced sysadmins mail that part of the configuration each day to themselves:

amadmin <config> export | Mail -s 'amanda export' [email protected]

That textfile can be used to recreate the info-database with 'amadmin <config> import'

It is also nice to store paper printed labels together with the tapes. These can be generated by the amreport command if you add a lbl-templ parameter in the tapetype for that tape. The templates are postscript files, that are filled in and printed by amreport. The sources contain several template files in the example directory to copy and adapt.

Examples

Assumptions
  • All the examples below discuss restoration of a complete disk, that is, replacing a bad disk with a new one, or created a new filesystem on top of old one. Changing the restore arguments (not amrestore), individual files can be restored.
  • Also, this section does not cover amrecover, a program that provides a text user interface similar to interactive restore (restore -i), but it allows you to select individual files to recover and automatically determines the tapes where they were stored. The backups must be performed with the `index' option enabled for this to work.
  • All the examples below assume the backups were done using DUMP program. Backups done using GNUTAR can be restored in similar manner. Instead of using restore command, gtar command will have to be used.
  • The server machine is called Aaron, the client machine is called Barney. Backup is being restored to Barney

Example 1: Client machine fails, non-system critical

Example: /home fails on Barney.

  • Use amadmin to find the tapes most recently used to backup the partition.
  • Log into Barney as root. Cd to the /home directory.
  • Insert the tape with the level 0 dump on it into the tape drive of Aaron.
  • On Barney run (replace <amanda> with the username under which amanda runs):
rsh -n -l amanda Aaron amrestore -p /dev/rmt/0cn Barney '/home\$' |
restore -ivbf 2 -

This step requires client root to have login access to amanda@Aaron, with a .rhosts entry (.amandahosts won't do). If you use ssh, you may be able to type a password in order to be authenticated. Another alternative is to start the operation in the server, and rsh to the client. You should be the amanda user or root in the tape server and run:

amrestore -p /dev/rmt/0cn Barney '/home$' |
rsh Barney -l root /usr/etc/restore -ivbf 2 - 

If you don't want to use rsh at all, you may run:

amrestore /dev/rmt/0cn Barney '/home$' 

This should create a file whose name contains the hostname, directory name, dump level and dump date of the backup. Now you have to move this file to the client somehow: you may use NFS, rcp, ftp. Suppose you rename that file to `home.0'. Then, on the client, you should become root and run:

restore -ivbf 2 home.0 
  • Repeat one of these steps, incrementing the level of the dump, until there are no more available backups.

Example 2: Client machine fails, system critical disk

Example: / fails on Barney.

  • Boot off the CD, and reinstall the system critical partition, restoring it to vendor supplied state.
  • Go through all steps in Example 1.

When restoring with GNU-tar, be aware that the /etc/passwd file from the bootable CD may not match with the /etc/passwd that will be restored. You better instruct GNU-tar to keep the numeric owner of files, instead of the name of the owner of the files, so that after restoring the original /etc/passwd and /etc/group file from the tape, and after rebooting, all the files are owned by the correct user again. The commands then become:

rsh -n -l amanda Aaron amrestore -p /dev/rmt/0cn Barney '/home\$' |
gtar --numeric-owner -xpGvf -

Example 3: Server machine fails, non-system critical, non-Amanda disk

  • Proceed with steps described in Example 1.
  • However, the rsh process need not be used. Use amrestore to replace the lost data directly.

Example 4: Server machine fails, system critical, non-Amanda disk

Example: / on Aaron

  • Boot off the CD, and reinstall the system critical partition, restoring it to vendor supplied state.
  • Follow steps in Example 3.

Example 5: Server machine fails, non-system critical, Amanda disk, with db

Example: /etc on Aaron

  • If the disk that contains the Amanda database is bad, then you need to rebuild the database.
  • Use the text file that is mailed using the 'amadmin <config> export' command, and import via the 'amadmin <config> import' command.
  • Follow the steps outlined in Example 4.
Note that Amanda does not mail the exported database automatically. Create a crontab entry to do 'amadmin <config> export'
It is a good idea to print out the text files as well and store the last 2 dumpcycles information.

Example 6: Server machine fails, non-system critical, Amanda disk, with binaries

Example: /usr/local on Aaron

If the disk with the amanda binaries is bad, there are three options.

  1. Reinstall the Amanda binaries from another tape or RPMs, on which you have conveniently backed up the binaries within the last couple of weeks (not using Amanda).
  2. Recompile Amanda, making sure not to overwrite your db files.
  3. Use dd to read Amanda formatted tapes. Amanda stores dump in client format and can be restored using client tools.

1. Find out the device name used by Amanda, by looking in amanda.conf. Example: /dev/rmt/0cn. If amanda.conf is not available, this must be a non-rewinding tape device. (the trailing `n' stands for non-rewinding tape device).

2. Look at a copy of the output of 'amadmin <config> export', and find out which tapes /usr/local was backed up on.

3. Grab the tapes that /opt was backed up on, and put the level 0 into the tape drive. cd to /usr/local.

4. Skip the first record, which is just the tape header, by using the appropriate tape command.

mt -f /dev/rmt/0cn fsf 1

5. Look for /usr/local on this tape.

dd if=/dev/rmt/0cn bs=32k skip=1 | gzip -d | /usr/sbin/restore -ivf -
This command provides interactive restore of this record, including partition, host, and level the backup was. The gzip -d portion of the pipe can be omitted if there was no compression.

6. If /usr/local could not be found, quit restore, and move forward one record.

mt -f /dev/rmt/0cn fsf 1
and try the dd/restore command shown above. Do this until you find /opt on the disk.
  • Another option is to create a quick tape index in case information about which partition /usr/local was on is not available.
#!/bin/sh
TAPEDEV=/dev/nrtape
while mt -f $TAPEDEV fsf 1 ; do
 dd if=$TAPEDEV bs=32k count=1 | head -1
 sleep 1
done
Example output:
 AMANDA: FILE 19971220 uri /root-sun4 lev 1 comp .gz program DUMP
 AMANDA: FILE 19971220 uri /misc lev 1 comp .gz program DUMP
 AMANDA: FILE 19971220 uri / lev 1 comp .gz program DUMP 

7. Restore the Amanda binaries and then exit restore. Use amrestore command as in Example 3.