Amanda Index: Difference between revisions

From wiki.zmanda.com
Jump to navigation Jump to search
mNo edit summary
 
(4 intermediate revisions by one other user not shown)
Line 2: Line 2:
Amanda's indexes contain a list of the directories and files in a dump, and are used in restore operations to determine which files are available and on what tapes they are stored.
Amanda's indexes contain a list of the directories and files in a dump, and are used in restore operations to determine which files are available and on what tapes they are stored.


==Database Format==
The index is required for {{man|8|amrecover}}. If the index is not available, the tape can restored using {{man|8|amrestore}} or using the backup command (gnutar or restore) and dd command.


The database consists of a directory tree rooted at <tt>indexdir</tt> from [[amanda.conf]] containing filenames of the format
=Database Format=
 
The database consists of a directory tree rooted at <tt>indexdir</tt> from {{man|5|amanda.conf}} containing filenames of the format
  ''host''/''disk''/''date''_''level''.gz
  ''host''/''disk''/''date''_''level''.gz
There is an index file for each dump.  The ''host'' and ''disk'' are those listed in the [[disklist]], formatted like the [[curinfo]] database: all '/' characters are changed for '_'.  The file will have the ''.gz'' suffix if they are compressed with gzip.
There is an index file for each dump.  The ''host'' and ''disk'' are those listed in the [[disklist]], formatted like the [[curinfo]] database: all '/' characters are changed for '_'.  The file will have the ''.gz'' suffix if they are compressed with gzip.
Line 10: Line 12:
As an example, <tt>server18/_usr/19991231_0.gz</tt> is the index of the level 0 made on 19991231 of the disk <tt>/usr</tt> of the host <tt>server18</tt>.
As an example, <tt>server18/_usr/19991231_0.gz</tt> is the index of the level 0 made on 19991231 of the disk <tt>/usr</tt> of the host <tt>server18</tt>.


The files are ASCII text files containing a list of the directory and files of the dump, one per line. Each entry is the filename relative to the mount point, starting with a /, e.g., /home/user1/data from the disk mounted on /home would generate the entry /user1/data. The index files are stored in compressed format (eg gzip or compress).
The files are ASCII text files containing a list of the directory and files of the dump, one per line. Each entry is the filename relative to the mount point, starting with a /, e.g., /home/user1/data from the disk mounted on /home would generate the entry /user1/data. Directories must end with a /.
 
==Database Browsing==
 
The tool browse database index is [[amrecover]]. amrecover has to be started by root user. This requires specifying the index server (amanda server) and the amanda config name (defaults for both are compiled in as part of the installation). Then the user has to specify the name of the host whose backup is being recovered, the disk name, and (optionally) the disk mount point. Finally a date needs to be specified. Given all this, the user can then browse the virtual file system using "ls" and "cd" much like in a FTP client. The file system contains all files backed up on the specified date, or before that date, back to the last level 0 backup. Only the most recent version of any file is shown.
 
As the file system is traversed, the user can add and delete files to a "shopping list" (''add'' and ''delete'' commands), and print the list out.
 
 
==File Extraction==
 
When a user has built up a list of files to extract, they can be extracted by issuing the command ''extract'' within amrecover.
 
Files are extracted by the following, for each different tape needed.
 
As part of the installation, a "tape server" daemon amidxtaped is installed on one or more designated hosts, which have an attached tape drive. This is used to read the tapes. See the config files for the options for specifying a default.
 
amrecover contacts amidxtaped on the tape server host specifying which tape device to use, which host and disk files are needed for.
 
* Amanda 2.4.X: On the tape server host, amidxtaped executes [[amrestore]] to get the dump image file off the tape, and returns the data to amrecover.
 
* Amanda 2.5.X: On the tape server host, amidxtaped reads the dump files itself, eventually contacting amrecover with instructions for the user to load the next tape.
 
If dumps are stored compressed for the client, then amrecover pipes the data through the appropriate uncompression routine to uncompress it before piping it into restore, which then extracts the required files from the dump image.
 
Note that a user can only extract files from a host running the same operating system as he/she is executing amrecover on, since the native dump/restore tools are used - unless GNU-tar is used.
 
 


==Configuration==
The index files are stored in compressed format (eg gzip or compress).


* Whether or not an index is created for a disk is controlled by a disk configuration option "index". So, in [[amanda.conf]] you need to define a disktype with this option, e.g.,
=Configuration=


Creation of an index when dumping a disk is controlled by the dumptype option <tt>index</tt>.  For example:
  define dumptype comp-user-index {
  define dumptype comp-user-index {
             comment "Non-root partitions on reasonably fast machines"
             #...
            compress client fast
             ''index yes''
             index yes
            priority medium
  }
  }
Amanda backups will almost always create indexes.  Sometimes indexes are disabled if the administrator knows that recovery will always be from an entire dump.


* You need to define disks that you want to generate an index for to be of one of the disktypes you defined which contain the index option. This cause sendbackup-dump on the client machine to generate an index file which is stored local to the client, for later recovery by amgetidx (which is called by amdump).
=Index In Action=
== Dump ==
During a dump operation, the driver reads three streams of information from the client concurrently: data (the dump itself), messages (errors, warnings, and so on), and the index.  The index stream is written (possibly through a compression program) directly to the index file.
{{Note|Previous versions of Amanda used a program called <tt>amgetidx</tt> to fetch the index; this program is not used in recent versions of Amanda.}}


* Amanda saves all the index files under a directory specified by "indexdir" in [[amanda.conf]]. In some versions of Amanda, you need to create this directory by hand. It needs to have read/write permissions set for the user you defined to run Amanda.
==Index retention==


* The index browser, [[amrecover]], currently gets installed as part of the client software. Its location may not be appropriate for your system and you may need to move it to a more accessible place such as /usr/local/bin.  
Amanda keeps the online info and index of a tape until
* the tape is reused, or
* the tape (with force flag) rewritten the same label, or
* {{man|8|amrmtape}} is done on the tape.


Note that amindexd, amgetidx, amidxtaped, and amtrmidx all write debug files on the server in /tmp/amanda/server, which are useful for diagnosing problems. amrecover writes a debug file in /tmp/amanda/server on the machine it is invoked.
== Restore  ==


The debug directory, usually /tmp/amanda, is hardcoded at compile time can be found by running "amadmin x version".
The tool browse database index is {{man|8|amrecover}}. This requires specifying the index server (amanda server) and the amanda config name (defaults for both are compiled in as part of the installation). Then the user has to specify the name of the host whose backup is being recovered, the disk name, and (optionally) the disk mount point. Finally a date needs to be specified. Given all this, the user can then browse the virtual file system using "ls" and "cd" much like in a FTP client. The file system contains all files backed up on the specified date, or before that date, back to the last level 0 backup.  


==Index retention==
Amrecover builds this virtual filesystem based on the index files stored on the index server.  The communication with the index server uses the [[amindexd protocol]].


Amanda keeps the online info and index of a tape until
As part of the installation, a "tape server" daemon, ''amidxtaped'', is installed on one or more designated hosts, which have an attached tape drive. This is used to read the tapes. See the config files for the options for specifying a default.  Files are extracted by the following process, repeated for each required tape:
* the tape is reused, or
* the tape (with force flag) rewritten the same label, or
* [[amrmtape]] is done on the tape.


The index is required for [[amrecover]]. If the index is not available, the tape can restored using [[amrestore]] or
# amrecover contacts amidxtaped on the tape server host specifying which tape device to use, and which host and disk files are needed.
using the backup command (gnutar or restore) and dd command.
#* Before Amanda 2.5.0: On the tape server host, amidxtaped executes {{man|8|amrestore}} to get the dump image file off the tape, and streams the data to {{man|8|amrecover}}.
#* Amanda 2.5.0 and later: On the tape server host, amidxtaped reads the dump files itself, eventually contacting {{man|8|amrecover}} with instructions for the user to load the next tape.
# If dumps are stored compressed by the client, then amrecover pipes the data through the appropriate uncompression routine to uncompress it before piping it into restore, which then extracts the required files from the dump image.


==Permissions==
=Permissions=


The userid chosen to run the Amanda client code must have permission to run restore since this is used by createindex-dump to generate the index files.
The userid chosen to run the Amanda client code must have permission to run restore since this is used by createindex-dump to generate the index files.
Line 76: Line 56:
For a user to be able to restore files from within amrecover, that user must have permission to run restore.  
For a user to be able to restore files from within amrecover, that user must have permission to run restore.  


==Known Bugs==
=Known Bugs=


*Empty directories don't get into the listing for a dump (at all dump levels).  
*Empty directories don't get into the listing for a dump (at all dump levels).  


*When amrecover starts up, it tries to guess the disk and mount point from the current directory of the working system. This doesn't work for disks specified by logical names, nor when an automounter is being used, or a link is in the path.
*When amrecover starts up, it tries to guess the disk and mount point from the current directory of the working system. This doesn't work for disks specified by logical names, nor when an automounter is being used, or a link is in the path.
{{Languages}}

Latest revision as of 08:53, 26 September 2012

Amanda's indexes contain a list of the directories and files in a dump, and are used in restore operations to determine which files are available and on what tapes they are stored.

The index is required for amrecover(8). If the index is not available, the tape can restored using amrestore(8) or using the backup command (gnutar or restore) and dd command.

Database Format

The database consists of a directory tree rooted at indexdir from amanda.conf(5) containing filenames of the format

host/disk/date_level.gz

There is an index file for each dump. The host and disk are those listed in the disklist, formatted like the curinfo database: all '/' characters are changed for '_'. The file will have the .gz suffix if they are compressed with gzip.

As an example, server18/_usr/19991231_0.gz is the index of the level 0 made on 19991231 of the disk /usr of the host server18.

The files are ASCII text files containing a list of the directory and files of the dump, one per line. Each entry is the filename relative to the mount point, starting with a /, e.g., /home/user1/data from the disk mounted on /home would generate the entry /user1/data. Directories must end with a /.

The index files are stored in compressed format (eg gzip or compress).

Configuration

Creation of an index when dumping a disk is controlled by the dumptype option index. For example:

define dumptype comp-user-index {
            #...
            index yes
}

Amanda backups will almost always create indexes. Sometimes indexes are disabled if the administrator knows that recovery will always be from an entire dump.

Index In Action

Dump

During a dump operation, the driver reads three streams of information from the client concurrently: data (the dump itself), messages (errors, warnings, and so on), and the index. The index stream is written (possibly through a compression program) directly to the index file.

NOTE: Previous versions of Amanda used a program called amgetidx to fetch the index; this program is not used in recent versions of Amanda.

Index retention

Amanda keeps the online info and index of a tape until

  • the tape is reused, or
  • the tape (with force flag) rewritten the same label, or
  • amrmtape(8) is done on the tape.

Restore

The tool browse database index is amrecover(8). This requires specifying the index server (amanda server) and the amanda config name (defaults for both are compiled in as part of the installation). Then the user has to specify the name of the host whose backup is being recovered, the disk name, and (optionally) the disk mount point. Finally a date needs to be specified. Given all this, the user can then browse the virtual file system using "ls" and "cd" much like in a FTP client. The file system contains all files backed up on the specified date, or before that date, back to the last level 0 backup.

Amrecover builds this virtual filesystem based on the index files stored on the index server. The communication with the index server uses the amindexd protocol.

As part of the installation, a "tape server" daemon, amidxtaped, is installed on one or more designated hosts, which have an attached tape drive. This is used to read the tapes. See the config files for the options for specifying a default. Files are extracted by the following process, repeated for each required tape:

  1. amrecover contacts amidxtaped on the tape server host specifying which tape device to use, and which host and disk files are needed.
    • Before Amanda 2.5.0: On the tape server host, amidxtaped executes amrestore(8) to get the dump image file off the tape, and streams the data to amrecover(8).
    • Amanda 2.5.0 and later: On the tape server host, amidxtaped reads the dump files itself, eventually contacting amrecover(8) with instructions for the user to load the next tape.
  2. If dumps are stored compressed by the client, then amrecover pipes the data through the appropriate uncompression routine to uncompress it before piping it into restore, which then extracts the required files from the dump image.

Permissions

The userid chosen to run the Amanda client code must have permission to run restore since this is used by createindex-dump to generate the index files.

For a user to be able to restore files from within amrecover, that user must have permission to run restore.

Known Bugs

  • Empty directories don't get into the listing for a dump (at all dump levels).
  • When amrecover starts up, it tries to guess the disk and mount point from the current directory of the working system. This doesn't work for disks specified by logical names, nor when an automounter is being used, or a link is in the path.

Other languages: [[::Amanda Index|English]] {{#ifexist: {{#if: | | {{#if: | :}}Amanda Index}}/Fr |  • {{#if: |français| [[::Amanda Index/Fr|français]]}}|}}

{{#ifexist: {{#if: | | {{#if: | :}}Amanda Index}}/Zh-cn |  • {{#if: |中文(中国大陆)| [[::Amanda Index/Zh-cn|中文(中国大陆)]]}}|}}