Amidxtaped protocol

From The Open Source Backup Wiki (Amanda, MySQL Backup, BackupPC)
Revision as of 01:09, 1 August 2010 by Dustin (talk | contribs) (link to new Amanda Protocol)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search

This page picks up where Amandad Service Protocol leaves off. Amidxtaped sends exactly one dump in response to a set of parameters on its control stream. To get another dump, start a new amidxtaped.

Startup

If amidxtaped is started from amandad, then it processes its REQ packet and produces a REP packet for the Amanda Protocol as described in Amandad Service Protocol.

An amidxtaped REQ packet looks like:

SERVICE amidxtaped
OPTIONS features=features;auth=auth;

All of the interesting specification of which data to send comes on the control connection, below.

Amidxtaped sets up two streams in its REP packet:

CONNECT CTL ctlfd DATA datafd\n\n

the control connection is bidirectional, while the data connection only carries data from amidxtaped.

Started from Inetd

If amidxtaped is started directly from inetd, then it assumes its stdin and stdout are tied to a TCP socket.

This service expects to see the string

 SECURITY USER root\r\n

(or some other user than root). Note the \r\n at the end. I couldn't say why, but without the \r amidxtaped will hang. This line triggers a BSDTCP-style security check: the username and remote IP must match .amandahosts, the source port must be privileged, and the forward and reverse DNS must match. Note that the installchecks override this particular test.

Note that when amidxtaped is started directly from inetd, the main socket becomes the control connection after the SECURITY line.

Control Connection

command

The control connection begins with a command, given as a set of specifier lines, followed by an "END" line. All lines are terminated by "\r\n". Are you still asking why? Accept the insanity.

Not all specifiers must be given in any particular request. The available specifiers are:

LABEL=label
either a label or a tapespec giving the files to be recovered. If it doesn't parse as a tapespec, it's probably a label.
FSF=fsf
seek to this file before starting the recovery (only used if LABEL isn't a tapespec)
HEADER
prefix the data with an Amanda header (DISK_BLOCK_BYTES long)
FEATURES=features
reiterates the requester's feature-set (in case amidxtaped was started from inetd). When run from amandad, amidxtaped will immediately respond with the equivalent line giving its own features. When run from inetd, amidxtaped will respond with its features only, with no terminating newline of any sort. This relies on "normal" buffering of the TCP connection to receive the features in a single read operation.
DEVICE=device
tape device (or changer) to restore from, or a holding file, maybe in tapespec format
HOST=host
host match expression for this dump (optional)
DISK=disk
diskname match expression for this dump (optional)
DATESTAMP=datestamp
datestamp for this dump (optional)
CONFIG=config
Amanda configuration to use (optional)

the list of specifiers is terminated by

END\r\n

or by any line that starts with a digit (.. good, you didn't ask why -- you're learning!). Any non-matching lines will be ignored.

If the DEVICE is the same as the amrecover_changer configuration parameter, then it should be interpreted to mean the default changer.

When the requested dump is on a holding disk, then LABEL is omitted, and DEVICE specifies the holding file. In older versions this was just the filename of the holding file. In newer versions, that filename is encoded as the label in a tapespec with filenum 0 -- that is, $quoted_holdingfile:0:

DEVICE=/A/p/hold\:ing/20100205181319/euclid._A_p_etc.1:0

(note that the : in the holding pathname is quoted). Yeah, it's weird.

Older Amanda versions provided only a label in the LABEL= field. There's no feature flag to indicate which to expect. When only a label is provided, then it's up to FSF to indicate the first file in the dump. If FSF is missing, then amidxtaped should seek to the first file matching the dumpspec (HOST/DISK/DATESTAMP).

In general, the command is often massively redundant. Older versions of amidxtaped would just take the most specific, easily used information in the command and use that -- that is, if a tapespec is provided, these versions would ignore FSF and the dumpspec entirely. Newer versions try to cross-check the various specifications, and give an error or warning if they do not agree. This should prevent incorrect recoveries in cases where the catalog is corrupt or incorrect.

header

If HEADER is provided in the command, then amrecover expects a 32k header in return. That header must be of type F_DUMPFILE, and

data

Now the time comes to transfer the data stream. Note that the data is always uncompressed and decrypted on the server.

If amidxtaped was invoked via amandad, then the data flows over the data stream. The fe_recover_splits feature must be set in this case.

Otherwise, if the FEATURES did not include fe_recover_splits, then the data is streamed directly over the control connection, and no further interaction is possible. Otherwise, amidxtaped opens a new, unprivileged TCP port, and sends a CONNECT line on the control connection.

 CONNECT 1234\n

(where 1234 is the port number) Note that there is no \r in this particular message. Amidxtaped then blocks waiting for an incoming connection. That incoming connection must be from a privileged port and begin with the same SECURITY USER line as appeared on the main connection. After that point, the data begins flowing.

abort

After the header is sent, if fe_amidxtaped_aport is set on the server, then the client can send an ABORT message if the user wants to abort the recovery. Amidxtaped must exit cleanly if it receive this message.

datapath

After the header is sent (or immediately, if no HEADER specification was given), if both ends of the connection have fe_amidxtaped_datapath, then a three-way DATAPATH negotiation takes place.

First, amidxtaped waits for AVAIL-DATAPATH, which is the string AVAIL-DATAPATH followed by a space separated list of allowed datapaths. AMANDA must always be in the list. Two examples::

AVAIL-DATAPATH AMANDA\r\n
AVAIL-DATAPATH AMANDA DIRECT-TCP\r\n

Second, based on the device it has in hand, amidxtaped makes a final decision about the data path to be used. If amrecover has requested AMANDA, or if the device is not DirectTCP-capable, then amidxtaped replies with

USE-DATAPATH AMANDA\r\n

Otherwise, amidxtaped replies with

USE-DATAPATH DIRECT-TCP xxx.xxx.xxx.xxx:xxx yyy.yyy.yyy.yyy:yyy\r\n

where the x's and y's are any number of IP:PORT pairs.

Finally, the recovery application indicates that it has its ducks in a row and is ready to begin transferring data:

DATAPATH-OK\r\n

Amidxtaped then accepts the directtcp connection (if necessary) and begins transferring data - the DirectTCP connection takes the place of the data stream.

interaction

If amidxtaped has a message for the user, it sends a MESSAGE line on the control connection:

MESSAGE I like green eggs and ham.\r\n

Such messages are sent to amidxtaped's stderr if amrecover does not have fe_amrecover_message or if the control stream is being used for data (no fe_recover_splits support).

If amidxtaped needs the user to feed a particular volume, and if amrecover has the fe_amrecover_FEEDME feature, then it sends a FEEDME line:

FEEDME DailySet1-013\r\n

Labels longer than 132 bytes are not currently supported. If amidxtaped does not have the (oddly named) fe_amrecover_feedme_tape feature, then it expects an OK response and does not support trying some other device:

OK\r\n

If amrecover does have the fe_amrecover_feedme_tape feature, then also supports a TAPE response, specifying either the same device as used in the original request, or a new device:

TAPE tape:/dev/nrst0\r\n

indicating that amidxtaped should use the given device. There is no official "cancel" response, but amrecover uses ERROR\r\n to trigger a fatal error in amidxtaped when the user cancels a feedme request.

Note that, in the case of a split dump, MESSAGE or FEEDME lines may come after a significant quantity of data has been transferred. Conversely, they may come before the DIRECT-TCP negotiation or even before the header is transmitted.

user-visible notes

If the device given by the user doesn't work, no MESSAGE is sent - amidxtaped merely prompts for the correct volume.

If the device works but the correct volume is not found, then a corresponding MESSAGE is sent.