Amanda::Taper::Scribe
step start_scribe => sub {
my $scribe = Amanda::Taper::Scribe->new(
taperscan => $taperscan_algo,
feedback => $feedback_obj);
$scribe->start(
write_timestamp => $write_timestamp,
finished_cb => $steps->{'start_xfer'});
};
step start_xfer => sub {
my ($err) = @_;
my $xfer_dest = $scribe->get_xfer_dest(
allow_split => 1,
max_memory => 64 * 1024,
can_cache_inform => 0,
part_size => 150 * 1024**2,
part_cache_type => 'disk',
part_cache_dir => "$tmpdir/splitbuffer",
part_cache_max_size => 20 * 1024**2);
# .. set up the rest of the transfer ..
$xfer->start(sub {
my ($src, $msg, $xfer) = @_;
$scribe->handle_xmsg($src, $msg, $xfer);
# .. any other processing ..
};
# tell the scribe to start dumping via this transfer
$scribe->start_dump(
xfer => $xfer,
dump_header => $hdr,
dump_cb => $steps->{'dump_cb'});
};
step dump_cb => sub {
my %params = @_;
# .. handle dump results ..
print "DONE\n";
$finished_cb->();
};
This package provides a high-level abstraction of Amanda's procedure for writing dumpfiles to tape.
Amanda writes a sequence of dumpfiles to a sequence of volumes. The volumes are supplied by a taperscan algorithm, which operates a changer to find and load each volume. As dumpfiles are written to volumes and those volumes fill up, the taperscan algorithm supplies additional volumes.
In order to reduce internal fragmentation within volumes, Amanda can "split" dumpfiles into smaller pieces, so that the overall dumpfile can span multiple volumes. Each "part" is written to the volume in sequence. If a device encounters an error while writing a part, then that part is considered "partial", and is rewritten from its beginning on the next volume. Some devices can reliably indicate that they are full (EOM), and for these devices parts are simply truncated, and the Scribe starts the next part on the next volume.
To facilitate rewriting parts on devices which cannot indicate EOM, Amanda must retain all of the data in a part, even after that data is written to the volume. The Scribe provides several methods to support this: caching the part in memory, caching the part in a special on-disk file, or relying on pre-existing on-disk storage. The latter method is used when reading from holding disks.
The details of efficiently splitting dumpfiles and rewriting parts are handled
by the low-level Amanda::Xfer::Dest::Taper subclasses. The Scribe creates
an instance of the appropriate subclass and supplies it with volumes from an
Amanda::Taper::Scan object. It calls a number of
Amanda::Taper::Scribe::Feedback methods to indicate the status of the dump
process and to request permission for each additional volume.
The Amanda::Taper::Scribe constructor takes two arguments:
taperscan and feedback. The first specifies the taper scan
algorithm that the Scribe should use, and the second specifies the
Feedback object that will receive notifications from the Scribe (see
below).
my $scribe = Amanda::Taper::Scribe->new(
taperscan => $my_taperscan,
feedback => $my_feedback);
Once the object is in place, call its start method.
Start the scribe's operation by calling its start method. This will invoke
the taperscan algorithm and scan for a volume. The method takes two parameters:
$scribe->start(
write_timestamp => $ts,
finished_cb => $start_finished_cb);
The timestamp will be written to each volume written by the Scribe. The
finished_cb will be called with a single argument - undef or an error
message - when the Scribe is ready to start its first dump. The Scribe is
"ready" when it has found a device to which it can write, although it does not
request permission to overwrite that volume, nor start overwriting it, until
the first dump begins (that is, until the first call to start_dump).
Once the Scribe is started, begin transferring a dumpfile. This is a
three-step process: first, get an Amanda::Xfer::Dest::Taper object from the
Scribe, then start the transfer, and finally let the Scribe know that the
transfer has started. Note that the Scribe supplies and manages the transfer
destination, but the transfer itself remains the responsibility of the caller.
Call get_device to get the first device the xfer will be working with.
$device = $scribe->get_device();
This method must be called after start has completed.
Call check_data_path, supplying the data_path requested by the user.
if (my $err = $scribe->check_data_path($data_path)) {
# handle error message
}
This method must be called after start has completed and before
get_xfer_dest is called. It returns undef on success or an error message
if the supplied data_path is incompatible with the device. This is mainly
used to detect when a DirectTCP dump is going to a non-DirectTCP device.
Call get_xfer_dest to get the transfer element, supplying information on how
the dump should be split:
$xdest = $scribe->get_xfer_dest(
allow_split => $allow_split,
max_memory => $max_memory,
# .. splitting parameters
);
This method must be called after start has completed, and will always return
a transfer element immediately. The underlying Amanda::Xfer::Dest::Taper
handles device streaming properly. It uses max_memory bytes of memory for
this purpose.
The splitting parameters to get_xfer_dest are:
allow_splitthis dle is allowed or not to split
part_sizethe split part size to use, or 0 for no splitting
part_cache_typewhen caching, the kind of caching to perform ('disk', 'memory' or the default, 'none')
part_cache_dirthe directory to use for disk caching
part_cache_max_sizethe maximum part size to use when caching
can_cache_informtrue if the transfer source can call the destination's cache_inform method
(e.g., Amanda::Xfer::Source::Holding).
The first four of these parameters correspond exactly to the eponymous tapetype
configuration parameters, and have the same default values (when omitted or
undef). The method will take this information, along with details of the
device it intends to use, and set up the transfer destination.
The utility function get_splitting_args_from_config can determine the
appropriate get_xfer_dest splitting parameters based on a
few Amanda configuration parameters. If a parameter was not seen in the
configuration, it should be omitted or passed as undef. The function
returns a hash to pass to get_xfer_dest, although that hash may have an
warning key containing a message if there is a problem that the user
should know about.
use Amanda::Taper::Scribe qw( get_splitting_args_from_config );
my %splitting_args = get_splitting_args_from_config(
# Amanda dumptype configuration parameters,
dle_allow_split => ..,
dle_tape_splitsize => ..,
dle_split_diskbuffer => ..,
dle_fallback_splitsize => ..,
dle_allow_split => ..,
# Amanda tapetype configuration parameters,
part_size => .., ## in bytes, not kb!!
part_size_kb => ..., ## or use this, in kb
part_cache_type => ..,
part_cache_type_enum => ..., ## one of the enums from tapetype_getconf
part_cache_dir => ..,
part_cache_max_size => ..,
);
if ($splitting_args{'error'}) { .. }
An Amanda::Taper::Scribe object can only run one transfer at a time, so
do not call get_xfer_dest until the dump_cb for the previous start_dump
has been called.
Armed with the element returned by get_xfer_dest, the caller should create a
source element and a transfer object and start the transfer. In order to
manage the splitting process, the Scribe needs to be informed, via its
handle_xmsg method, of all transfer messages . This is usually accomplished
with something like:
$xfer->start(sub {
my ($src, $msg, $xfer) = @_;
$scribe->handle_xmsg($src, $msg, $xfer);
});
Once the transfer has started, the Scribe is ready to begin writing parts to
the volume. This is the first moment at which the Scribe needs a header, too.
All of this is supplied to the start_dump method:
$scribe->start_dump(
xfer => $xfer,
dump_header => $hdr,
dump_cb => $dump_cb);
The c<dump_header> here is the header that will be applied to all parts of the
dumpfile. The only field in the header that the Scribe controls is the part
number. The dump_cb callback passed to start_dump is called when the
dump is completely finished - either successfully or with a fatal error.
Unlike most callbacks, this one takes keyword arguments, since it has so many
parameters.
$dump_cb->(
result => $result,
device_errors => $device_errors,
config_denial_message => $cdm,
size => $size,
duration => $duration,
total_duration => $total_duration,
nparts => $nparts);
All parameters will be present on every call, although the order is not guaranteed.
The result is one of "FAILED", "PARTIAL", or "DONE". Even when
dump_cb reports a fatal error, result may be "PARTIAL" if some data
was written successfully.
The device_error key points to a list of errors, each given as a string,
that describe what went wrong to cause the dump to fail. The
config_denial_message parrots the reason provided by $perm_cb (see below)
for denying use of a new tape if the cause was 'config', and is undef
otherwise.
The final parameters, size (in bytes), duration, total_duration (in
seconds), and nparts describe the total transfer, and are a sum of all of
the parts written to the device. Note that nparts does not include any
empty trailing parts. Note that duration does not include time spent
operating the changer, while total_duration reflects the time from the
start_dump call to the invocation of the dump_cb.
After you have requested a transfer destination, the scribe is poised to begin the
transfer. If you cannot actually perform the transfer for some reason, you'll need
to go through the motions all the same, but cancel the operation immediately. That
can be done by calling cancel_dump:
$scribe->cancel_dump(
xfer => $xfer,
dump_cb => $dump_cb);
When all of the dumpfiles are transferred, call the quit method to
release any resources and clean up. This method takes a typical
finished_cb.
$scribe->quit(finished_cb => sub {
print "ALL DONE!\n";
});
The get_bytes_written returns the number of bytes written to the device at
the time of the call, and is meant to be used for status reporting. This value
is updated at least as each part is finished; for some modes of operation, it
is updated continuously. Notably, DirectTCP transfers do not update
continuously.
The start_scan method initiate a scan of the changer to find a usable tape.
The Amanda::Taper::Scribe::Feedback class is intended to be
subclassed by the user. It provides a number of notification methods
that enable the historical logging and driver/taper interactions
required by Amanda. The parent class does nothing of interest, but
allows subclasses to omit methods they do not need.
The request_volume_permission method provides a means for the caller
to limit the number of volumes the Scribe consumes. It is called as
$fb->request_volume_permission(perm_cb => $cb);
The perm_cb is a callback which expects a hash as arguments. If allow
is set, then the scribe is allowed to use a new volume, if scribe is set,
then the xfer must be transfered to that scribe, otherwise a cause
and a message describing why a new volume should not be used. must be
set. e.g.
perm_cb->(allow => 1); perm_cb->(scribe => $new_scribe); perm_cb->(cause => 'config', message => $message); perm_cb->(cause => 'error', message => $message);
A cause of 'config' indicates that the denial is due to the user's
configuration, and thus should not be presented as an error. The default
implementation always calls perm_cb->().
All of the remaining methods are notifications, and do not take a callback.
$fb->scribe_notif_new_tape(
error => $error,
volume_label => $volume_label);
The Scribe calls scribe_notif_new_tape when a new volume is started. If the
volume_label is undefined, then the volume was not successfully
relabled, and its previous contents may still be available. If error
is defined, then no useful data was written to the volume. Note that
error and volume_label may both be defined if the previous
contents of the volume were erased, but no useful, new data was written
to the volume.
This method will be called exactly once for every call to
request_volume_permission that calls back with perm_cb->().
$fb->scribe_notif_tape_done(
volume_label => $volume_label,
size => $size,
num_files => $num_files);
The scribe_notif_tape_done method is called after a volume is completely
written and its reservation has been released. Note that the scribe waits
until the last possible moment to release a reservation, so this may be called
later than expected, e.g., during a quit invocation.
$fb->scribe_notif_part_done(
partnum => $partnum,
fileno => $fileno,
successful => $successful,
size => $size,
duration => $duration);
The Scribe calls scribe_notif_part_done for each part written to the volume,
including partial parts. If the part was not written successfully, then
successful is false. The size is in bytes, and the duration is
a floating-point number of seconds. If a part fails before a new device
file is created, then fileno may be zero.
Finally, the Scribe sends a few historically significant trace log messages
via scribe_notif_log_info:
$fb->scribe_notif_log_info(
message => $message);
A typical Feedback subclass might begin like this:
package main::Feedback; use base 'Amanda::Taper::Scribe::Feedback';
sub request_volume_permission {
my $self = shift;
my %params = @_;
$params{'perm_cb'}->(cause => "error", message => "NO VOLUMES FOR YOU!");
}
This page was automatically generated Tue Feb 21 19:14:01 2012 from the Amanda source tree, and documents the most recent development version of Amanda. For documentation specific to the version of Amanda on your system, use the 'perldoc' command.