Amanda::Taper::Scribe
my $scribe = Amanda::Taper::Scribe->new(
taperscan => $taperscan_algo,
feedback => $feedback_obj);
$subs{'start_scribe'} = make_cb(start_scribe => sub {
$scribe->start($datestamp, finished_cb => $subs{'start_xfer'});
});
$subs{'start_xfer'} = make_cb(start_xfer => sub {
my ($err) = @_;
my $xfer_dest = $scribe->get_xfer_dest(
max_memory => 64 * 1024,
split_method => 'disk',
part_size => 150 * 1024**2,
disk_cache_dirname => "$tmpdir/splitbuffer");
# .. set up the rest of the transfer ..
$xfer->start(sub {
my ($src, $msg, $xfer) = @_;
$scribe->handle_xmsg($src, $msg, $xfer);
# .. any other processing ..
});
# tell the scribe to start dumping via this transfer
$scribe->start_dump(
xfer => $xfer,
dump_header => $hdr,
dump_cb => $subs{'dump_cb'});
});
$subs{'dump_cb'} = make_cb(dump_cb => sub {
my %params = @_;
# .. handle dump results ..
print "DONE\n";
Amanda::MainLoop::quit();
});
$subs{'start_scribe'}->();
Amanda::MainLoop::run();
This package provides a high-level abstraction of Amanda's procedure for writing dumpfiles to tape.
Amanda writes a sequence of dumpfiles to a sequence of volumes. The volumes are supplied by a taperscan algorithm, which operates a changer to find and load each volume. As dumpfiles are written to volumes and those volumes fill up, the taperscan algorithm supplies additional volumes.
In order to reduce internal fragmentation within volumes, Amanda can "split" dumpfiles into smaller pieces, so that the overall dumpfile can span multiple volumes. Each "part" is written to the volume in sequence. If a device encounters an error while writing a part, then that part is considered "partial", and is rewritten from its beginning on the next volume. Some devices can reliably indicate that they are full (EOM), and for these devices parts are simply truncated, and the Scribe starts the next part on the next volume.
To facilitate rewriting parts on devices which cannot indicate EOM, Amanda must retain all of the data in a part, even after that data is written to the volume. The Scribe provides several methods to support this: caching the part in memory, caching the part in a special on-disk file, or relying on pre-existing on-disk storage. The latter method is used when reading from holding disks.
The details of efficiently splitting dumpfiles and rewriting parts are handled
by the low-level Amanda::Xfer::Dest::Taper subclasses. The Scribe creates
an instance of the appropriate subclass and supplies it with volumes from an
Amanda::Taper::Scan object. It calls a number of
Amanda::Taper::Scribe::Feedback methods to indicate the status of the dump
process and to request permission for each additional volume.
The Amanda::Taper::Scribe constructor takes two arguments:
taperscan and feedback. The first specifies the taper scan
algorithm that the Scribe should use, and the second specifies the
Feedback object that will receive notifications from the Scribe (see
below).
my $scribe = Amanda::Taper::Scribe->new(
taperscan => $my_taperscan,
feedback => $my_feedback);
Once the object is in place, call its start method.
Start the scribe's operation by calling its start method. This will invoke
the taperscan algorithm and scan for a volume. The method takes two parameters:
$scribe->start(
dump_timestamp => $ts,
finished_cb => $start_finished_cb);
The timestamp will be written to each volume written by the Scribe. The
finished_cb will be called with a single argument - undef or an error
message - when the Scribe is ready to start its first dump. The Scribe is
"ready" when it has found a device to which it can write, although it does not
request permission to overwrite that volume, nor start overwriting it, until
the first dump begins (that is, until the first call to start_dump).
Once the Scribe is started, begin transferring a dumpfile. This is a
three-step process: first, get an Amanda::Xfer::Dest::Taper object from the
Scribe, then start the transfer, and finally let the Scribe know that the
transfer has started. Note that the Scribe supplies and manages the transfer
destination, but the transfer itself remains the responsibility of the caller.
Call get_device to get the first device the xfer will be working with.
$device = $scribe->get_device();
This method must be called after start has completed.
Call check_data_path, supplying the data_path requested by the user.
if (my $err = $scribe->check_data_path($data_path)) {
# handle error message
}
This method must be called after start has completed and before
get_xfer_dest is called, it return undef on success or an error message
if the supplied data_path is incompatible with the device.
Call get_xfer_dest to get the transfer element, supplying information on how
the dump should be split:
$xdest = $scribe->get_xfer_dest(
max_memory => $max_memory,
# .. split parameters
);
This method must be called after start has completed, and will always return
a transfer element immediately.
The underlying Amanda::Xfer::Dest::Taper handles device streaming
properly. It uses max_memory bytes of memory for this purpose.
The arguments to get_xfer_dest differ for the various split methods.
For no splitting:
$scribe->get_xfer_dest(
# ...
split_method => 'none');
For buffering the split parts in memory:
$scribe->get_xfer_dest(
# ...
split_method => 'memory',
part_size => $part_size);
For buffering the split parts on disk:
$scribe->get_xfer_dest(
# ...
split_method => 'disk',
part_size => $part_size,
disk_cache_dirname => $disk_cache_dirname);
Finally, if the transfer source is capable of calling
Amanda::Xfer::Dest::Taper's cache_inform method:
$scribe->get_xfer_dest(
# ...
split_method => 'cache_inform',
part_size => $part_size);
An Amanda::Taper::Scribe object can only run one transfer at a time, so
do not call get_xfer_dest until the dump_cb for the previous start_dump
has been called.
Armed with the element returned by get_xfer_dest, the caller should create a
source element and a transfer object and start the transfer. In order to
manage the splitting process, the Scribe needs to be informed, via its
handle_xmsg method, of all transfer messages . This is usually accomplished
with something like:
$xfer->start(sub {
my ($src, $msg, $xfer) = @_;
$scribe->handle_xmsg($src, $msg, $xfer);
});
Once the transfer has started, the Scribe is ready to begin writing parts to
the volume. This is the first moment at which the Scribe needs a header, too.
All of this is supplied to the start_dump method:
$scribe->start_dump(
xfer => $xfer,
dump_header => $hdr,
dump_cb => $dump_cb);
The c<dump_header> here is the header that will be applied to all parts of the
dumpfile. The only field in the header that the Scribe controls is the part
number. The dump_cb callback passed to start_dump is called when the
dump is completely finished - either successfully or with a fatal error.
Unlike most callbacks, this one takes keyword arguments, since it has so many
parameters.
$dump_cb->(
result => $result,
device_errors => $device_errors,
size => $size,
duration => $duration,
total_duration => $total_duration);
All parameters will be present on every call.
The result is one of "FAILED", "PARTIAL", or "DONE". Even when
dump_cb reports a fatal error, result may be "PARTIAL" if some data
was written successfully.
The final parameters, size (in bytes), duration, and total_duration
(in seconds) describe the total transfer, and are a sum of all of the parts
written to the device. Note that duration does not include time spent
operating the changer, while total_duration reflects the time from the
start_dump call to the invocation of the dump_cb.
TODO: cancel_dump
When all of the dumpfiles are transferred, call the quit method to
release any resources and clean up. This method takes a typical
finished_cb.
$scribe->quit(finished_cb => sub {
print "ALL DONE!\n";
});
The get_bytes_written returns the number of bytes written to the device at
the time of the call, and is meant to be used for status reporting. This value
is updated at least as each part is finished; for some modes of operation, it
is updated continuously. Notably, DirectTCP transfers do not update
continuously.
The Amanda::Taper::Scribe::Feedback class is intended to be
subclassed by the user. It provides a number of notification methods
that enable the historical logging and driver/taper interactions
required by Amanda. The parent class does nothing of interest, but
allows subclasses to omit methods they do not need.
The request_volume_permission method provides a means for the caller
to limit the number of volumes the Scribe consumes. It is called as
$fb->request_volume_permission(perm_cb => $cb);
where the perm_cb is a callback which expects a single argument:
undef if permission is granted, or reason (as a string) if permission
is denied. The default implementation always calls perm_cb->(undef).
All of the remaining methods are notifications, and do not take a callback.
$fb->notif_new_tape(
error => $error,
volume_label => $volume_label);
The Scribe calls notif_new_tape when a new volume is started. If the
volume_label is undefined, then the volume was not successfully
relabled, and its previous contents may still be available. If error
is defined, then no useful data was written to the volume. Note that
error and volume_label may both be defined if the previous
contents of the volume were erased, but no useful, new data was written
to the volume.
This method will be called exactly once for every call to
request_volume_permission that calls perm_cb->(undef).
$fb->notif_part_done(
partnum => $partnum,
fileno => $fileno,
successful => $successful,
size => $size,
duration => $duration);
The Scribe calls notif_part_done for each part written to the volume,
including partial parts. If the part was not written successfully, then
successful is false. The size is in bytes, and the duration is
a floating-point number of seconds. If a part fails before a new device
file is created, then fileno may be zero.
Finally, the Scribe sends a few historically significant trace log messages
via notif_log_info:
$fb->notif_log_info(
message => $message);
A typical Feedback subclass might begin like this:
package main::Feedback; use base 'Amanda::Taper::Scribe::Feedback';
sub request_volume_permission {
my $self = shift;
my %params = @_;
$params{'perm_cb'}->("NO VOLUMES FOR YOU!");
}
This page was automatically generated Tue Nov 19 20:05:35 2013 from the Amanda source tree, and documents the most recent development version of Amanda. For documentation specific to the version of Amanda on your system, use the 'perldoc' command.