Device API

From The Open Source Backup Wiki (Amanda, MySQL Backup, BackupPC)
Revision as of 19:54, 14 December 2008 by Dustin (talk | contribs) (→‎write_block: write_block no longer takes a short_write parameter)
Jump to navigationJump to search

The Device API is a clean interface between Amanda and data-storage systems. It provides a tape-like model -- a sequence of bytestreams on each volume, identified only by their on-volume file number -- even of non-tape devices.

Background

Device API Features

The Device API also adds a number of new features not previously available in Amanda. These include:

  • Device properties allow drivers to describe themselves (and the devices and media they control), as well as allow arbitrary user settings to propagate down to the driver.
  • Smart locking means unrelated accesses can be performed without issue, while conflicting accesses wait for one another.
  • Appending to volumes is supported for capable devices.
  • A device can describe its supported blocksize range to the Amanda core, instead of the other way around.
  • Deleting parts of volumes (without erasing the whole thing) is supported for capable devices.
  • The amount of free space on a device can be reported to the Amanda core, where the device supports it (e.g., VFS Device)

History

The Device API is designed to replace the ancient tapeio subsystem (sometimes called the vtape API or Virtual Tape API), originally introduced to support virtual tapes. The original design of the tapeio system (located in tape-src/) was to abstract tape-related functionality into a separate library. Thus, different devices (tape, vtape and RAIT) were all phrased in terms of tape operations: Rewind, fast-forward, read a block, etc. Operations were assumed to have the same semantics as a UNIX tape device. Furthermore, the API revealed a file descriptor, with the assumption that other parts of Amanda could perform operations such as stat() or dup2() on it.

The Device API clears up a large number of limitations and device assumptions that the tapeio system made, allowing native support for new devices (such as CDs and DVDs) as well as new device-related functionality (such as parallel access, partial recycling, and appending).

The Device API does not change (or even address) media formats.

What the Device API is not

The Device API does not distinguish between random-access and linear-access media: The seek operation may take a long time for some devices, or it may be instantaneous for others. It does, however, distinguish between concurrent devices and exclusive devices: Concurrent devices may be accessed by multiple readers (and sometimes multiple writers) simultaneously, while exclusive devices cannot.

Although the Device API deals in on-medium headers and blocks, it is otherwise agnostic to media format issues. Changes to the Amanda media format, including split vs traditional formats, can be made without change to the Device API.

Using the Device API

Broadly speaking, the Device API revolves around a single glib virtual class -- Device -- which is actually implemented as one of many possible subclasses. Subclasses are identified with device-types, seen by the user as prefixes, such as tape: for the Tape Device or s3: for the S3 Device. Device names are a combination of a device type and an arbitrary string that is interpreted by the subclass. The API takes the form of a set of methods, member variables, and behaviors common to all devices.

The Device API is available from perl as Amanda::Device, using normal Perl object syntax. All member variables are available via accessor functions. Example:

$success = $device->finish_file();

The C interface defines methods with the device_ prefix, taking the device object as a first argument, e.g.,

success = device_finish_file(device);

Most of the discussion here is agnostic to the language in use.

Error Handling

Like the C library, most Device methods return a particular value to signal the presence of an error condition. In many cases, this is simply false (exceptions are listed below).

When a device signals an error, its status and error contiain details of what went wrong. Status is a bitfield where each bit that is sets indicates a possible problem. Unfortunately, some devices are unable to distinguish states due to limitations of an external system. For example, the tape device often cannot distinguish an empty drive (VOLUME_MISSING) from a hard device error (DEVICE_ERROR), leading to the status DEVICE_STATUS_VOLUME_MISSING|DEVICE_STATUS_DEVICE_ERROR.

DEVICE_STATUS_SUCCESS (zero)
All OK.
DEVICE_STATUS_DEVICE_ERROR
The device is in an unresolvable error state, and further retries are unlikely to change the status
DEVICE_STATUS_DEVICE_BUSY
The device is in use, and should be retried later
DEVICE_STATUS_VOLUME_MISSING
The device itself is OK, but has no media loaded. This may change if media is loaded by the user or a changer
DEVICE_STATUS_VOLUME_UNLABELED
The device is OK and media is laoded, but there is no Amanda header or an invalid header on the media.
DEVICE_STATUS_VOLUME_ERROR
The device is OK, but there was an unresolvable error loading the header from the media, so subsequent reads or writes will probably fail.

In addition to the status bitfield, a Device also provides a user-comprehensible error message, available from the methods error() (returning the error message), status_error() (returning the string form of the status), or status_or_error() (returning the error message if one is set, otherwise the string form of the status). None of these functions will ever return NULL/undef.

At the risk of being repetitive, never test a device's status with ==, unless it is to DEVICE_STATUS_SUCCESS. Furthermore, never try to parse the device error messages -- they are only for user consumption, and may differ from device to device.

Properties

Device properties provide a bidirectional means of communication between devices and their users. A device provides values for some properties, which other parts of Amanda can use to adjust their behavior to suit the device. For example, Amanda will only attempt to append to a volume if the device's properties indicate that it supports this activity. Some devices have additional properties that can be set to control its activity. For example, the S3 Device requires that the users' keys be given via properties.

See amanda-devices(7) for more information on device properties and their meanings.

The methods property_get and property_set are used to get and set properties, respectively. If the indicated property simply does not exist, these functions return an error indication (FALSE), but the device's status remains DEVICE_STATUS_SUCCESS. If a more serious error occurs, then the device's status is set appropriately.

From Perl, device properties are easy to handle, as the Perl-to-C glue takes care of all necessary type conversions.

$success = $device->property_set("BLOCK_SIZE", $blocksize);
$blocksize = $device->property_get("BLOCK_SIZE");

if there is an error fetching the property, property_get returns undef.

From C, properties are a little more complicated, as they use the Glib GType system. The following example provides a start:

  {
    GValue val;
    gboolean success;
    uint blocksize = 32768;
    
    /* set value */
    bzero(&val, sizeof(val));
    g_value_init(&val, G_TYPE_UINT);    /* property type */
    g_value_set_uint(&val, blocksize);  /* fill in value */
    success = device_property_set(device,
                                  PROPERTY_BLOCK_SIZE,
                                  &val);

...

    /* get value */
    bzero(&val, sizeof(val));
    if (device_property_get(self, PROPERTY_BLOCK_SIZE, &val)
        || !G_VALUE_HOLDS(&val, G_TYPE_UINT)) {
        blocksize = g_value_get_uint(&val);
    } else {
        /* handle error */
    }
  }

The method property_list returns a list of a device's available properties. This function also provides informaion on when the properties may be read or written, although this information is not currently used and is not available from Perl.

Data Model

A volume is a container for data which can be "loaded" into a particular device. For tape devices, a volume is a tape, but most other devices do not deal with such physical objects. A volume has a volume header giving, among other things, the label of the volume and the timestamp on which it was written. The header may also indicate that the volume is not an Amanda volume. Aside from the header, a volume contains a sequence of files, numbered starting at 1. While writing, devices number files sequentially, but devices that support partial volume recycling may have "holes" in the sequence of file numbers where files have been deleted. The seek_file section, below, describes how the API represents this situation. Each file has a header, too, which contains lots of information about the file. See dumpfile_t in Amanda::Types for the full list. After the header, a file is just a sequence of bytes.

Reads and writes to devices take place in blocks. Unlike a typical operating-system file, in which any block boundaries are lost after the file is written, devices must be read back with the block sizes that were used to read. See amanda-devices(7) for more in block sizes, and the read_block and write_block sections, below, for more information.

Member Variables

A device object has the following member variables:

file
the current file number, if any
block
the current block number, if any
in_file
true if the device is in the middle of reading or writing a file
device_name
the name with which the device was constructed; note that this is not set until after open_device is finished -- it is an error to access this variable in an open_device implementation
access_mode
the current access mode (ACCESS_NULL, or that supplied to start)
is_eof
true if an EOF occurred on reading, or if the volume is out of space
volume_label
the label of the current volume, set by start and read_label
volume_time
the timestamp of the current volume, set by start and read_label
volume_header
the header of the current volume, set by read_label
status
the device's error status
block_size
the device's currently configured block size. This is also available via the BLOCK_SIZE property. Writers should use block_size-byte blocks, and readers should initially use block_size, and expand buffers as directed by read_block().
min_block_size
minimum allowed block size for this device

max_block_size : maximum allowed block size for this device

These are available as regular structure members in C. From Perl, they are available via accessor methods:

print $dev->device_name(), "\n";

Methods

Device users generally call device methods in the following order:

  1. open
  2. read_label (optional)
  3. start
    • ACCESS_READ (repeated):
      1. seek_file (optional)
      2. read_block (repeated)
    • ACCESS_WRITE or ACCESS_APPEND (repeated):
      1. start_file
      2. write_block (repeated)
      3. finish_file
  4. finish

device_open

dev = device_open("tape:/dev/nst0");
$dev = Amanda::Device->new("tape:/dev/nst0");

This function takes a device name and returns a device object. This function always returns a Device, although it may be a Null device with an error condition. Any call to device_open should be followed by a check of the device's status:

$dev = Amanda::Device->new($device_name);
if ($dev->status() != $Amanda::Device::DEVICE_STATUS_SUCCESS) {
  die "Could not open '$device_name': " . $dev->error();
}

This function does not access the underlying hardware or any other external systems in any way: that doesn't happen until read_label or start.

The member variable device_name is set when this function has returned.

read_label

status = device_read_label(dev);
$status = $dev->read_label();

This function reads the tape header of the current volume, returning the Device's status. Since this is often the first function to accses the underlying hardware, its error status is the one most often reported to the user. In fact, amdevcheck(8) is little more than a wrapper around read_label.

The method sets the following member variables:

volume_header
if any header data was read from the volume, it is represented here. The header's type may be F_WEIRD if the header was not recognized by Amanda.
volume_label
if read_label read the header successfully, then volume_label contains the label
volume_time
smililarly, if read_label read the header successfully, then volume_time contains the timestamp from the header

start

success = device_start(dev, ACCESS_WRITE, label, timestamp);
$succss = $dev->start($Amanda::Device::ACCESS_WRITE, $label, $timestamp);

Start starts the device and prepares it for the use described by its second parameter. This function can be called regardless of whether read_label has already been called.

If the access mode is ACCESS_WRITE, then label and timestamp must be supplied (although leaving the timestamp NULL/undef will use the current time), and they will be used to write a new volume header. Otherwise, these parameters should be NULL/undef.

On completion, start leaves the device's access_mode, volume_label and volume_time member variables set, by reading the tape header if necessary. Note that in ACCESS_APPEND, the file member variable is not set until after start_file has been called.

start_file

 success = device_start_file(dev, header);
 $success = $dev->start_file($header);

This method prepares the device to write data into a file, beginning by writing the supplied header to the volume. On successful completion, file is set to the current file number, block is zero, and in_file is true.

write_block

 success = device_write_block(dev, blocksize, buf);
 # (not available from Perl)

Finally, it's time to write some data. This method writes a single block of data to the volume. Blocksize must be the device's block size, unless this is a short write. A short write must be the last block of a file. Some devices will zero-pad a short write to a full blocksize. This method returns false on error, including running out of space. If the device can distinguish this condition from other errors, then is_eof is set. This function ensures that block is correct on exit. Even in an error condition, it does not finish the current file for the caller.

write_from_fd

 queue_fd_t *qfd = queue_fd_new(fd, NULL);
 success = device_write_from_fd(dev, qfd);
 if (!success) {
   if (qfd->errmsg)
     fprintf(stderr, "%s", qfd->errmsg);
   if (dev->status != DEVICE_STATUS_SUCCESS)
     fprintf(stderr, "%s", device_status_or_error(dev));
 }
 my $qfd = Amanda::Device::queue_fd_t->new(fileno($fh));
 if (!$dev->write_from_fd($fd)) {
   print STDERR $qfd->{errmsg}, "\n" if ($qfd->{errmsg});
   print STDERR $dev->status_or_error(), "\n" if ($dev->status());
 }

This method reads from the given file descriptor until EOF, writing the data to a Device file which should already be started, not returning until the operation is complete. The file is not automatically finished. This method is only used for testing; real uses of devices use the transfer architecture.

This is a virtual method, but the default implementation in the Device class uses write_block, so there is no need for subclasses to override it.

finish_file

 success = device_finish_file(dev);
 $success = $dev->finish_file();

Once an entire file has been written, finish_file performs any cleanup required on the volume, such as writing filemarks. On exit, in_file is false.

seek_file

 header = device_seek_file(dev, fileno);
 $header = $dev->seek_file($fileno);

In ACCESS_READ, seek_file sets up the device to read from file fileno. This function is not available in ACCESS_WRITE and ACCESS_APPEND. It returns the header from the requested file on success, or NULL/undef on error.

If the requested file doesn't exist, as might happen when a volume has had files recycled, then seek_file will seek to the next file that does exist. The file this function selected is indicated by the file member variable on exit. If the requested file number is exactly one more than the last valid file, this function returns a F_TAPEEND header ($header->{type} == $Amanda::Types::F_TAPEEND).

As an example, on a volume with only files 1 and 3:

 $dev->seek_file(1) returns header for file 1, $dev->file == 1
 $dev->seek_file(2) returns header for file 3, $dev->file == 3
 $dev->seek_file(3) returns header for file 3, $dev->file == 3
 $dev->seek_file(4) returns a tapend header, $dev->file == 4
 $dev->seek_file(5) returns NULL/undef

On exit, is_eof is false, in_file is true unless no file was found (tapeend or NULL), file is the discovered file, and block is zero.

seek_block

 success = device_seek_block(dev, block);
 $success = $dev->seek_block($block);

After seeking to a file, the caller can optionally seek to a particular block in the file. This function will set block appropriately. Note that it may not be possible to detect EOF, so this function may fail to set is_eof even though a subsequent read_block will return no data.

read_block

 bytes_read = device_read_block(dev, buffer, *blocksize);
 # (not available from Perl)

This method is the complement of write_block, and reads the next block from the device, or returns -1 on error. Pass a buffer and its size. If the buffer is not big enough, no read is performed, the parameter size is set to the required blocksize, and the method returns 0. As a special case, passing a NULL buffer and a zero size is treated as a request for the required block size. It is not an error to pass a buffer that is too large (and, in fact, this is precisely the effect of setting the read_block_size configuration parameter).

On EOF, this method returns -1, but sets is_eof and leaves the device's status set to DEVICE_STATUS_SUCCESS. Some devices may be able to detect EOF while reading the last block, and will set is_eof at that time. Others must wait for the next read to fail. It is never an error to call read_block after an EOF, so there is no need to check is_eof except when read_block returns -1.

read_to_fd

 queue_fd_t *qfd = queue_fd_new(fd, NULL);
 success = device_read_to_fd(dev, qfd);
 if (!success) {
   if (qfd->errmsg)
     fprintf(stderr, "%s", qfd->errmsg);
   if (dev->status != DEVICE_STATUS_SUCCESS)
     fprintf(stderr, "%s", device_status_or_error(dev));
 }
 my $qfd = Amanda::Device::queue_fd_t->new(fileno($fh));
 if (!$dev->read_to_fd($fd)) {
   print STDERR $qfd->{errmsg}, "\n" if ($qfd->{errmsg});
   print STDERR $dev->status_or_error(), "\n" if ($dev->status());
 }

This method reads the current file from the device and writes to the given file descriptor, not returning until the operation is complete. This method is only used for testing; real uses of devices use the transfer architecture.

This is a virtual method, but the default implementation in the Device class uses read_block, so there is no need for subclasses to override it.

finish

 success = device_finish(dev);
 $success = $dev->finish();

This undoes the effects of start, returning the device to a neutral state (ACCESS_NULL). After finish, it is not an error to call start again, even with a different mode.

recycle_file

 success = device_recycle_file(dev, fileno);
 $success = $dev->recycle_file(fileno);

On devices that support it, this removes the indicated file from the volume, presumably freeing its space to be used for other files. File numbers of existing files will not change, so this operation may leave "holes" in the sequence of file numbers. See seek_file to see how this is handled.

This method cannot be called while in a file.

erase

 success = device_erase(dev);
 $success = $dev->recycle_file(fileno);

On devices that support it, this erases all data from the volume, presumably freeing the space.

This method must be called before start and after finish — that is, while the device is in a neutral state (ACCESS_NULL). You can detect whether or not this operation is supported using the 'full_deletion' property.

Concurrency

Some devices can perform more than one operation simultaneously, while others are more limited. For example, a tape device is exclusive to a single process while it is in use, while a VFS device can support concurrent reads and writes on the same volume.

As of this writing, device locking is not correctly implemented in many devices; consult the source code and check with the Amanda developers before depending on concurrent operation of devices. If concurrency is implemented and you're still reading this, click the "edit" button up top and fix it, or complain to the developers.

Implementing a Device Driver

To add a new device to the Device API, do the following:

  • Subscribe to the amanda-hackers mailing list and/or connect to #amanda and solicit the help of existing developers.
  • Figure out how the data model and operational model described above will map onto your device.
  • Implement a subclass of one of the existing Device classes named above. In particular, implement the various virtual functions provided for in device.h.
    • Consult the existing device implementations for hints -- in particular, the VFS Device and S3 Device make good examples.
    • Note that in addition to the (mostly virtual) functions discussed above, there are some additional protected functions that you may find useful, described in device.h.
  • Make sure your device calls device_add_property() to register its properties.
  • Arrange to call the register_device() function to register the device, so it will recieve relevant calls to device_open(). The easiest way to do this is to write a initialization function, and add a call to your function from device_api_init() in device.c. Dynamically loadable device drivers are an open project.
  • add documentation of your device to the amanda-devices(7) manpage.

The Device API relies heavily on GLib's type system, so developers unfamiliar with the GObject system are encouraged to consult the relevant documentation.

Properties Interface

Most code related to device-independent properties is in property.h and property.c. Since property values are passed around as a GValue, all value types must be registered in the GLib type system.

Every abstract property has a DevicePropertyBase, which refers to a DevicePropertyId. The Base structure is the same for all properties of all devices; it holds information about the property outside other context. Each property has a unique DevicePropertyId, though the ID of a property is only guaranteed within a single run of a program: Between runs, IDs might change, so it's best to refer to properties by name outside of a particular program.

When a device starts up, it creates a DeviceProperty structure for each supported property. This structure refers to the DevicePropertyBase, but also holds information about when the property may be accessed: Not all properties may be set or gotten at any time. The PropertyAccessFlags field access holds this information: it is the bitwise OR of any of the various PROPERTY_ACCESS_GET_ and PROPERTY_ACCESS_SET_ values defined. Note that there are five distinct "time periods" as far as access is concerned:

  • BEFORE_START: Before device_start() has been called; i.e., before any permanant action may have been taken.
  • BETWEEN_FILE_WRITE: When in write mode, but not actually writing a file. Specifically, after device_start() has been called with ACCESS_WRITE or ACCESS_APPEND, but outside of a pair of calls to device_start_file() and device_finish_file().
  • INSIDE_FILE_WRITE: In the middle of writing a file. Specifically, after device_start_file() has been called, but before the coresponding call to device_finish_file().
  • BETWEEN_FILE_READ: When in read mode, but not actually reading a file. This means after device_start() has been called with ACCESS_READ, but either before a call to device_seek_file, or after a call to device_read_block returns EOF (provided no intervening call to device_seek_file() or device_seek_block() since the EOF).
  • INSIDE_FILE_READ: When in read mode, and while actually reading a file. This means after a call to device_seek_block(), but before a call to device_read_block() returns EOF.

Note that these access flags are not automatically enforced, and are currently never consulted. They exist for future expansion of the properties interface.

Device-specific properties should be defined in your own device, not in property.h. See the S3 device's properties for an example.

Future Directions

Future directions for the Device API include:

  • new devices
  • runtime loading of device modules as shared objects
  • more flexible device configuration