XFA/Xfer Mechanisms

Individual transfer elements communicate with their upstream and downstream elements via xfer mechanisms. The available mechanisms are tailored to the different kinds of transfer elements Amanda uses. When assembling elements into a transfer, the XFA examines the mechanisms supported by all of the constituent elements, and selects the most efficient combination, adding "glue" code if necessary.

Available Mechanisms

As of this writing, four mechanisms are available:

XFER_MECH_READFD

In this mechanism, the downstream element is given a file descriptor from which it should read. This might be used, for example, for an element that executes a compression application in a pipeline.

XFER_MECH_WRITEFD

In the mirror image, the upstream element is given a file descriptor to which it writes data.

XFER_MECH_PULL_BUFFER

Here, the downstream element calls a method of the upstream element to fetch a buffer of arbitrary size.

XFER_MECH_PUSH_BUFFER

And in this mechanism, the upstream element calls a method of the downstream element to "push" a buffer of arbitary size.

Combinations and Glue

Each element lists the *pairs* of mechanisms that it supports. For example, a simple buffer-based encryption filter might be able to pull data from upstream (PULL_BUFFER) in response to a pull request from its downstream (PULL_BUFFER), or to push data downstream (PUSH_BUFFER) every time its upstream pushes data (PUSH_BUFFER). This element would have two supported pairs:

• (PUSH_BUFFER, PUSH_BUFFER)
• (PULL_BUFFER, PULL_BUFFER)

In neither case does this element "own" the thread in which it is operating -- it is merely reactionary. Imagine it were combined with an FdSource:

• (NONE, READ_FD)

and an FdDest:

• (WRITE_FD, NONE)

There is no simple combination of mechanisms which will correctly encrypt the data -- in fact, none of the elements contain a loop that would drive the transfer of data.

The solution is to add glue elements between the given elements to "translate" between these mechanisms. At least one such glue element should implement a loop in a new thread. One possible combination is:

FdSource --READ_FD--> Glue1
         --PUSH_BUFFER--> CrytpoFilter
         --PUSH_BUFFER--> Glue2
         --WRITE_FD--> FdDest

where Glue1 runs a loop:

 while not eof:
   buf = read(upstream.fd)
   downstream.push_buffer(buf)

and Glue2 simply calls write from its push_buffer mechanism. The result is a single thread which reads bytes from the source file, places them in a buffer, encrypts the buffer, and writes it out to the destination fd. This is about as efficient as this operation can be!

This is a simple example, but the XFA has a complete set of glue elements and can link any collection of transfer elements with maximal efficiency.