Driver-Chunker protocol

Communication method

Driver talks via two pipes connected to each chunker's stdin and stdout. The commands and responses are newline-terminated plain text lines.

Like the Driver-Taper protocol, this protocol is almost turn-based, although there are situations where either side may send a message. In particular, there is a race condition between the EOF from the dumper and the DONE/FAILED command from the driver: chunker must do some gymnastics to request more space after the dumper has sent EOF but before the chunker has reached EOF.

Notes

Unlike dumper or taper, the chunker only operates on one dumpfile before exiting.

The driver gives the chunker one or more filenames to write to. However, the chunker appends .tmp to these filenames before writing to them. The driver takes care of renaming these files after the chunker has exited.

Chunker rounds the supplied USE and CHUNKSIZE values down to the next multiple of DISK_BLOCK_KB (32).

Commands

START

Driver -> Chunker:

 START timestamp

Sets the chunker's timestamp. The Driver will next send either QUIT or PORT-WRITE.

QUIT

Driver -> Chunker:

 QUIT

PORT-WRITE

Driver -> Chunker

 PORT-WRITE handle filename host features disk level dumpdate chunksize progname use options

Start writing a dump to holding disk. The handle uniquely identifies this operation, but a given chunker only performs one operation at a time so this is not required. The combination of filename, chunksize, and use correspond to those described for CONTINUE.

The chunker will respond either with PORT or TRYAGAIN (if TRYAGAIN, then the chunker will exit immediately after transmission). If the creation of the first chunk file fails, it may also respond with NO-ROOM followed by TRYAGAIN and exit.

PORT

Chunker -> Driver

 PORT header-port ip-port-pairs

Indicates that the driver should cause a connection to header-port containing the header, followed by a connection to one of the ip-port-pairs containing the data. Note that the chunker will close the header connection before accepting the data connection. This is similar to the PORT command in the Driver-Taper protocol.

The chunker will block waiting for the incoming header connection, although if there is an error or timeout it will respond with TRYAGAIN and exit. Once the data starts flowing, it may send NO-ROOM, RQ-MORE-DISK, or if it reaches EOF on the data conncetion, chunker will begin waiting for a status command (DONE, TRYAGAIN, FAILED, or ABORT_FINISHED) from the driver.

NO-ROOM

Chunker -> Driver

NO-ROOM handle unused-space

This is an informational message indicating that the chunker has actually run out of space while writing (e.g., encountered ENOSPC on a write() operation). The chunker will subsequently send RQ-MORE-DISK. The unused space remaining in the latest USE request is given in unused-space.

RQ-MORE-DISK

Chunker -> Driver

RQ-MORE-DISK handle

This indicates that the chunker needs further instruction on where it should write. The driver should reply with one of CONTINUE or ABORT.

CONTINUE

Driver -> Chunker

CONTINUE handle filename chunksize use

This instructs the chunker to continue writing up to use kb of data at filename using chunk size chunksize. The chunker goes back to work and may send another NO-ROOM, RQ-MORE-DISK, or wait for a status command as described under PORT, above.

ABORT

Driver -> Chunker

ABORT message

The chunker should abort the current run with the given message. Chunker should reply with an ABORT-FINISHED message.

ABORT-FINISHED

Chunker -> Driver

ABORT-FINISHED handle

Indicates that the abort operation is complete. The chunker will then send a status (FAILED) and exit.

DONE

Driver -> Chunker

DONE handle

Indicates that the entire dumpfile was sent to the chunker, and that it should log a SUCCESS message and reply with a DONE message.

Chunker -> Driver

DONE handle datasize-kb "[sec sec kb kb kps kps]"

Indicates that the chunker has successfully completed the dumpfile, and will subsequently exit.

PARTIAL

Chunker -> Driver

PARTIAL handle datasize-kb "[sec sec kb kb kps kps]"

Indicates that the chunker wrote part of the dump, but that the dump was not complete. The chunker will subsequently exit.

FAILED

Driver -> Chunker

FAILED handle

Sent after an EOF on the data stream, this indicates that the entire dumpfile was not sent to the chunker, due to a dumper error, and that the chunker should send PARTIAL or FAILED and log the dump as such. The driver expects a FAILED or PARTIAL response.

Chunker -> Driver

FAILED handle "[sec sec kb kb kps kps]"
FAILED handle "[error_message]"

Indicates that the chunker failed to record any of the dump. The chunker will subsequently exit.

TRYAGAIN

Chunker -> Driver

TRYAGAIN handle errmsg

This indicates that the chunker could not start. The chunker will exit immediately afterward.