How to create custom plugins for MySQL ZRM?

From The Open Source Backup Wiki (Amanda, MySQL Backup, BackupPC)
Jump to navigationJump to search

MySQL ZRM provides plugin interfaces to tailor the ZRM backups to the MySQL server environment. Currently, there are five plugin interfaces.

  1. Pre Backup plugin
  2. Post Backup plugin
  3. Pre Scheduler plugin
  4. Binary log parser plugin
  5. Copy plugin

Pre Backup plugin

Pre Backup plugin is called by mysql-zrm before the backup run for a backup set is started. This plugin can be used for preparing the MySQL server environment for backup. For example: Notification can be sent to all MySQL database users that the backup is going to begin.

Pre Backup plugin is optional.

Configuration

Pre Backup plugin can be configured in the global mysql-zrm.conf or backup set specific mysql-zrm.conf configuration file.

The pre-backup-plugin parameter should be set to the full path to the plugin. The recommended location for plugins in /usr/share/mysql-zrm/plugins directory. The pre-backup-plugin-options parameter can be used to send a list of parameters to the pre-backup plugin.

Plugin interface

The first parameter is the value of pre-backup-plugin-options specified in the mysql-zrm.conf configuration file.

The second parameter to the plugin is the databases/tables that will be backed up in the backup run. The parameter can be one of the following:

  • --all-databases - all databases in the MySQL server are being backed up
  • --database <db1 db2 ...> - select list of databases are being backed up
  • --database <db> and --tables <table1 table2 ...> - select list of tables in the database are being backed up.

The plugin should return 0 on success and and non-zero value on failure. If the pre-backup plugin fails, the backups for the backup set are not attempted and backup status is set to "Backup failed".

Template

/usr/share/mysql-zrm/plugins/pre-backup.pl is a template for a pre-backup plugin. Users are expected to modify it to suit their environment.

Logs

Errors and Messages from the plugin are logged to mysql-zrm logs (/var/log/mysql-zrm/mysql-zrm.log).

Post Backup plugin

Post Backup plugin is called by mysql-zrm after the backup run for a backup set (backup is complete). This plugin can be used for cleaning up the MySQL server environment after backup run. Post Backup plugin is optional.

Configuration

Post Backup plugin can be configured in the global mysql-zrm.conf or backup set specific mysql-zrm.conf configuration file.

The post-backup-plugin parameter should be set to the full path to the plugin. The recommended location for plugins in /usr/share/mysql-zrm/plugins directory. The post-backup-plugin-options parameter can be used to send a list of parameters to the pre-backup plugin. The post-backup-plugin-options parameter is optional.

Plugin interface

The first parameter is the value of post-backup-plugin-options specified in the mysql-zrm.conf configuration file.

The second parameter to the plugin is the databases/tables that were backed up in the backup run. The parameter can be one of the following:

  • --all-databases - all databases in the MySQL server were backed up.
  • --database <db1 db2 ...> - select list of databases were backed up
  • --database <db> and --tables <table1 table2 ...> - select list of tables in the database were backed up.

The plugin should return 0 on success and and non-zero value on failure. If the post-backup plugin fails, the backup status is set to "backup done with errors". Backup for the backup set is completed and databases/tables can be recovered from the backup images.

Post Backup plugin is executed even when the backup actually fails. So it can be used for cleaning up the backup environment.

Template

/usr/share/mysql-zrm/plugins/post-backup.pl is a template for a post backup plugin. Users are expected to modify it to suit their environment.

Logs

Errors and messages from the plugin are logged to mysql-zrm logs (/var/log/mysql-zrm/mysql-zrm.log).

Pre Scheduler plugin

The pre scheduler plugin allows administrator to control when the backup run is started. It allows the administrator to delay backup run start time from the scheduled time or even skip the backup run. Plugin can delay the backup run for a backup set in hourly intervals. This can be useful when backups are being done for 24*7 mission critical databases. One plugin use case to do backups when the MySQL application load or server load is below certain minimum threshold. Another use case is to prevent doing backups when MySQL application or database is being upgraded.

The pre scheduler plugin is optional and can be specified globally or for a backup set. The pre-scheduler plugin is executed before backup run is started.

Configuration

The pre-scheduler-plugin parameter in the mysql-zrm.conf should be set to the full path to the plugin.

Plugin interface

The first parameter to the plugin is the backup set name. The syntax is

--backup-set <backup_set_name>

If plugin returns 0, backups are performed immediately. If the plugin returns "n" where n is a positive integer less than or equal to 11, the backup run is tried again after "n" hours. If the plugin returns any number greater than 11, the backup run is skipped. Skipping a backup run does not have any impact on the other scheduled backup runs for the backup set. So, the backup can be delayed upto 11 hours in one execution of the plugin.

The mysql-zrm-scheduler will skip a daily backup run if the pre-scheduler plugin could not find a suitable time ( has not returned 0 ) to do a backup within 24 hours from the initial scheduled time. For weekly or monthly backup runs, this tool will perform a backup at the end of the 24 hours from the initial scheduled time even if pre-scheduler plugin have not found a suitable time in the 24 hours cycle.

Template

/usr/share/mysql-zrm/plugins/pre-scheduler.pl is a template for a pre scheduler plugin. Users are expected to modify the file to suit their environment. The example plugin pre-scheduler.pl uses the number of processes running in the system to estimate the server load and delays the backup by 1 or 2 hours. If the number of server process is higher than a threshold, it skips the backup run by returning a number greater than 11.

Logs

Errors and messages from the plugin are logged to mysql-zrm-pre-scheduler logs (/var/log/mysql-zrm/zrm-pre-scheduler.log").

Binary log parser plugin

Selective recovery of MySQL database requires the administrators to look at the database events in the incremental dumps using binary log parser. The MySQL binary logs can have lots of database events (millions of database events/transactions in many cases). The binary log parser plugin can help in the filtering the database events of interest. For example: Administrator interested only in tables being dropped can create a plugin to filter "DROP TABLES" database events.

Usage

The binary log parser plugin is optional and is specified as a command line parameter to mysql-zrm tool. Example:

# mysql-zrm --action parse-binlogs --bin-logs "/var/lib/mysql-zrm/backupset1/20060819121532/mysql-bin.0" \
  --parse-binlogs-plugin "/usr/share/mysql-zrm/plugins/parse-binlogs.pl" \
  --parse-binlogs-plugin-options="DROP TABLE"

parse-binlogs-plugin parameter should have the full path to the plugin as parameter and parse-binlogs-plugin-options takes strings as parameter. The parse-binlogs-plugin-options values are passed to the binary log parser as argument.

Plugin interface

The first parameter is the parse-binlogs-plugin-options value that was passed as parameter to mysql-zrm.

The second parameter is the string containing one database event. The fields are separated by white spaces and fields are in following format:

Log filename | Log Position | Timestamp | Event Type | Event

The plugin should return 0 if the line should be displayed and 1 if it should be filtered out.

Template

/usr/share/mysql-zrm/plugins/parse-binlogs.pl is a template for binary log parser plugin. Users are expected to modify it to suit their environment.

Logs

Errors and messages from the plugin are logged to mysql-zrm logs (/var/log/mysql-zrm/mysql-zrm.log").

Copy plugin

Copy plugin is the interface to transfer files from the MySQL server machine to machine running MySQL ZRM and vice versa in a secure manner. The MySQL server and machine running MySQL ZRM might be separated by a firewall and only some file transfer protocols such as ssh or ftp might be permitted.

Copy plugin is used when

  • Incremental backups of remote MySQL server is performed
  • Replication files from a remote MySQL server are transferred to machine running MySQL ZRM
  • Raw backups of remote MySQL server using MySQL tool - mysqlhotcopy is performed
  • Database is being restored to a remote machine

Configuration

The copy plugin is optional and can be specified globally or for a backup set. It must be noted that copy plugin for the above mentioned scenarios.

The copy-plugin parameter in the mysql-zrm.conf should be set to the full path to the plugin.

Copy plugins (ssh and socket) read configuration parameters from mysql-zrm.conf. ZRM plugin parameters - ssh-user, remote-mysql-binpath, socket-remote-port can be used for configuring the sample copy plugins.

Plugin interface

If the plugin is used for performing raw backups (using mysqlhotcopy) of remote MySQL server, the first parameter will be --mysqlhotcopy[=path_to_mysql_binaries]. The remaining parameters will be same as mysqlhotcopy command line parameters. If "raw" backup method is not used, it is not necessary to implement "mysqlhotcopy" interface.

In all other circumstances, the plugin takes four parameters:

  • --source-host
  • --source-file
  • --destination-host
  • --destination-directory

The file (argument to --source-file) from server (argument to --source-host) must be securely transferred to server (argument to --destination-host) to the directory (argument to --destination-directory).

If the plugin exit status is 0, the file transfer is considered as successful. Otherwise, it is treated as failure. Copy plugin failure might result in database backup or database restoration failure.

Template

Two examples of copy plugin are available in /usr/share/mysql-zrm/plugins and /usr/share/mysql-zrm/plugins directories. There are README files available in /usr/share/doc/MySQL-zrm*/ directory that document the configuration changes required to use the example plugins. Users are expected to the file to suit their environment.

The ssh based plugin is called ssh-plugin.pl and the socket based plugin is called socket-copy.pl. The socket copy plugin requires MySQL ZRM socket server package to be installed on all MySQL servers. The socket-server package installs xinetd/inetd socket server service and reloads the xinetd/inetd.

Logs

Errors and messages from the plugin are logged to mysql-zrm logs (/var/log/mysql-zrm/mysql-zrm.log). The error messages from socket server plugin on the MySQL server is logged to /tmp/client.log file on the MySQL server.