Mysql-zrm: Difference between revisions

From wiki.zmanda.com
Jump to navigation Jump to search
No edit summary
 
(19 intermediate revisions by the same user not shown)
Line 1: Line 1:
== NAME ==
==NAME==


mysql-zrm - MySQL backup and recovery tool
mysql-zrm - Backup and recovery utility for ZRM for MySQL


==SYNOPSIS==
==SYNOPSIS==
        
       mysql-zrm --action [ backup |
mysql-zrm --action <backup|restore|list|purge|parse-binlogs|check|verify-backup>
                            restore |
          [--backup-set <name>]
                            schedule-backup |
          [--source-directory <directory name>]
                            report |
          [--replication | --noreplication]
                            list |
          [--backup-name <name>]
                            purge  |
          [--all-databases]
                            parse-binlogs  |
          [--databases <"name1 name2 ...">]
                            check   |
          [--destination <directory name>]
                            verify-backup |
          [--database <name> [--tables <"name1 name2...">]]
                            extract-backup |
          [--backup-level <0|1>]
                            abort-backup ]
          [--backup-mode <raw|logical>]
          [--lvm-snapshot <size>]
          [--retention-policy   <backup retention time>]
          [--bin-logs <"name1 name2 ...">]
          [--start-position <#>]
          [--stop-position <#>]
          [--offset <#>]
          [--start-datetime <name>]
          [--stop-datetime <name>]
          [--index|--noindex]
          [--till-lastfull]
          [--all-backups]
          [--options-file <filename>]
          [--mailto <mail address>]
          [--user <user>]
          [--password <password>]
          [--host <hostname>]
          [--port <portnumber>]
          [--mysql-binpath <location  of MySQL commands>]
          [--ssl-options <"MySQL ssl options">]
          [--comment <"Note about the backup">]
          [--quiet|--no-quiet]
          [--verbose]
          [--help]


==DESCRIPTION==
==DESCRIPTION==


This tool is part of Zmanda  Recovery  Manager for  MySQL (MySQL ZRM) and can  be used to backup and restore MySQL databases. This tool does full and incremental backups as well  as full and selective
ZRM for  MySQL  provides  automated scheduling of live MySQL database backup.  You only need to configure it once to reflect the MySQL deployment, then schedule it to run unattended backups. Various options can help an administrator:
restores.  The tool also provides  a list option to provide information about the prior backups. The tool also implements retention  policy  for the backup imagesThis  tool  can  be  used to parse binary logs to find out the timestamp and/or log position of specific events. The action ''check'' can be used to verify the configuration of ZRM. The action ''verify-backup'' verifies consistency of backup images.


Users are expected to schedule backups with mysql-zrm tool using [[mysql-zrm-scheduler]](1) toolMySQL database recovery has to be done using mysql-zrm tool.
* Verify the backups created.
* Generate various types of reports about the backups created.
* Specify compression and encryption as desired.
* Report information about the backups via an email or RSS feed.
* Access pre- and post- backup plugins to extend the suite further.
* Control full and selective  restore operations  on  the database, both locally and remotely.


===BACKUP SETS===
ZRM  for  MySQL ’s  primary  command  line  utility  is mysql-zrm, which provides the core backup/restore functions for MySQL databases.


MySQL ZRM organizes the backups in terms of backup sets. Each backup set defines the list of databases or tables within a database, how it should be backed up and the backup schedule for the data. Backup set is identified by an unique string for each MySQL ZRM instance.
Use  the [[mysql-zrm.conf]](5)  configuration file to set mysql-zrm options.


Every  backup run (each time mysql-zrm is  executed)  is associated with one backup set.
Use the --action check option to verify the configuration of mysql-zrm.


The mysql-zrm  tool reads  global MySQL ZRM configuration file (/etc/mysql-zrm/mysql-zrm.conf). The  global configuration file can be overridden by backup set specific configuration  file
Use mysql-zrm  to create full or incremental backups of MySQL  databases and perform full, incremental and selective restores as well. The --list option provides information about prior backups.
stored  in /etc/mysql-zrm/<backup set name>/ directory. The mysql-zrm command line options will override the  parameters specified in the configuration files.


The order in the which backup set parameters will be read:
To  display  help on individual action options, append the --help  option to the appropriate command line. For  example:
# /etc/mysql-zrm/mysql-zrm.conf
  mysql-zrm --action backup --help
# /etc/mysql-zrm/<backup set>/mysql-zrm.conf
# Parameters read from --options-file command line  argument
# Options specified on the command line


===BACKUP METHODS===
===BACKUP SETS===


The  mysql-zrm  full backups can be logical backups or raw backups. Logical  backups  contain  SQL  statements  to  recreate  the database. Raw backups are actual copy of the database files. If ''backup-mode'' is  specified as ''raw'' the tool has the ability to decide which backup mechanism to use for a particular database. If there are no transactional storage engine based tables  in  the  specified  database,  then  the utility will use mysqlhotcopy  for  backing up  that  database.  Otherwise, it  will  use mysqldump  for backing up the specified database. If ''backup-mode'' option        is  specified  as logical then only mysqldump will be used for backing up specified databases and tables.
ZRM for MySQL organizes the backups into ''backup sets''.  Each backup set defines:


If the ''lvm-snapshot'' option is specified and if the specified database is  put on an LVM volume then a snapshot of that volume is taken and the database   is  backed  up from  that snapshot.  If  the database is not on a lvm volume then the mysqlhotcopy or mysqldump will be used as per the logic in the last paragraph.
* a list of database(s) or table(s) within a database to back up
* backup options to use on this backup set
* scheduling options to use on this backup set.


If the ''replication'' option is specified  and  if  the mysql server is a slave server, then the utility will also backup all replication related files.
Backup sets are identified by a unique name.


If the ''destination'' option is specified  then the backups are stored in a sub-directory under the specified directoryDefault value is "/var/lib/mysql-zrm". If the destination option is specified and does not exist, the utility will throw an error and exit.
Every backup run (each time mysql-zrm is executed) is  associated with one  backup  setVarious parameters can be associated with each backup run.


An index file is also created in the backup directory. The index file contains the details of the backup run.
mysql-zrm    first    reads    the    global      configuration  file (/etc/mysql-zrm/mysql-zrm.conf).  The  global configuration file can be overridden  by  any  backup  set  configuration    file stored  in ''/etc/mysql-zrm/backup set  name/'' directory. The mysql-zrm command line options will override the parameters specified  in  the configuration files.
===BACKUP METHODS===


If incremental backup is specified, then the options ''backup-mode'', ''lvm-snapshot'', ''all-databases'', ''databases'', ''database'' and ''tables'' are ignored.
mysql-zrm  backups  can  be either  full or incremental. Full backups can be logical backups or raw backups. Logical backups contain SQL statements  to recreate the database. Raw backups are actual copies of the database files.


===RECOVERY===
When logical backup is specified, the  utility  uses  only mysqldump for backing up specified databases and tables.


For restores, only  ''all-databases''  and  ''databases''  options are  supported. So if the backup contains only specific tables from a database, you need to specify mysql-zrm  action  as  restore and  the database name  to restore  the  backed  up  tablesThis will only restore the backed up        tables. If there are other tables in the  databasethey will be left untouched.
When  raw  backup  is  specified,  the utility has the ability to decide which backup mechanism to use for particular database.  If there are no transactional storage engine based tables in the specified database, it uses mysqlhotcopy for backing up that database. Otherwiseit  uses mysqldump for backing up the specified database.


===OTHER ACTIONS===
An  index file is also created in the backup directory.  The index contains the details of what was backed up, how much data was backed  up, what  parameters  were  used to backup the data, how much time it took to backup etc.


If the ''list'' action  is  specified,  the tool will display the backup level, the name of the directory containing the backup and the index of the last backup of the specified backup set.
===RECOVERY===
 
If the ''all-backups'' option is specified then the above specified data will be displayed for all of the backups  that  has  happened  till  date for the given backup-set.
 
If  the ''till-lastfull'' option is specified then the data since last full backup will be displayed.
 
If the ''source-directory'' option is  specified  then  the index  of  that backup is displayed.
 
If  the  ''noindex'' option  is specified then the level of backup and the name of the directory containing the backup will  be  displayed  and  the contents of the index file will not be displayed.
 
If  the  ''parse-binlogs''  action  is specified, the utility will display the parsed output of the binary logs. This is useful  to  find out  the  log positions  and/or  timestamp  to be used for restore operations.  Either the ''source-directory'' or  the  ''bin-logs''  option  should  be specified. If the ''source-directory''  option  is  specified  then  the parsed output of the binary logs from that backup will be  displayed.  If ''bin-logs''  option  is specified  then  parsed output  of  the binary log file specified will be displayed.
 
If the ''purge'' action is specified, the utility will remove all backups whose retention policies have been exceeded in the backup directory specified by the ''destination'' option.
 
If the ''verify-backup'' action is specified, the utility will check if the backed up data is consistent. The directory containing the backup needs to be specified using the ''source-directory'' option.
 
==OPTIONS==
 
; --action <backup|restore|list|purge|parse-binlogs|check|verify-backup> : Specify  which  action to do. This option cannot be specified  in the configuration file.
 
; --backup-set <name> : Specify  backup-set  for  the  backup run. This option cannot be specified  in  the configuration file.  The default value is ''BackupSet1''
 
; --source-directory <directory name> : Specify  the  directory containing a backup. This option is              ignored if the action specified is backup. For restore  action, both  ''source-directory'' and ''bin-logs'' options  should not be specified.
 
; --replication | --noreplication : Specify  if  replication files should be backed up. The replica              tion  related  files  will  be  backed  up only  if the host is a replication  slave.  Default option is ''--noreplication''
 
; --backup-name <name> : Specifies the name of the directory in which  backup  should  be done. This  should be a unique name for each backup run. If not specified, the current timestamp will be used as the  backup name.
 
; --all-databases : Specifies  that  all  databases should be backed up or restored.  This is the default if ''databases'' or ''database'' is not specified.
 
; --databases <"name1 name2 ..."> : Specifies  the  specific databases to backup or restore.
 
; --database <name> [--tables <"name1 name2 ...">] : Specifies which specific tables  should  be  backed up. This option is only supported during the backup operation.
 
; --destination <directory name> : Directory  to  which  backup should be done. Default location is ''/var/lib/mysql-zrm/''. If this  option  is  specified and the  directory  must  be  exist and should be writable by the MySQL ZRM user.
 
; --backup-level <0|1> : If 0 is specified, then a full backup is  done.  If 1  is  specified,  then  incremental backup is done. Default is full backup.
 
; --backup-mode <raw|logical> : If ''raw''  is  specified  then  mysqlhotcopy is  used to backup the              specified  database/tables if the database does not contain  any tables that uses a transactional engine. Otherwise, mysqldump is used.  If  logical  backup  mode  is specified then mysqldump is only used. Default value is ''raw''.
 
; --lvm-snapshot <size> : This  specifies the size of the lvm snapshot to  be used. For raw              backups,  each specified database is checked to see if that is on a  LVM volume. If it is then  a  snapshot  of  the specified size is created and that is used to backup up the database. If  not  it will  use  either  mysqlhotcopy  or mysqldump based on the logic described in the ''backup-mode'' option. This option is ignored  if the ''backup-mode'' is ''logical''.
 
; --retention-policy <backup retention time> : Backup  image  for  this  backup  run will retained for the time specified as parameter. A suffix of ''D'' indicates time specified in  days.  A  suffix of  ''W''  indicates  amount specified in weeks (number of days are computed as 7*amount).  A suffix of ''M'' indicates  amount  in  months  (number  of  days  are computed  as 30*amount). A suffix of ''Y'' indicates amount specified  in  years (number  of  days  are  computed  as  365*amount). For e.g. if --retention-policy  10M  is  specified,  it  indicates  that  the backup  should  be retained for  300  days.  If  no suffix is specified,  the amount specified is assumed to be number of  days  the backup  should  be retained. The default  value  is retain backups forever.
 
; --bin-logs <"name1 name2 ...> : List  of  binary  log files to be used for restore. The  full  path              for  each  file  should  be  specified. For ''restore'' action, both ''source-directory'' and ''bin-logs'' options should not be specified.
 
; --start-position N : Start selective restore of all events after log position N.  Log positions can be determined using ''parse-binlogs'' action.
 
; --stop-position N : Stop selective restore of all  events  before log position N. Log positions can be determined using ''parse-binlogs'' action.
 
; --offset N : Skip the first N entries of the  first  binary  log file  in  ''bin-logs'' parameter.
 
; --start-datetime <datetime> : Start  selective  restore  from specified date and time (should be            in MySQL DATETIME or TIMESTAMP format).
     
; --stop-datetime <datetime> : Selective restore till  specified  date  and  time (should  be  in MySQL DATETIME or TIMESTAMP format).
 
; --index|--noindex : Specify  if  the  contents of the index file should be displayed during the ''list'' action.
 
; --all-backups : Specifies  that  information  about  all  available backups should be displayed.
 
; --till-lastfull : Specifies  that  information  till  the last full backup should be displayed.
 
; --options-file <filename> : Specify the file containing all of the options.
 
; --mailto <mail address> : Address to which backup report to be sent.
 
; --user <user> : Specify MySQL backup/restore user
 
; --password <password> : Specify password for the MySQL user
 
; --host <hostname> : MySQL server host name or IP name
 
; --port <portnumber> : MySQL server port
 
; --mysql-binpath <mysql binaries directory> : Full path where  mysql  binaries  are  installed. For  example: /opt/lampp/bin
 
; --ssl-options <"MySQL ssl options"> : Any  --ssl*  options that MySQL supports.  Please refer to Zmanda Recovery Manager documentation for SSL options details.
 
; --comment <"Note about the backup run"> : Comment or a note about the backup run. This can be used to  tag backup  runs  and  the  note  can  retrieved  using  [[mysql-zrm-reporter]](1) tool.


; --quiet|--no-quiet : Specifying no-quiet will lead to the  log messages also being displayed on stdout. Default is quiet.
When  restoring  databases,  only  the  all-databases  and databases  options are  allowed.    In  the  case of clusters,  only  the all-databases option is allowed.


; --verbose : Provide more verbose output in the log.
When the backup  contains  only  specific  tables  from  a database, you need to  specify  mysql-zrm  action as restore and the database name to restore the backed up tables.  Tables not included in  the  backup but present in the database will be left untouched.


; --help : Display  help  message and exit. This option cannot be specified in the options file.
===ACTIONS===


==EXAMPLES==
Specify backup to initiate a backup run.


For  backup  the most common usage of mysql-zrm will be to specify replication,  an appropriate lvm-snapshot size and the destination directory for  backup.  Usually, these backup parameters are ecified in the configuration  file, [[mysql-zrm.conf]](5) and the MySQL backups are done using [[mysql-zrm-scheduler]](1).
Use restore to restore from a specified backup.


  mysql-zrm  --action  backup --replication  --lvm-snapshot 100MB
Use schedule-backup to setup the schedule for the backup.


For a logical backup of specific tables you would use:
Specify report to generate reports on backup runs.


mysql-zrm  --action  backup  --replication  --lvm-snapshot 100MB \
Use check to verify if a backup set configuration is correct.
            --backup-mode  logical  --database dbname --tables "table1 table2 table"


For listing contents of a backup the following command can be used:
Use  list  to  display the backup level, the name of the directory containing the backup and the index of the  last  backup  of the  specified backup set.


  mysql-zrm --action list --source-directory /var/lib/mysql-zrm/backupset1/20060818121532
Action parse-binlogs will display the parsed output of the binary logs. This is useful to  find  out  the log positions  and/or timestamp  to  be used for restore operations.


For restoring the following command can be used:
Specify  purge  to remove backups present in the directory specified by the destination option whose retention policy has been exceeded.


mysql-zrm --action restore --source-directory /var/lib/mysql-zrm/backupset1/20060818121532
Specify verify-backup to verify a backup


The above command will  restore  all  databases that were backed up. If the source directory points to a full backup, it will do a full restore else it will do a incremental restore.
Use extract-backup to uncompress/decrypt a specified backup


The following command does a selective restore of all specified binary log files using a single MySQL server connection.
Specify abort-backup to abort the backup run for  given backup-set.


mysql-zrm  --action restore \
===OPTIONS===
            --bin-logs /var/lib/mysql-zrm/backupset1/20060818121532/mysql-bin.[0-9]* \
                      /var/lib/mysql-zrm/backupset1/20060819121532/mysql-bin.[0-9]*


The following command parses and displays the relevant contents of all specified binary log files. This can be used to find out what position and or timestamp to use for selective restores.
  --action backup | restore | schedule-backup | report | list |
          purge | parse-binlogs  |  check  | verify-backup |
          extract-backup | abort-backup


  mysql-zrm --action parse-binlogs \
These mutually exclusive --action options  choose the basic functionality  of  the  utility. This form cannot be used in the configuration file.
            --bin-logs /var/lib/mysql-zrm/backupset1/20060818121532/mysql-bin.[0-9]* \
                        /var/lib/mysql-zrm/back-upset1/20060819121532/mysql-bin.[0-9]*


==FILES==
==FILES==
Line 205: Line 103:
; /var/lib/mysql-zrm : Directory under which all backup data is stored.
; /var/lib/mysql-zrm : Directory under which all backup data is stored.


; /etc/mysql-zrm/<backup set name>/mysql-zrm.conf : Configuration file read by mysql-zrm tool.
; /etc/mysql-zrm/backup set name/mysql-zrm.conf : Configuration file ZRM for MySQL.


==RETURN VALUES==
==RETURN VALUES==


On success, zero is returned. On error, non-zero value is returned.
On success, zero is returned. On error, non-zero value is returned.


==SEE ALSO==
==SEE ALSO==


[[mysql-zrm-scheduler]](1), [[mysql-zrm-reporter]](1),  [[mysql-zrm.conf]](5), mysqldump(1), mysqlbinlog(1), mysql(1), lvm(8), Zmanda  Recovery  Manager  for  MySQL  (http://mysqlbackup.zmanda.com/)
[[mysql-zrm-backup]](1), [[mysql-zrm-manage-backup]](1), [[mysql-zrm-restore]](1),  [[mysql-zrm-check]](1), [[mysql-zrm-list]](1),  [[mysql-zrm-parse-binlogs]](1),  [[mysql-zrm-purge]](1), [[mysql-zrm-extract-backup]](1), [[mysql-zrm-verify-backup]](1), [[mysql-zrm-abort-backup]](1), [[mysql-zrm-scheduler]](1), [[mysql-zrm-reporter]](1),  [[mysql-zrm.conf]](5), mysqldump(1), mysqlbinlog(1), mysql(1), lvm(8)


==AUTHOR==
==AUTHOR==


Zmanda Inc. (http://www.zmanda.com)
Zmanda Inc. (http://www.zmanda.com/)

Latest revision as of 21:16, 26 November 2008

NAME

mysql-zrm - Backup and recovery utility for ZRM for MySQL

SYNOPSIS

      mysql-zrm --action [ backup |
                           restore |
                           schedule-backup |
                           report |
                           list |
                           purge  |
                           parse-binlogs  |
                           check   |
                           verify-backup  |
                           extract-backup  |
                           abort-backup ]

DESCRIPTION

ZRM for MySQL provides automated scheduling of live MySQL database backup. You only need to configure it once to reflect the MySQL deployment, then schedule it to run unattended backups. Various options can help an administrator:

  • Verify the backups created.
  • Generate various types of reports about the backups created.
  • Specify compression and encryption as desired.
  • Report information about the backups via an email or RSS feed.
  • Access pre- and post- backup plugins to extend the suite further.
  • Control full and selective restore operations on the database, both locally and remotely.

ZRM for MySQL ’s primary command line utility is mysql-zrm, which provides the core backup/restore functions for MySQL databases.

Use the mysql-zrm.conf(5) configuration file to set mysql-zrm options.

Use the --action check option to verify the configuration of mysql-zrm.

Use mysql-zrm to create full or incremental backups of MySQL databases and perform full, incremental and selective restores as well. The --list option provides information about prior backups.

To display help on individual action options, append the --help option to the appropriate command line. For example:

 mysql-zrm --action backup --help

BACKUP SETS

ZRM for MySQL organizes the backups into backup sets. Each backup set defines:

  • a list of database(s) or table(s) within a database to back up
  • backup options to use on this backup set
  • scheduling options to use on this backup set.

Backup sets are identified by a unique name.

Every backup run (each time mysql-zrm is executed) is associated with one backup set. Various parameters can be associated with each backup run.

mysql-zrm first reads the global configuration file (/etc/mysql-zrm/mysql-zrm.conf). The global configuration file can be overridden by any backup set configuration file stored in /etc/mysql-zrm/backup set name/ directory. The mysql-zrm command line options will override the parameters specified in the configuration files.

BACKUP METHODS

mysql-zrm backups can be either full or incremental. Full backups can be logical backups or raw backups. Logical backups contain SQL statements to recreate the database. Raw backups are actual copies of the database files.

When logical backup is specified, the utility uses only mysqldump for backing up specified databases and tables.

When raw backup is specified, the utility has the ability to decide which backup mechanism to use for a particular database. If there are no transactional storage engine based tables in the specified database, it uses mysqlhotcopy for backing up that database. Otherwise, it uses mysqldump for backing up the specified database.

An index file is also created in the backup directory. The index contains the details of what was backed up, how much data was backed up, what parameters were used to backup the data, how much time it took to backup etc.

RECOVERY

When restoring databases, only the all-databases and databases options are allowed. In the case of clusters, only the all-databases option is allowed.

When the backup contains only specific tables from a database, you need to specify mysql-zrm action as restore and the database name to restore the backed up tables. Tables not included in the backup but present in the database will be left untouched.

ACTIONS

Specify backup to initiate a backup run.

Use restore to restore from a specified backup.

Use schedule-backup to setup the schedule for the backup.

Specify report to generate reports on backup runs.

Use check to verify if a backup set configuration is correct.

Use list to display the backup level, the name of the directory containing the backup and the index of the last backup of the specified backup set.

Action parse-binlogs will display the parsed output of the binary logs. This is useful to find out the log positions and/or timestamp to be used for restore operations.

Specify purge to remove backups present in the directory specified by the destination option whose retention policy has been exceeded.

Specify verify-backup to verify a backup

Use extract-backup to uncompress/decrypt a specified backup

Specify abort-backup to abort the backup run for a given backup-set.

OPTIONS

--action backup | restore | schedule-backup | report | list  |
         purge  |  parse-binlogs  |  check  | verify-backup |
         extract-backup | abort-backup

These mutually exclusive --action options choose the basic functionality of the utility. This form cannot be used in the configuration file.

FILES

/var/lib/mysql-zrm
Directory under which all backup data is stored.
/etc/mysql-zrm/backup set name/mysql-zrm.conf
Configuration file ZRM for MySQL.

RETURN VALUES

On success, zero is returned. On error, non-zero value is returned.

SEE ALSO

mysql-zrm-backup(1), mysql-zrm-manage-backup(1), mysql-zrm-restore(1), mysql-zrm-check(1), mysql-zrm-list(1), mysql-zrm-parse-binlogs(1), mysql-zrm-purge(1), mysql-zrm-extract-backup(1), mysql-zrm-verify-backup(1), mysql-zrm-abort-backup(1), mysql-zrm-scheduler(1), mysql-zrm-reporter(1), mysql-zrm.conf(5), mysqldump(1), mysqlbinlog(1), mysql(1), lvm(8)

AUTHOR

Zmanda Inc. (http://www.zmanda.com/)