Difference between revisions of "Wishlist/Features planned"

From The Open Source Backup Wiki (Amanda, MySQL Backup, BackupPC)
Jump to navigationJump to search
m
Line 127: Line 127:
  
 
* write multiple copies to multiple (possibly heterogenous) backup devices (8mm, DLT, and LTO) to increase backup media diversity, and increase chances of recovery.  the same method could possibly be used for duplicate (mirrored) backups, and might only need to be implemented at the taper stage, provided sufficient holding disk.  [[User:Tfinn|Tfinn]] 10:36, 23 August 2007 (PDT)
 
* write multiple copies to multiple (possibly heterogenous) backup devices (8mm, DLT, and LTO) to increase backup media diversity, and increase chances of recovery.  the same method could possibly be used for duplicate (mirrored) backups, and might only need to be implemented at the taper stage, provided sufficient holding disk.  [[User:Tfinn|Tfinn]] 10:36, 23 August 2007 (PDT)
 +
 +
* How about a Webmin module. I have a lot of PC's that each have their own backup and none of them use the same type of backup. I'm using various methods to do the backups and would like to migrate to Amanda for all, so a web interface would be helpful.

Revision as of 01:20, 14 March 2008

Jean-Louis Martineau, Stefan G. Weichinger, AMANDA Core Team Oct. 2004.

  • Samba: Ports to non-Unix platforms, specifically Macs and PCs. The hooks are in the AMANDA protocol to support non-dump backup programs, but no-one has volunteered to implement the client side. sgw: Mac OS X is able to run the client, so only the PC is left, and I suggest that we should go the SAMBA-way, I think. Samba is working well, is documented and developed further, so I think this will be a stable path to go and support. And people don't need to compile/install anything on their Win-boxes, just add a user and shares ... opinions welcome.
  • Samba: Samba should be treated as a different backup program, not as GNUTAR, because it cannot handle dump-style incrementals.
  • Samba: multiple exclusion-patterns. Since Samba 3.0.3 smbclient supports the usage of multiple exclude-patterns. This would enable AMANDA to exclude more than one pattern per SMB-share, allowing to exclude pagefile.sys AND the registry-files, for example.
  • Instead of hard-coding the interface with tape devices in AMANDA, there should be a higher level interface that allowed different storage devices to be used. AMANDA should also be able to retain backups in disk, even after they are taped, for faster restore of recently backed up data. It should also be possible to store a single backup in multiple tapes, for redundancy.
  • We need a better protocol between the driver and dumpers. setup terminated (to not start to dump on the same host at the same time). driver should ask periodicaly if the dumper is still alive (in case the dumper hang).
  • Retry failed backups in a single run. If backup fails because of active filesystems or lack of memory, AMANDA could throw the failed backup away and run it again, instead of trying it again in the next run only.
  • Support for client-initiated backups might be interesting, but the server would have to keep listening for clients backup requests for a configurable period of time. This could be used to back up secure hosts, for instance.
  • Backups to remote tape devices (i.e., not in the main AMANDA server), as well as to remote filesystems should be supported.
  • multi-tape : AMANDA should be able to write to many tape at the same time. Need some criteria to select which dump should go on which tape? By level, host name, ???
  • A way to tell if some dump must be done before/after some other. (eg. DLE X must be started after DLE Y is started/dumped/taped).
  • Write to tape and holding disk in parallel (For dump to tape), the dump to tape could be started first, while doing some dump to holding disk.
  • Keep files on holding disk after taped, that will permit faster recovery because they will be from holding disk, these dump will be erase when the same is needed for newer dump.
  • Append to tape
  • chg-disk

This script writes to disks which can be accessed in a parallel way (contrary to the serial access to tapes). This could enable AMANDA to do writes and reads to several vtapes in parallel (e.g. doing an amrestore while the regular amdump is running).

It would be helpful to have a script which generates the needed directory-structure for a given chg-disk configuration. This script should test for valid settings (using amgetconf to get the values out of amanda.conf), create the necessary slot-directories and label the new vtapes by using amlabel. (there are drafts available already)

  • amrecover should be able to set and "rewind" the correct vtape. Currently it is necessary to do this manually in another tty.
  • It should be possible to re-generate databases and indexes from tapes.
  • AMANDA could append meta-data like databases and indexes to tape, so that each tape contains its own current indexes and everything to rebuild the server-config.
  • AMANDA should install man-pages for installed programs only.
  • It should be possible to configure whether amidxtaped should decompress the dump stream or not (so amrecover could decompress it locally).
  • amstatus:
    • It should read degraded schedule and write which are delayed.
    • It should print number of byte written to tape for the current flush.
    • The taper should write a file with a byte count for the current files (every GB) and amstatus could read it.
    • It could report the expected time before the dump finishes.
  • amverify/ amverifyrun:
    • It should look at the log file and compare the result.
  • amrecover:
    • should cd, add, remove, ... with a path with glob or regex (cd o*/linux)
    • find [file] # where is that file in the current DLE? (I don't know the path)
    • when [file] # when was this file dumped?
    • parsing accept '\': cd HP890\ Color
    • our own completion
  • amkill:
    • A new script to kill all process on client and server
  • Enhance the protocol between client-server to allow white-space and any character in DLE/exclude/include.
  • More tools in amadmin. The administrator should be able to look at the database in various ways. Adding / deleting / moving disks and hosts should be done through amadmin instead of editing the disklist directly. This will allow AMANDA to do some sanity checks for the operators, to make sure permissions are set up right, etc.

Suggested by Chris Jones <[email protected]>.

  • You should be able to force full dumps for nights other than tonight. Rather than one command at a time on the command line, amadmin could be a little shell with a help facility (ala ckermit or gnuplot, if you've seen those).
  • A tape-verify pass after the AMANDA run (we already have one, but it doesn't work with dump as well as it does with GNU tar). Perhaps taper could calculate a CRC for each file and store that in the database, to be checked by the verifier.
  • More sophisticated tape management. Should AMANDA track tapes globally, counting the number of times tapes were used, and recommending retirement for tapes at the appropriate time?
  • Automatically notice that external dumps have been done. Sendsize could also notice if a filesystem was dumped externally from AMANDA. Right now the planner assumes that the incrementals it is doing are relative to the full dumps it is doing. If someone does a full dump of one of its filesystems (and writes /etc/dumpdates) outside of AMANDA, data could be lost. Sun's Backup-Copilot handles this well. We should too.
  • What if we made the "length" in a tapetype definition always be the "no compression" value? Then change the dumptype "compress" option to accept "hardware" as another type (ala "client" and "server") and let planner do its normal thing with that information (including "comprate", which at the current default of 50% is the usual first guess for hardware compression). This would make setting the tape length value less confusing, and make the amtapetype program easier to run. You could even get more accurate planning than what is currently available by setting the comprate to what you know the data is like on a dumptype by dumptype basis. Suggested by John R. Jackson <[email protected]>.
  • The way to specify the schedule should be redesigned, all those strategy (standard, nofull, noinc, incronly, force-full) and options (skip-full, skip-incr) are confusing. We should have two options, one for full dump and one for incrementals.
    • full [AUTOMATIC | SKIP | NOTIFY | FORCE | FIXED]
    • incr [NONE | BUMP | NOBUMP] with the following values:
      • AUTOMATIC: follow AMANDA scheduling (allow promoted and delayed)
      • SKIP : full dump are done externally on an fixed schedule, dump nothing when a full is due (like skip-full).
      • NOTIFY : full dumps are done externally, but are notified with the NOTIFY command ( amadmin <conf> notify <host> <disk>).
      • FORCE : full dumps are done only with the FORCE_FULL command.
      • FIXED : do full dumps on a fixed schedule (like skip-incr).
      • NONE : don't do incremental dumps.
      • BUMP : allow incremental dumps to bump.
      • NOBUMP : do not allow incremental dumps to bump.
  • Remove all compiled options that can be moved to a (the?) configuration file. (eg. GNU tar path, if configure can't find it, AMANDA should be able to use GNU tar if the path is specified on a client config file) Many people would like this, it would maybe also bring us closer to the possibility of working and usable rpms?
  • Documentation:
    • There is pretty much going on with the AMANDA-docs. The docs have been converted to Docbook/XML and form the new module xml-docs in the AMANDA-CVS-repository.
    • The xml-docs need more formatting and reviewing.
    • The tapetypes from the FOM should go into the XML-docs.
    • The docs would benefit from adding some illustrations.


Suggestions from Fran Fabrizio (zmanda at franfabrizio dot com) Jan 2006:

  • Pre and Post Processing config. An easy (easier?) way to specify tasks to be run on the client prior to starting the dump (or I suppose actually the estimate?) such as database dumping, etc...
  • Bottleneck Analysis. It's often hard to tell from currently available reports where bottlenecks might be occuring. A script or docs to distill this data into something more meaningful ("The system was blocked at two dumpers 82% of the run because Amanda did not have any more network bandwidth available to it." or similar would be killer). Something more informative/analytical than amplot.
  • Auto-naming of volumes based on bar code labels.
  • Amrecover "Load tape CIS-003" or similar - would be nice to offer option to load this volume for you. Easy to do?? Paul Bijnens pointed out how to do this...

in amanda.conf:
  amrecover_do_fsf true
  amrecover_check_label true
  amrecover_changer "changer"
# amrecover -d changer 

Suggestion from Terry Burton (tez at terryburton dot co dot uk) October 2007:

  • When using amrecover with a changer device, optionally avoid the "Load tape XXX now" prompt unless automatic loading to the tape fails. This reduces the supervision required when performing high dump level restores.

Suggestion from Holger Adams (adams at internet-xs dot de) April 2006:

  • Add a process priority option to the the dumptypes. Heavily loaded machines break down when I start Amanda at peak hours.

Suggestion from Rémi Demarthe (remi dot demarthe at obs dash besancon dot fr) May 2006:

  • Choose the amadapass file in amanda.conf to be more flexible. The path /etc/amandapass is hard coded in client-src/findpass.c.

Suggestion from Holger Adams (adams at internet-xs dot de) July 2006:

  • Move the decompression to the client-side (amrecover). It requires a lot of time to transfer >50Gb over ethernet (also with 1 Gbit/s). Maybe you can add a commandline to the config file where you can choose where the decompression takes place?
  • write multiple copies to multiple (possibly heterogenous) backup devices (8mm, DLT, and LTO) to increase backup media diversity, and increase chances of recovery. the same method could possibly be used for duplicate (mirrored) backups, and might only need to be implemented at the taper stage, provided sufficient holding disk. Tfinn 10:36, 23 August 2007 (PDT)
  • How about a Webmin module. I have a lot of PC's that each have their own backup and none of them use the same type of backup. I'm using various methods to do the backups and would like to migrate to Amanda for all, so a web interface would be helpful.