Monthly Status Updates/May 2007

May 2007 saw the release of Amanda 2.5.2, which was followed by a flurry of discussion and development activity.

Wiki Changes

• Taper scan algorithm was supplemented with additional text, mostly by Chris Hoogendyk, with some additional contributions from Ian Turner.
• The new Coding Guidelines portal, mostly written by Dustin Mitchell, provides information useful to new Amanda developers about various generally useful functions available in the codebase.
• Dustin Mitchell expanded and organized the User documentation, Developer documentation, Amanda log files, and Installation portals and linked pages, with help from Chris Hoogendyk.
• The new Platform Experts program started to take off, with six registered experts agreeing to test 10 platforms.
• The new How To Help page by Dustin Mitchell provides an outlet for new contributors of all strokes.
• Paul Bijnens wrote some text about the "Warning - block NNN is beyond the end of DEVICE" message from dump.
• Dustin Mitchell ported a large number of FAQs from the amanda.org documentation and performed significant work on the Amanda Index, Curinfo, and Encryption articles.

Forums Discussions

• A few users had their issues resolved, most notably:
  • gregc had problems restoring on Windows; these were addressed by Jean-Louis Martineau and Paddy Sreenivasan.
  • swsisrom had some problems with backup and restore and questions about performance on his large (7TB) installation using split dumps. These were addressed by Jean-Louis Martineau, Dustin Mitchell, Greg Grant, Sachin Kale, and Kevin Till.

Mailing List Discussions

amanda-users

• Richard Stockton reported problems running Amanda 2.5.2 on Solaris resulting from the new IPV6 code. Jean-Louis Martineau provided a fix.
• There was some discussion about testing of tapes using amverify or tape monitoring. Lewis Donofrio posed the initial question; Gene Heskett pointed out that the act of running amverify will actually increase the likelihood of tape failure, Jon Labadie pointed out that one could use snapshots to confirm data integrity in the presence of a changing data source and that there is no substitute for trained and tested restore procedures, and Chris Hoogendyk wrote some text about the proper statistical sampling technique to use when estimating the tape failure rate.
• Peter Bowman reported a number of problems running Amanda 2.5.2 on an old IRIX installation (6.5.4m, released May 1999); Jean-Louis Martineau fixed these in collaboration with the reporter.
• Steven Backus and Christopher McCrory reported problems with Autoflush, which were fixed by Jean-Louis Martineau.
• Brian Cuttler asked a question about the right version of GNU tar on Solaris; Dustin Mitchell provided links to various precompiled versions and pointed out that Amanda looks for gtar (not tar) in its path at runtime if it can't find an absolute location at configure time. Brian tried tar 1.13, but it turned out to be buggy, so Jon Labadie pointed out that tar 1.13.19 or 1.13.25 are a safer bet that vanilla 1.13, but Brain ended up successful with tar 1.15.91. Stephan Weichinger suggested the possibility of detecting good and bad versions of tar at runtime, but no further progress was made.
• Olivier Nicole started a discussion about hot-swappable hard disks for vtapes. Kai Zimmer suggested eSata as being cheap, reliable, and easy to replace; Dustin Mitchell suggested that a FireWire enclosure provides good performance and reliability at a reasonable price; Jon Labadie brought up the possibility of a removable iomega device; and Geert Uytterhoeven suggested that Olivier should be able to get much better performance with his existing configuration.
• Jeanna Geier had problems setting up vtapes; it turned out most of her problems were from following an out-of-date HOWTO. Her configuration was also confused by the presence of the GNU 'backup' tool, which is different from the dump-like tool of the same name. Stephan Weichinger, Robert Echlin, Paul Bijnens, and Dustin Mitchell all offered their help.

amanda-hackers

This month featured an extensive thread about testing started by Tobias Weingartner, with contributions from a large number of community members. After much discussion, the general consensus was that Amanda releases could use testing on a wider variety of platforms, architectures, and hardware. Jon Labadie suggested creating an automated test suite that users could run; Dustin Mitchell agreed, and said he was already working on an automated test environment for Zmanda and suggested integrating community testers into the suite. Jean-Louis Martineau proposed a Platform Experts program to coordinate testing and configuration for specific platforms. Since then, a number of users have signed up for a variety of platforms.

• Mark Muehlfeld and Hasib reported networking problems with 2.5.2, which were addressed by Jean-Louis Martineau and Dustin Mitchell respectively.

Code Development

Jean-Louis Martineau made some final bugfixes early in the month before releasing Amanda 2.5.2. After the release (and the end of the associated freeze), a large number of other changes landed on the trunk. Many of these were older changes from Zmanda that were merged into the Sourceforge subversion repository. Thanks to Dustin Mitchell and Jean-Louis Martineau for spending many hours on these code merges.

• Jean-Louis Martineau made a number of fixes, including:
  • Update the manpages, xml-docs, and configure scripts for Amanda 2.5.3.
  • Fix various ipv4 and ipv6 bugs in the security API, plus some other Security API fixes and added debugging.
  • Fix amcrypt-ossl.sh and amcrypt-ossl-asym.sh to use the proper client username's home directory, rather than assuming the amandabackup user.
  • Fix a bug where some disklists would cause Amanda executables to crash.
  • Improved shell script portability.
  • Smarter dumper timeout to accomodate slow systems.
  • Make amcheck check all the various tapelist filenames, not just tapelist, in response to a problem reported by John (no relation to John Franks).
  • Use printf __attribute__ for GCC to generate warnings for bad printf arguments, and fix all the outstanding bad arguments.
  • Split amreport FAILED and STRANGE dump summary into two different sections.
  • Change chg-manual so as to not eject the loaded tape on a "slot current" operation following an "eject" operation.
  • Fix a bug where planner would complain about overwriting the last dump even if it was on the holding disk.
  • Change amstatus to report compressed sizes.
  • One minor bugfix in dumper.
• Dustin Mitchell made a number of improvements, including the following. He also did substantial work on publishing and integrating other patches elsewhere in this list that had been previously overlooked.
  • Improve the handling of built manpages by detecting the docbook xslt stylesheets version and insisting on an adequately recent one. Jean-Louis Martineau expanded on this patch by ensuring that the new files were properly distributed.
  • Reorganize and expand the collection of ipv4/ipv6 compatibility functions, with support from Jean-Louis Martineau. This includes support throughout the codebase for ipv6 hostname resolution.
  • Standardize the way the various security-api plugins are built.
  • Combine redundant code for reading holding disk headers.
  • Fix the parsing of include/exclude directives as applied to older clients.
  • Add configure support for Mac OS X.
• Pieter Bowman submitted a patch to fix the setting of permissions on temporary files.
• John Franks made numerous changes throughout the codebase:
  • Improved portability when debugging I/O in various places.
  • Proper handling of rename() on cygwin, which doesn't overwrite by default.
  • Print nanosecond timestamps in debug files.
  • Various debugging code refactoring and improvements, with supplemental changes by Dustin Mitchell.
  • Some new string allocation functions, and fixes to old ones. Dustin Mitchell followed up on this patch with some additional changes.
  • Kerberos 4 and 5 fixes.
  • Initial support for internationalization using gettext. This patch included build changes and code changes to use _() throughout the codebase, plus support for _T() for static strings and large buffers for error messages. Jean-Louis Martineau and Kevin Till contributed additional changes to this work.
  • A number of build fixes on Solaris and elsewhere.
  • Disable IPV6 on cygwin, whose support is not ready yet.
  • Trim executable names in the tape header, on systems (namely Windows) with an executable extension (namely .exe).
• Kevin Till made some other important changes:
  • Move the cryptographic scripts into common-src/, so they can be used by both client and server.
  • Properly handle holding disk directory names containing spaces.
  • Set the default usage for interfaces to 8000.
• Maitreyee Karmarkar improved a number of amcheck messages.
• Satya Ganga rewrote amcleanup from scratch. The new version searches for and optionally kills all the active processes (and their children) for a given configuration.