Archived copy. This page was recovered from the Internet Archive snapshot of /display/KB/Tools taken on 2011-04-23. The original wiki at wiki.opf-labs.org is being decommissioned.

Tools

Added by Andrew Jackson · last edited by Frank Feng · on Apr 08, 2011 (view change)

File format management utilities

DROID (Digital Record Object Identification): Automated identification of file formats.
JHOVE (JSTOR/Harvard Object Validation Environment) & JHOVE2: Provides functions to perform format-specific identification, validation, and characterization of digital objects.
XENA (XML Electronic Normalising for Archives): Detects the file formats of digital objects; converts digital objects into open formats for preservation. Part of NAA DSPS tool suite.
Fido: A python implementation of the DROID identification system, using regular expressions.
Fine Free File Command: Identifies a very wide range of files, although not focused on preservation issues.
FITS: The File Information Tool Set (FITS) identifies, validates, and extracts technical metadata for various file formats. Aggregates results from other tools.
NZNL Metadata Extraction Tool: See here for further information.
JMimeMagic: Although not in development at present, this open source identifier is very close to 'file' in structure, and may provide a useful base for new developments.

File integrity utilities

ACE (Audit Control Environment): Validates the integrity of digital files through mathematical techniques.
- Note that Andrew Jackson uses ACE-AM to manage multi-Terabyte data prior to curation and later ingest into our digital library system.
Checksum Checker: Monitors the contents of a digital archive for data loss or corruption. Part of NAA DSPS tool suite.
FastSum: Manifest creation and checking with a reasonably usable GUI.
Manifest Maker: Supports the transfer of data objects by producing a manifest file which satisfies the requirements for a digital transfer. Part of NAA DSPS tool suite.
JackSum: Support 58 popular standard algorithms. Integrated with File Browser.

File transfer utilities

BagIt Library: Java software library that supports the creation, manipulation and validation of BagIt bags.
BagIt Transfer Utilities: Collection of tools for validation and transfer of BagIt bags.
GNU Wget: File transfer utility: Permits retrieval of files using HTTP, HTTPS and FTP
rsync: Checksum-validated robust re-startable transfers of all sorts. Useful for cloning large collections reliably over a flaky connection.