xzip

xzip is a FUSE filesystem for duplicating zip files which share zipped contents.

To install use:

 pip install xzip

or

 easy_install xzip

xzip provides three executables zipexplode, zipanalyze, and mount.xzip which will "explode" a zip, analyze a zip file, and mount exploded zips respectively.

The file structure for an exploded zip is the following:

zipexplode path/to/zip/name-of-zip.zip
tree .
.
├** data
│ ├** < sha1-data-file1 >
│ ├** < sha1-data-file2 >
│ ├** ...
│ └** < sha1-data-filen >
└** meta
 ├** name-of-zip.zip.dir
 ├** name-of-zip.zip.jump
 └** name-of-zip.zip.stream

Data files may be shared between an arbitrary number of exploded zips files, and the meta tuple (*.dir, *.jump, *.stream) describe the original zip file.

zipexplode accepts two options --directory and --depth to modify where it creates the data and meta directories and how many levels deep the data directory should be. zipexplode can explode multiple zip files at once, and additional help is provided with the --help option.

zipanalyze simply prints out the sha1 of different segments of the original zip file. This script was used to determine what could be deduplicated, and what needed to be stored per zip file. This executable is mainly of historical use.

mount.xzip will mount the directory structure described above, and needs to be supplied with matching --directory and --depth arguments to when zipexplode was called. Additional arguments --debug, --background, and --single-threaded which are passed to FUSE and control underlying functionality. For more information see the --help for mount.xzip.

Requirements:

• Python
• Filesystem in Userspace