S tar

S tar or Star is an open source command-line program that can compress/archive many files together into a single tape or disk archive. It is also able to restore individual files from the generated archive.

S tar includes a FIFO for speed, multivolume support, a pattern matcher, the ability to archive sparse files, automatic byte order recognition, automatic archive format detection, automatic archive compression and decompression, remote archiving, as well as other special functionality that allows the application to be used for full system backups.

S tar also includes `rmt', a truly portable version of the remote tape server that supports remote operation between different operating systems and machine architectures, as well as a portable `mt' tape drive control program.

S tar is the fastest known implementation of a tar archiver, even faster than ufsdump in nearly all cases.

What is new in this release:

• The Schily makefilesystem now supports CC++_COM for all platforms to allow to overwrite the low level command for the C++-compiler.
• Fixed a problem in star with "star --xattr -find relative_path ..." Thanks to Philip Spencer for the patch.
• libschily::fexecv() now allows to use the parameter "ac" with the value -1 to indicate that the arg count is to be determined by the NULL pointer at the end of the arg list.
• include/schily/stat.h now contains macros to set the nanoseconds in timestamps in a OS independent way
• New autoconf tests for sys/capability.h and cap_*() functions from Linux -lcap
• WARNING: If you do not see this:
• checking for sys/capability.h... yes
• ...
• checking for cap_get_proc in -lcap... yes checking for cap_get_proc... yes checking for cap_set_proc... yes checking for cap_set_flag... yes checking for cap_clear_flag... yes
• your Linux installation is insecure in case you ever use the command "setcap" to set up file capabilities for executable commands.
• Note that cdrtools (as any other command) need to be capabylity aware in order to avoid security leaks with enhanced privileges. In most cases, privileges are only needed for a very limited set of operations. If cdrtools (cdrecord, cdda2wav, readcd) are installed suid-root, the functions to control privileges are in the basic set of supported functions and thus there is no problem for any program to control it's privileges - if they have been obtained via suid root, you are on a secure system.
• If you are however on an incomplete installation, that supports to raise privileges via fcaps but that does not include developer support for caps, the programs get the privileges without being able to know about the additional privileges and thus keep them because they cannot control them.
• WARNING: If you are on a Linux system that includes support for fcaps (this is seems to be true for all newer systems with Linux >= 2.6.24) and there is no development support for capabilities in the base system, you are on an inherently insecure system that allows to compile and set up programs with enhanced privileges that cannot control them.
• In such a case, try to educate the security manager for the related Linux distribution. Note that you may turn your private installation into a secure installation by installing development support for libcap.
• Allow to compile without Linux libcap using "smake COPTX=-DNO_LINUX_CAPS LIB_CAP="
• The autofconf tests for broken Linux kernel headers now avoid to warn for /usr/src/linux/include if this directory is missing.
• include/schily/priv.h now includes sys/capabilitiy.h if available.
• Trying to support suid-root-less installation of librmt users on Linux. librmt now understands that a non-root program may be able to create sockets for a privileged port.
• WARNING: the include structure of include/schily/*.h and several sources has been restructured to cause less warnings with older OS platforms. If you see any new problem on your personal platform, please report.
• New includefiles:
• schily/poll.h Support poll() schily/stdarg.h An alias to schily/varargs.h (but using the std name) schily/sunos4_proto.h Missing prototypes for SunOS-4.x to make gcc quiet schily/timeb.h Needed for users of ftime()
• Many minor bug-fixes for the files include/schily/*.h
• Added #include to libdeflt
• Added #include to libschily/fnmatch.c
• Added #include to libedc
• Updated README.compile:
• Some typo patches from Jan Engelhardt
• Documented the "LINKMODE=" macro to explain how to create dynamically linked bynaries.
• include/schily/archconf.h now defines __SUNOS5 for easier coding
• include/schily/priv.h now defines platform independent fine grained privileges
• include/schily/stdio.h now implements #ifndef NO_SCHILY_STDIO_H to avoid including stdio.h from wchar.h
• The Schily Makefilesystem no longer uses CC= for internal tasks. People who now set $CC to a different value will fail in a similar way as they would fail with other build systems.
• Allow "static" compilation (a compilation that does not use the dynamic defines from the Schily Makefilesystem) on Linux x86
• Make sure that all schily include files except the primary wrappers include schily/*.h instead of including the system include files directly.
• Added automatic support for ARMv5 and ARMv6. This makes compilation on the RaspberryPI also possible with the non-automake aware gmake.
• Allow "static" compilation (a compilation that does not use the dynamic defines from the Schily Makefilesystem) on Linux ARMv6 (which is used by RaspberryPI).
• Star (spax) no longer extracts files with the original UID/GID if run by root unless the option -po or -pe was in use. Thanks to a hint from Pavel Raiskup
• Many sources have been modified to deal __CYGWIN32__ like __CYGWIN__ This is neded as newer Cygwin versions that run in 64 Bit mode do no longer define __CYGWIN32__ but __CYGWIN__
• Support for NT-6.2 (Win-8) and NT-6.3 (Win-8.1) was aded to the makefile system
• The makefile system now includes support for ppc64-linux-*cc to allow to compile using non automake-aware make programs like gmake.
• New option -xcopy to star is an alias for -copy -sparse -acl
• "star" and "mt" now need libcap on Linux as librmt is aware of Linux capabilities
• "defltsect" was added as exported global function to libdeflt.
• libfind now calls closedir() with -empty even on empty directories.
• New autoconf test on whether includes struct timespec
• New autoconf tests check whether some functions (such as clock_gettime()) are in libc or need a separate library
• New: struct timespec in schily/time.h is granted to exist on all platforms
• New functions getnstimeofday() and setnstimeofday() have been added to libschily.
• Star now supports to extract time stamps with nanosecond granularity on platforms that support utimensat().
• Star now also supports to extract time stamps for symlinks using utimensat(AT_FDCWD, name, tp, AT_SYMLINK_NOFOLLOW).
• Note that star could (on some platforms) also permit to set the file permission for symlinks using fchmodat() but this is already done using umask() since a long time and the only platform that is known to support different permissions on symlinks seems to be HP-UX. On a recent Solaris, you will not be able to set the file permissions for symlinks.
• Star was converted to be completely timespec based. This offers nanosecond granularity for all times on platforms that support a nanosecond clock.
• Star now may be told to modify the path name (in case of -s/o/n/ or interactive change -w) before the time stamp is compared to existing files in the filesystem. This is done by using the new option -uncond-rename

What is new in version 1.5.2:

• Try to make star immune against signals that appear while opening a file from a network filesystem.
• Let the star FIFO better handle the media change operation at the end of a tape volume in multi volume create mode.
• Allow star to auto-detect the "EOF" condition on a raw disk when writing in multi volume create mode. ENXIO has been added as an alias for EFBIG and ENOSPC.
• The star.4 manpage was enhanced.
• star_sym.c now uses O_SEARCH for opening directories.
• star/longnames.c adds more workarounds against wrong buffer overflow aborts from gcc.
• star/extract.c now uses access(name, F_OK) instead of access(name, 0)
• star now only sets up signal handlers for existing signals to support MSC
• star fixed a typo in dumpdates.c that hit on WIN-DOS because flock() is not present
• star now checks HAVE_FSYNC to support MSC
• star::findinfo.c now checks whether stat.st_blksize exists
• star/dumpdates.c works around a problem with LOCK_EX/LOCK_EX/LOCK_UN defined although the flock() call no longer exists.
• New (previously missing) man pages: bsh/pfbsh.1 sh/bosh.1 sh/jsh.1 sh/pfsh.1 mt/mt.1 scgskeleton/scgskeleton.1 label/label.1 star/ustar.1 star_sym/star_sym.1 ved/ved-e.1 ved/ved-w.1
• star now supports automatic recognition of xz(1) compressed archives and offers a new option -xz
• star offers new behavior:
• A tar archive created with "star -c -dump -meta ..." does not include the content for plain files but only the meta data. With "star -c -xmeta" these files could be extracted as 0 by sized files in case that this would not overwrite existing files. Star now supports "star -x -xmeta -fore-hole" to extract these files as files of the original size but as sparse files without any data inside. This is sufficient as a reference for wget -mirror.
• The man page star/spax.1 fixed a typo.
• star/header.c now uses a new structure "ndbuf" that defines a namelength of 101 instead of 100 in order to be able to access the 101th element in the arrays without accessing an aray out of bounds.
• Fixed a typo in star/buffer.c that caused star to dump core on Linux with -c -mulvivol due to the unusual memory layout found on Linux.
• The star -multivol buffer management now mallocs less memory. Before it malloc()d the same size as the FIFO size, now it only malloc()s 2 * bufsize
• star -c -find is now interruptable by ^C
• star -x -U -k now works as well as star -x -U -keep-old
• Some typo fixes in star.4 man page
• Star -sparse -force-hole now archives all files as sparse files in case that they at least contain 512 block aligned bytes of nulls. Thanks to markk@clara.co.uk
• star -find now removes leading "./" in filenames as star does in other cases.
• star -r/-u now correctly depends on the old archive format type.
• Star adds a new option -lzip and automated support to recognize lzip compressed archives. Thanks to markk@clara.co.uk
• star now accepts a command like:
• star -c -onull -find . -ls
• as there is no archive output to stdout that could mix with the -ls output
• Fixed a bug in libfind/walk.c (calling strcatl() with last parameter 0 instead of (char *)0)
• libfind bumped to version 1.5
• New file libfind/find_tok.h to separate the token names from the rest of the implementation.
• Moved token definitions from find.c to find_tok.h
• The primary -perm now implements a new feature:
• -perm +mode will evaluate as true if any of the bits set in mode are set in the permission bits fro a file.
• A bug with -exec introduced with using vfork() instead of fork() was fixed. The replacement of the string "{}" now works correctly again even if the location of the path in memory was changed during a realloc().
• New primary operators: -amin deprecated - only for GNU find compatibility -cmin deprecated - only for GNU find compatibility -mmin deprecated - only for GNU find compatibility -empty TRUE zero sized plain file or empty directory -execdir program [argument ...] \\; -fls file list files similar to 'ls -ilds' into 'file' (always TRUE) -fprint file print file names line separated into 'file' (always TRUE) -fprint0 file print file names nul separated into 'file' (always TRUE) -fprintnnl file print file names space separated into 'file' (always TRUE) -ilname glob TRUE if symlink name matches shell glob -ilpat pattern TRUE if symlink name matches pattern -iname glob TRUE if path component matches shell glob -ipat pattern TRUE if path component matches pattern -ipath glob TRUE if full path matches shell glob -ippat pattern TRUE if full path matches pattern -okdir program [argument ...] \\; -print0 print file names nul separated to stdout (always TRUE) -executable TRUE if file is executable by real user id of process -readable TRUE if file is readable by real user id of process -writable TRUE if file is writable by real user id of process
• libfind no longer exits the process in treewalk() in case that the path strings could not be allocated.
• libfind::walkclose() now free()s the internal data structures in case of a user that does not call treewalk() (e.g. star -x)
• The function defltcntl() in libdeflt is now working and allows to select a case insensitive mode.
• New function defltsect() in libdeflt allows to create sub-sections in configuration files. These subsections need to be named: "[section-name]" and the '[' must be at the beginning of a line.
• Support for "pcc" on Solaris x86 was added as a first step. The C-Compiler "pcc" http://pcc.ludd.ltu.se/ is based on the Portable C-Comppiler from AT&T from the late 1970s. It is under BSDl and thus offers compilation with a really free build environment.
• Note that due to a pcc bug, the autoconf results for Large File support are incorrect. Unless you edit the file incs/i386-sunos5-pcc/xconfig.h and manually correct the file to contain this:
• #define _FILE_OFFSET_BITS 64 /* # of bits in off_t if settable */
• You will not get working large file support with pcc.
• Support for more Operating system will follow after the issues in the compiler have been resolved.
• conf/mkdep-aix.sh now calls "cc -M -qsyntaxonly" instead of "cc -M -E" as the IBM C-preprocessor does not like #pragma weak a = b although the compiler supports it.
• Added support for DragonFly/x86_64
• New autoconf test for nanosecond time stamps on AIX.
• conf/mkdep-aix.sh was changed to avoid warnings for #pragma weak a = b as the IBM C-compiler calls a non "#pragma weak" cpp when called with -E
• include/schily/stat.h now supports nonosecond timestamps in struct stat on AIX.
• include/schily/stat.h now supports struct stat.st_atime_n for AIX nanosecond support.
• Trying to add support for 32 bit compilation on newer Mac OS X version via smake CCOM=cc32 This only works if 32 Bit development support is installed on Mac OS X.
• Trying to add support for 32 bit compilation on newer Linux version via smake CCOM=cc32 This only works if 32 Bit development support is installed on Linux.
• New autoconf test checks whether fnmatch() has a working FNM_IGNORECASE
• New autoconf tests for supporting stateful multibyte locales mbsinit() mbrtowc() wcrtomb()
• New autoconf tests for wctype_t mbstate_t
• New autoconf tests for basename() dirname() strndup() wcsndup()
• include/schily/wchar.h enhanced to support mbrtowc() and wcrtomb()
• include/schily/regex.h added
• include/schily/libgen.h added
• New autoconf test for strlcat() wcslcat() strnlen() wcsnlen()
• New autoconf test for strndup() wcsndup()
• libschily fnmatch() now supports FNM_IGNORECASE
• libschily fnmatch() now supports multibyte characters
• libschily: new files strlcat.c wcslcat.c strnlen.c wcsnlen.c strndup.c wcsndup.c basename.c dirname.c
• A new algorithm in libschily strncat.c strncmp.c strncpy.c strnlen.c wcsncat.c wcsncmp.c wcsncpy.c wcsnlen.c makes the code faster and smaller.
• Conf-files in /etc/default are now installed 0644
• Man pages are now by default installed in $(INS_BASE)/share/man The base directory "share" may be changed by caling
• smake DEFMANBASE=something
• README.compile now mentions:
• INS_RBASE= Set different "root" install path DEFMANBASE= Set different man base directory STRIPFLAGS=-s Strip all binaries with "make install" RUNPATH= Overwrite ELF RUNPATH DEFUMASK= Set different default permissions for creating directories DEFINSUMASK= Set different default permissions for creating directories in the install path
• Man pages are now by default installed in $(INS_BASE)/share/man The base directory "share" may be changed by caling
• All Makefile.dfl now detect whether /etc/default/* already exists
• The macros XARCH_GEN & XARCH_OPT are now in RULES/cc-sunpro*.rul
• The include file include/schily/ccomdefs.h now allows to disable the printf() format checks for GCC with -DNO_PRINTFLIKE This is needed in order to allow star to compile in the Solaris ON cosolidation as GCC id too dumb for the printf() extensions.
• libschily/?eterrno.c no longer overwrite the _TS_ERRNO definition in order to pass the compile constrints in the Solaris ON consoliation.
• Several new global symbols in the mapfile for libschily/libschily-mapvers
• libschily/wcsndup.c had frgotten code and still did internaly implement strndup()
• librmt.so now includes libsocked in the NEEDED ELF tag
• All include files (even those created by autoconf) are now protected against multiple inclusion.
• New autoconf test for pathconf() and fpathconf()
• The linker map files are now processed via "cpp" (or $CC -E) and the output is stored in $(ARCHDIR). This allows to have a single linker map file source for all platforms.
• New autoconf test for a working cpp.
• New autoconf test for /dev/stdin, /dev/stdout, /dev/stderr, /dev/fd/0, /dev/fd/1, /dev/fd2
• New autoconf tests for: qecvt() qfcvt() qgcvt() _ldecvt() _ldfcvt() _ldgcvt() _ldecvt_r() _ldfcvt_r() _ldgcvt_r() qeconvert() qfconvert() qgconvert() needed for long double support.
• All linker map files have been modified to allow platform dependent #ifdef's
• libschily now by default renames getline(), fgetline() and fexec*() into js_getline(), js_fgetline() and js_fexec*(). This was needed because the POSIX standard commitee ignored POSIX basic rules not to define functions that are in conflict with existing published functions. The functions from linschily have been in the public since more than 25 years and the functions defined by POSIX.1-2008 even violate POSIX naming rules. The POSIX fexec*() functions would e.g. have to be correctly called fdexec*().
• Existing programs can keep the official names getline(), fgetline() and fexec*() as they are by default #defined to the new js_*() names. To disable this convenience, add:
• #define NO_SCHILY_GETLINE To disable schily getline(), fgetline() #define NO_SCHILY_FEXEC To disable schily fexec*() #define NO_SCHILY_FSPAWN To disable schily fspawn*()
• Libschily now has new "long double" support functions: qftoes() and qftofs()
• Libschily now supports a locale dependend decimal point for printing floating point numbers.
• Libschily now supports to print long double numbers using the format() based portable printf() implementation. If this does not work for your platform or if this does not work correctly for your platform, please report. This was needed for out new "od" implementation.
• The option parsing implementation getargs()/getfiles() from libschily now supports to combine single character boolean flags even in case the option string mentions to call a callback function if a valid option was found. This was needed for out new "od" implementation.
• filereopen() from libschily no longer calls open() internally as long as there is no need to create or truncate the file.
• include/schily/wchar.h now includes a workyround for platforms that do not have wcwidth().
• libschily's findinpath() now includes a new path parameter to allow to use the function from bsh.
• fnmatch is now always available as js_fnmatch()
• New autoconf test for _qecvt() _qfcvt() _qgcvt() and _qecvt_r() _qfcvt_r() _qgcvt_r() on AIX
• libschily/getexecpath.c new
• include/schily/nlsdefs.h now includes macros _(s) and __(s) to support gettext() and xgettext(1)
• Disable Simple suffix rules and Sufffix rules in smake to speedup builds
• Added a comment in RULES/rules.rdi that this rule is no longer needed. Use RULES/rules.dir in all cases.
• New configuration files in the Schily makefilesystem for:
• - *BSD on amd64
• - MacOS X on amd64
• - kfreebsd (Debian on a FreeBSD kernel)
• - Android
• The Schily makefilesystem now supports CC_COM for all platforms to allow to overwrite the low level command for the C-compiler.
• The Schily makefilesystem now supports CC++_COM for all platforms to allow to overwrite the low level command for the C++-compiler.
• Mingw now by default links against -lminwg32 instead of -lmgw32
• Mac OS X now by default links against -lgcc_s.1 instead of -lgcc
• Speed up the dependency generation for cl.exe by pooling some sed commands in a single sed call
• Fixed a bug with the dependency generation for HP-UX
• The Schily makefilesystem restructuresd some dependencies to allow to run a parallel make.
• Android is not POSIX (by not defining various functions as functions in libc as required by POSIX) because it tries to define many functions that are part of the standard as inline macros in include files only. This breaks autoconf, so we needed to rewrite some tests (e.g for getpagesize, tcgetpgrp, tcsetpgrp)
• The Schily autoconf system has been enhanced to support cross compilation. Schily autoconf is based on GNU autoconf and GNU autoconf does not support cross compilation because it needs to run scripts on the target system for some of the tests.
• The "configure" script that is delivered with the Schily makefile system runs 718 tests and 68 of them need to be run on the target system.
• The Schily autoconf system now supports a method to run these 65 tests natively on a target system. You either need a machine with remote login features or you need an emulator with a method to copy files into the emulated system and to run binaries on the emulated system as e.g. the Android emulator.
• We currently deliver three scripts for "remote" execution of programs on the target system:
• runrmt_ssh runs the commands remove via ssh runrmt_rsh runs the commands remove via rsh runrmt_android runs the commands remove via the debug bridge
• If you need to remotely run programs on a system that is not supported by one of there three scripts, you need to modify one of them to match your needs.
• To enable Cross Compilation use the following environment variables:
• CONFIG_RMTCALL= Set up to point to a script that does the remote execution, e.g.:
• CONFIG_RMTCALL=`pwd`/conf/runrmt_ssh
• CONFIG_RMTHOST= Set up to point to your remote host, e.g.:
• CONFIG_RMTHOST=hostname or CONFIG_RMTHOST=user@hostname
• use a dummy if you like to use something like to the Android emulator.
• CONFIG_RMTDEBUG= Set to something non-null in order to let the remote execution script mark remote comands. This will result in configure messages like:
• checking bits in minor device number... REMOTE 8
• Note that smake includes automake features that automatically retrieve system ID information. For this reason, you need to overwrite related macros from the command line if you like to do a cross compilation.
• Related make macros:
• K_ARCH= # (sun4v) Kernel ARCH filled from uname -m / arch -k M_ARCH= # (sun4) Machine filled from arch P_ARCH= # (sparc) CPU ARCH filled from uname -p / mach OSNAME= # sunos, linux, .... OSREL= # 5.11 OSVERSION= # snv_130 CCOM= # generic compiler name (e.g. "gcc") CC_COM= # compiler to call (name + basic args)
• ARCH= overwrites M_ARCH and P_ARCH
• It is usually suffucient to set ARCH and OSNAME.
• In order to use a cross compiler environment instead of a native compiler, set the make macro CC_COM to something different than "cc".
• If you are on Linux and like to compile for Android, do the following:
• 1) set up CC acording to the instructions from the cross compiler tool chain
• 2) set environment variables CONFIG_RMTCALL / CONFIG_RMTHOST, e.g.: setenv CONFIG_RMTCALL `pwd`/conf/runrmt_android setenv CONFIG_RMTHOST NONE
• 3) call smake:
• smake ARCH=armv5 OSNAME=linux CCOM=gcc "CC_COM=$CC"
• Several programs no longer test for HAVE_DEV_* but for HAVE__DEV_* as we did switch from hand written tests for /dev/tty, /dev/null and similar to AC_CHECK_FILES(/dev/tty /dev/null /dev/zero)
• Support for Pyro (a Syllable clone) was added. Thanks to Flemming H. Sorensen
• Better support for Win32 sockets when using cl.exe
• Avoid problems with the non-standard prototypes for link() unlink() read() and write() from cl.exe
• Add support for semi-static (preconfigured) include files for Android: include/schily/armv5tejl-linux-gcc/xconfig.h
• New include files include/schily/err_*.h are used to encapsulate #error CPP statements that would otherwise always prevent compilation with K&R compilers.
• Added a workaround for a llvm compiler bug that creates code with endless loops in some cases.
• Libschily is now linked against $(LIB_INTL) when a shared libschily is created.
• Avoid warnings from cl.exe for libschily/gettimeofday.c
• The make program specific configuration files for the Schily Makefilesystem RULES/mk-.id (for SunPro make), RULES/mk-smake.id RULES/mk-gmake.id now include definitions for the macros:
• SPACE A single space character (' ') NUMBER_SIGN The ASCII number sing ('#')
• Note that the POSIX standard otherwise requires the make program to be unable to have the '#' appear in non-comment sections of Makefiles.
• Try to work around an imprudent change in the file windef.h from Cygwin-1.7.17 (Autumn 2012) related to the type BOOL that makes windefs.h from the recent Cygwin no longer compatible to the original file from Microsoft. The fix is in include/schily/windows.h
• Thanks to Alain Hoang, Thomas Plank and others for reporting and help with getting a fix for the problem named above
• Better comment in libschily/comerr.c to help to understand exit() code folding to 8 bits and how it may be avoided.