Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Unreleased

Added

libsquashfs: Add a helper function to initialize objects
libsquashfs: Add sqfs_xattr_t and helpers, for handling combined, decoded key-value pairs
libsquashfs: sqfs_xattr_reader_t and sqfs_xattr_writer_t functions for handling sqfs_xattr_t directly
libsquashfs: sqfs_istream_t and sqfs_ostream_t stream I/O interfaces, cleaned up and moved from internal helper libraries into libsquashfs, along with native implementations used by tools.
libsquashfs: Add a get_filename method to sqfs_file_t
libsquashfs: Add native file handle type and I/O wrappers, used internally by native file implementations.
libsquashfs: Add sqfs_dir_iterator_t interface, Windows & Unix native implementations, stacked recursive implementation for tools
libsquashfs: Add a data reader based sqfs_istream_t implementation
Tools: collect and print statistics about the kind of files we are packing
tar2sqfs: Add option to exclude files

Fixed

Fix broken C++ guard in rbtree.h
documentation: Compressor ID enumerator
rdsquashfs: improve unpacking error message on Windows
gensquashfs: sort by file breaking up the directory list
Win32: Fix fstree defaults in Windows version of fstree_from_dir
Win32: Fix fstree CLI mtime range check

Changed

libsquashfs: add a threshold for extended directory inodes with index
libsquashfs: Make sqfs_object_t to reference counted
Internal cleanups and restructuring

Removed

Build system: Remove without-tools feature switch
libsquashfs: remove the default block writer alignment feature

1.2.0 - 2022-12-03

Added

Make it possible for gensquashfs to add xattrs from a description file
Make it possible for gensquashfs to specify file sorting order
Support . and .. directory browsing support in sqfs_dir_reader_t
Add sqfs_tree_node_get_path to libsquashfs

Fixed

Only use actually available CPUs in gensquashfs and tar2sqfs
Overzealous bounds check in the libsquashfsblock processor (#110)
In sqfs_compressor_create, clear the output pointer on error (#110)
Typo in gensquashfs and tar2sqfs block count statistics (#108)
Hard link resolution: report error if path cannonicalization fails
Documentation: signedness of the directory header inode number
Seek position in libsquashfs meta data reader erroneously out-of-bounds
Memory leak in libfstree error path
Double-free in libfstree error path

Changed

Allow NULL pointer for sqfs_destroy, sqfs_dir_tree_destroy
Use 32 bit hard link counter in libfstree instead of just 16 bit
Internal cleanup and restructuring
Upgrade autotools version
Check in autogen.sh if the required tools are available
Trigger a build failure if no compressor library is available

1.1.4 - 2022-03-30

Added

libsquashfs: A flag for sqfs_open_file to not perform any local charset based transformations, e.g. UTF-8 to UTF-16 on Windows (#96).

Fixed

libsquashfs: uninitializeed state in the directory reader caused by rewind call after reading an emtpy directory.
libsquashfs: stricter bounds checking in directory reader.
libsquashfs: Guard against multiplication overflow in the data reader on 32 bit systems.
structure alignment test on 32 bit x86 systems (#93)
Windows: error handling on FlushFileBuffers (#97)
Windows: loop exit condition in mkdir_p (#97)
Windows: error handling when reading from a pipe (#102)
Windows: heap corruption in stdout stream wrapper (#100)
Use the correct printf specifiers for 64 bit types on 32 bit systems.
Miscellaneous static analysis and compiler warnings.

Changed

Windows: add a wrapper for main() that reads the UTF-16 command line and converts it to UTF-8 for propper Unicode support (#96).
Windows: when opening a file, treat the path as UTF-8, convert it to UTF-16 and use the WCHAR API for propper Unicode support (#96).
Windows: redirect stdio print functions, convert from UTF-8 to UTF-16 if the target is the console for propper Unicode support (#96).
libsquashfs: Windows: when opening files, interpret the API argument path as UTF-8, convert it to UTF-16 and use the WCHAR API, unless explictly told not to (#96).
Upgraded version of the builtin copy of zlib.

1.1.3 - 2021-08-15

Added

Overhaul format documentation, convert to ASCIIdoc (#88, #90)
Add an explicit license boiler plate to all the example programs (#89)
Additional example code & documentation

Fixed

Typos and formating issues in the documentation (#86)
Symlink based path traversal in rdsqaushfs when unpacking to a directory
libsquashfs directory writer size accounting (#85)
tar2sqfs, sqfs2tar: 4 GiB limitation for files in tarballs (#87)

1.1.2 - 2021-06-25

Added

Test cases for concatenated stream decompression
A more "real-world" test suite for tar2sqfs pre-release testing

Fixed

Replace tabs with spaces in format.txt
Some documentation clarifications and typo fixes
libsquashfs: preserve alignment flag in block processor
libsquashfs: broken block alignment in block write
allow concatenated Bzip2 streams
Use Automake conditional for zstd stream compression support
Use *_MAX from limits.h instead of configure-time type size checks
Additional compiler warnings were turned on and addressed
libfstream: Add printf format specifier attribute
libfstream: guard against potential integer overflows
libfstree: guard against link count and inode number overflow
libfstree: guard against possible overflow in readlink()
libcommon: potentially un-aligned data access in LZO compressor
libsquashfs: potentially unaligned data access in meta data handling
Some format string type/signedness mismatch issues

1.1.1 - 2021-05-07

Fixed

tar2sqfs: currectly process concatenated xz streams from parallel compression
sqfs2tar: don't report an error if fsync() returns EINVAL
libsquashfs: static linking on Windows
libsquashfs: visibillity of internal mempool functions
libsquashfs: add sqfs_free() function, mainly for Windows portabillity
libsquashfs: block processor: Fix account for manually submitted blocks
Fix build failure if configured with --without-tools

1.1.0 - 2021-03-28

Added

tar2sqfs: support transparently reading stream compressed archives
sqfs2tar: support creating stream compressed archives
gensquashfs: support file globbing/filtering in the description file
Support bzip2 compression for tar
Implement directory scanning for Windows
a glob keyword to gensquashfs for find like globbing from a directory.

Changed

Rewrite file I/O in the tools around an I/O stream wrapper which is used to implement the transparent compression.
Internal cleanups & restructuring, trying to improve maintainabillity and testabillity of the code.
Updated benchmarks, including benchmarks for decompression.
Drastic performance improvements of the xattr writer (#68).
Performance improvements in the thread pool block processor.
Bump zstd & xz versions for Windows binary releases.

Fixed

libsquashfs: Allow shared read access to generated images on Windows (#79)

1.0.6 - 2021-08-15

Added

Overhaul format documentation, convert to ASCIIdoc (#88, #90)
Add an explicit license boiler plate to all the example programs (#89)
Additional example code & documentation

Fixed

Typos and formating issues in the documentation (#86)
Fix symlink path traversal in rdsqaushfs
Add a test case for the path traversal bug
Fix libsquashfs directory writer size accounting
tar2sqfs, sqfs2tar: 4 GiB limitation for files in tarballs (#87)

1.0.5 - 2021-06-25

Fixed

libsquashfs: Allow shared read access to generated images on Windows (#79)
libsquashfs: preserve alignment flag in block processor
libsquashfs: broken block alignment in block write
libsquashfs: add sqfs_free() function, mainly for Windows portabillity
libsquashfs: static linking on Windows
Some documentation clarifications and typo fixes
libfstree: guard against link count and inode number overflow
libcommon: potentially un-aligned data access in LZO compressor
libsquashfs: potentially unaligned data access in meta data handling
More compiler warning flags turned on and partially fixed

1.0.4 - 2021-01-23

Fixed

typos in documentation
libsquashfs: xattr_writer: fail if allocation fails
remove broken normalization of backslashes (#77, #78)
congestion in block processor when processing lots of tiny fragments
exact byte-for-byte matching during block deduplication
exact byte-for-byte matching during fragment deduplication

Added

new, extensible block processor create function required for deduplication fixes

1.0.3 - 2020-11-05

Fixed

tar2sqfs: if --root-becomes is used, also retarget links (#70)
tar2sqfs: when skipping non-prefixed files, also skip contents (#70)
tar2sqfs: null-pointer dereference if a file with a broken path is skipped
rdsquashfs: correctly escape of the input file paths if prefixed

Added

Packing scripts for a various GNU/Linux distributions (#65, #67)

1.0.2 - 2020-09-03

Fixed

Fix block processor single block with don't fragment flag bug (#63)
Fix libtar treatment of link targets that fill the header field (#64)
Fix tree node path generation for detached sub trees (#66)
Fix rdsquashfs unpack under Windows if a directory exists
Clarify copyright status of documentation & build system

Added

Support for GNU tar sparse format 1.0.
A spec file to build RPM packages (#65).
A --stat option to rdsquashfs

1.0.1 - 2020-08-03

Fixed

Xattr reader: re-read the header after seaking to an OOL value (#60)
Documentation: mention the file name limit imposed by the kernel
Documentation: fix wrong magic value and stray tabs in format.txt (#61)
Fix block bounds checking in libsquashfs data reader (#58)
Fix build issue caused by demo code that didn't have (#57, #59)

1.0.0 - 2020-06-13

Added

Expose more fine grained control values & flags on the XZ and LZMA compressors
A flag for the libsquashfs block processor to micro manage block hashing and sparse block detection.
A raw block submission function.
A user pointer that can be forwarded to the block writer.

Changed

sqfsdiff doesn't abort if it fails to read the compressor options
in libsquashfs, turn the block writer into an interface with a default implementation and remove the statistics and hooks.
Block processor can function without fragment table and without inode pointer
Strictly enfore min/max dictionary size in XZ & LZMA compressors
Make compression level a generic compression option in libsquashfs
More libsquashfs API and internal cleanups
Bump the major version number of libsquashfs

Fixed

Propperly set the last block flag if fragments are disabled
Compilation on GCC4 and below
libtar: size computation of PAX line length (#50)
Semantics of the super block deduplication
Actually set the ZSTD compression level to something greater than 0
Only add Selinux compile flags if WITH_SELINUX is set. Fixes Mingw cross build on Fedora.
Make rdsquashfs describe mode terminate with an error message if an illegal filename is encountered (#52).
Don't include alloca.h on systems that don't provide that header.

0.9.1 - 2020-05-03

Added

Options to gensquashfs for overriding the ownership if packing a directory.

Changed

Various internal code and build system related cleanups.
Various performance improvements for gensquashfs and tar2sqfs.
Make tar2sqfs compeltely ignore PAX global headers, even if --no-skip is set (#45).
Make gensquashfs fail if extra arguments are passed, similar to tar2sqfs.

Fixed

Missing assert header inclusion if built with --without-lzo (#41)
sqfs2tar: also emit trailing slashes for empty directories (#42)
Illegal implicit cast in public libsquashfs headers if used from C++.
If tar2sqfs or gensquashfs fails, delete the output file.
Make sure test cases also work if built with NDEBUG defined.
Add missing --with-gzip configure option.
In tar2sqfs, actually apply the --no-tail-packing if set.
Potential nondeterministic order of extended attributes in gensquashfs if packing a directory.
Manpages lagging behind.

0.9.0 - 2020-03-30

Added

Support parsing [device] block size argument with SI suffix.
Add a write-up on the on-disk format.
A couple demo programs that make use of libsquashfs.
Add statistics counters to the block writer and processor.
A compressor interface function to retrieve its configuration.

Changed

For better compatibility, sqfs2tar appends / to directory names. (#37)
Extra data in sqfs_inode_generic_t no longer handled via payload pointers.
Add a currently unsued flag field to the id table create function.
Split off fragment table handling from data reader and writer into dedicated fragment table data type.
Split data writer up into a block writer and a block processor.
All abstract data types inhert from an sqfs_object_t with common functionality.
Lots of performance improvements.
Make block processor sync and flush seperate operations.
Combined the various payload size values in generic inode structure.
Change block processor API to manage creation/resizing of file inodes.
A "do not deduplicate flag" for the block processing pipeline.
Lots of API cleanups.

Fixed

Include sys/sysmacros.h on any GNU libc platform.
Directory index accounting.
Memory leak in hard link detection code.
Broken iteration over directory children in sqfsdiff.
Data reader returning -1 instead of an error code.
Size accounting for sparse files in tar parsing code.
Stricter verification of the compressor configuration.
Broken builds with older liblz4 and zstd versions (e.g. on Ubuntu Xenial).
Various build issues on MacOS.

Removed

A number of hook callbacks from the block writer.
sqfs_block_t from the public API.

0.8.0 - 2019-12-30

Added

Experimental Windows port using MinGW cross compilation toolchain.
Port to BSD systems.
Explicit argument invalid error code in libsquashfs.
A --root-becomes option to tar2sqfs and sqfs2tar.
A --no-tail-packing option to tar2sqfs and sqfs2tar.
CHANGELOG.md now references GitHub issue numbers.
Simple integration and regression test suit.
Simple creation of an NFS export table throught libsquashfs.
Support for non-directory hard links in gensquashfs, tar2sqfs and sqfs2tar.

Changed

Return propper error code from sqfs_get_xattr_prefix_id.
Return propper error code from sqfs_compressor_id_from_name.
Combined error and value return from sqfs_compressor_id_from_name.
Return error code from compressor functions if input is larger than 2G.
Lots of cleanups, including the build system.

Removed

Entirely redundant sqfs_has_xattr function from libsquashfs.

Fixed

LZO compressor moved out of libsquashfs to avoid licensing problems.
Make overriding configure variables for LZO library actually work.
Do not follow symlinks when reading xattrs from input files.
Ignore directory entry named ./ in tar2sqfs. (#22, #31)
Reject empty string as directory name in libsquashfs. (#22, #31)
Fix memory leak in tar2sqfs if entries are skipped.
Fix tar_fuzz error check after seek.
Fix the fstree_init test to account for defaults from SOURCE_DATE_EPOCH.
Honor the no_xattr flag when generating SquashFS images.
Block size check in sqfs_super_init. (#29)
Fix pthread block processor interfering with application signal handling.
Added pthread flags to the programs using libsquashfs.
Fix "buffer too small" being treated as fatal error by the zstd compressor.
Fix out of bounds write in the LZO compressor.
Fix queue accounting in the compressor thread pool. (#29)
Fix name of libsquashfs pkg-config file.
Reading of binary (i.e. non-textual) xattr values from tar files. (#32)
A bug in parsing the GNU.sparse.name PAX attribute from tar files.
sqfsdiff: recurse into directories that are only in one image.

0.7.0 - 2019-10-08

Added

LGPLv3 licensed, shared library libsquashfs.so containing all the SquashFS related logic.
Sanitized, public headers and pkg-config file for libsquashfs.
Doxygen reference manual for libsquashfs.
Legacy LZMA compression support. (#17)
User configurable queue backlog for tar2sqfs and gensquashfs.

Changed

Make sqfsdiff continue comparing even if the types are different, but compatible (e.g. extended directory vs basic directory).
Try to determine the number of available CPU cores and use the maximum by default.
Start numbering inodes at 1, instead of 2.
Only store permission bits in inodes, the reader reconstructs them from the inode type.
Make "--keep-time" the default for tar2sqfs and use flag to disable it.