aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2021-07-21Fix libsquashfs directory writer size accountingDavid Oberhollenzer
The squashfs readdir() implementation in the Linux kernel returns non-existing "." and ".." entries for offsets 0 and 1, and after that reads from disk. For convenience, it was decided to store an off-by-3 value on disk instead of doing complex primary school math to adjust for this. This didn't show up until now, because the kernel implementation trusts the value from the directory header more than the actual size in the inode and happily reads 3 more than the inode would allow it to. This only showed up with 7-zip which subtracts 3 from the size and expects the result to be exact and bails if the directory headers suggest otherwise. And yes, I did consider making a "Holy Hand Granade of Antioch" reference, but consciously decided not to. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-07-21Add a separate architecture/structure writeupDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-07-09Fix printf format specifiers used for generating tarballsDavid Oberhollenzer
When processing files > 4G, using "%o" truncates the result and the tarball is not readable. This should have been discovered when auto-patching the printf format specifiers, but a cast was added instead and the issue was overlooked. This commit replaces the down-cast and printf format specifiers. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25Release version 1.0.5v1.0.5David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25Fix format string type/signedness mismatch issuesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25Add default cases for every switch blockDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25Remove casual un-const casting in various placesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25libsquashfs: get rid of potentially unaligned access and VLAsDavid Oberhollenzer
The same problem with the meta data header again, 16 bit read from a buffer: copy the buffer data into a 16 bit variable instead of casting to something potentially unaligned. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25libcommon: remove potentially un-aligned access in LZO compressorDavid Oberhollenzer
When accessing the 16 bit header, don't cast the buffer pointer to an uint16_t pointer, the result might not be aligned propperly. Instead memcpy to and from an uint16_t. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25libfstree: guard against link count and inode number overflowDavid Oberhollenzer
If the hard link counter or the inode number counter overflow the maximum representable value (for SquashFS 16 bit and 32 bit respecitively), abort with an error message. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25Switch travis-ci.org links in README.md to travis-ci.comDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25Add a FreeBSD target to the travis-ci config fileDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25Backport release scripts and script changesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25Backport documentation clarifications and typo fixesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-07Fix: libsquashfs: allow static linking on WindowsDavid Oberhollenzer
If SQFS_STATIC is defined, dummy out the SQFS_API definition, so we don't try to pull stuff from a (in this case) non-existant DLL or try to export functions. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-07Fix: libsquashfs: add sqfs_free() functionDavid Oberhollenzer
On systems like Windows, the dynamic library and applications can easily end up being linked against different runtime libraries, so applications cannot be expected to be able to free() any malloc'd pointer that the library returns. This commit adds an sqfs_free function so the application can pass pointers back to the library to call the correct free() implementation. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-07libsquashfs: fix: also preserve alignment flag in block processorDavid Oberhollenzer
Currently, when the block processor aggreagtes fragments into a fragment block, it applies the "don't compress" flag if any of the original framgnets has it set, but the "align to device block" flag is lost. This commit ensures that both flags get applied to the fragment block if set. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-07libsquashfs: fix block alignment if requestedDavid Oberhollenzer
1) If the block alignment flag is set, the padding bytes must be inserted _before_ recording the start position, otherwise the resulting image is not readable. 2) Also perform alignment if the flag is set on a fragment block. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-02-28added shared read access when opening sqfs image with read-only flags (win32)Thomas Lang
2021-01-22Patch level release 1.0.4v1.0.4David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-01-19libcommon: backport block processor API updateDavid Oberhollenzer
The changes from commit 5191a25b92f903bcc2142be7ea1bfbe5ea1f5096 are backported here in a separate commit since master had some restructuring of libcommon.a after version 1.0.2 and the commit cannot be rebased directly. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-01-19libsqfs: block processor: backport exact fragment matchingDavid Oberhollenzer
This commit is an amalgamation of the commits on master that implement exact matching of fragment blocks during deduplication. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-01-19libsqfs: implement exact matching in the default block writer.David Oberhollenzer
Instead of comparing (compresed, disk-size, checksum) tuples to find block matches, do an exact, byte-for-byte comparison of the data stored on disk to avoid the possibility of a spurious colision. Since this is the desired behaviour, make it the default, optionally overrideable through a flag. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-01-19Fix: Move fragment consolidation back to block processor serial partDavid Oberhollenzer
Keeping a list of fragments stored away in the current fragment block and consolidating them in the thread pool takes them out of circulation. If we have a lot of tiny fragments, this can lead to a situation where all the limit is reached, but we cannot do anything, because we are waiting for a block to complete, but they are all attached to the current fragment block and the queue is empty. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-01-15Fix more normalization of slashes in filenames.Scott Moser
It looks like the last commit missed a couple more occurences where '\' was treated incorrectly. Fixes were still needed in sqfs_dir_reader_find_by_path and sqfs_dir_reader_get_full_hierarchy. This path is used in extras/browse.c.
2020-12-29Fix normalization of slashes in filenamesDavid Oberhollenzer
All paths were canonicalized internally, which includes filtering sequences of slashes and converting backslashes to slashes. Furthermore, when unpacking files, filenames are sanity checked and rejected if they contain forward OR backward slashes. This is a problem on Unix-like systems, where files containing backslashes are a legitimate use case (*cough* SystemD *cough*). This patch removes the backslash conversion from the canonicalization and modifies the sanity check to reject backslashes only on Windows. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-12-29Fix: libsquashfs: xattr_writer: return NULL if calloc failsDavid Oberhollenzer
Instead of dereferencing the NULL pointer and crashing. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-12-29Minor "late night typing" fixes in documentationDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-11-05Patch level release 1.0.3v1.0.3David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-11-02Backport changes to builtin copy of zlibDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-11-02Backport changes to package scriptsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-11-02Backport changes to the test setupDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-11-02Backport changes to the benchmark writeupDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-10-31Add a test case for tar2sqfs root becomes file & link filteringDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-10-31Fix: tar2sqfs: if --root-becomes is used, also retarget linksDavid Oberhollenzer
In addition to skipping non-prefixed files and stripping the prefix off of entries we accept, the targets of links also have to be altered, since they can be absolute paths with the root prefix attached. This can affect symbolic links as well. Altough they are allowed to point into nowhere, across filesystem boundaries, they may also be absolute paths refering to existing locations in the filesystem, so no distinction is made by default. However, the later may be intended (e.g. only a subdirectory is packed into SquashFS and then mounted at that location), so a command line switch is added to disable symlink retargetting. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-29Fix: rdsquashfs: describe: don't escape the prefixed file input pathDavid Oberhollenzer
If rdsquashfs describes a filesystem, is configured to use a prefix-path and a file contains a space, the resulting input path has the prefix printed as is and the rest of the string in quotation marks. gensquashfs simply takes the entire rest of the line as is as its input path and no longer finds the file. Fix this by omitting the escapes and quotation marks for the input path. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-24Fix: tar2sqfs: when skipping non-prefixed files, also skip contentsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-24Fix: tar2sqfs: don't touch the path after determining it is brokenDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-03Patch level release 1.0.2v1.0.2David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-03Fix integer bounds checking in GNU tar sparse format 1.0 parserDavid Oberhollenzer
- Make sure the file actually has that many records before trying to read one and fail if not. - Use the helper macros for size_t overflow checking instead of assuming size_t == uint64_t. - Impose a "reasonable" upper bound on the number of data segments and insist that there is at least one entry. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-03Remove tar test cases for file flags (not applicable)David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-03Cleanup: reduce tar test cases to a few generic C filesDavid Oberhollenzer
This commit removes the existing tar test cases that simply call the generic test case function with several different paths with generic test case source files that are parameneterized via the pro-processor. For each tar archive, a separate test case is generated. On the one hand, this reduces the test source code to practically nothing. On the other hand, a test binary is generated for every distinct test case, instead of one per group and we get more detailed insights if something goes wrong. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-03Cleanup copy & paste mess in the tar parser test casesDavid Oberhollenzer
Many of the patterns tested are very repetetive. This commit moves the two common test cases out into helper functions and uses them for the test cases. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-02Fix nonexistant gnu tar sparse format 1.0 supportDavid Oberhollenzer
Contrary to previous claims, support for the GNU tar sparse format 1.0 was missing entirely (the newest of their 3 different sparse mapping formats). This oversight wasn't caught, because the unit test was compiling the wrong source file and tar2sqfs had no problem processing the test file because it is still a valid POSIX-ish tar archive (but the sparse part was missing and the mapping embedded in the file). Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-08-29Add package scripts to the release tarballDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-08-26Fix rdsquashfs unpack under Windows if a directory existsDavid Oberhollenzer
Behave the same way as the POSIX port and do not treat that as an error. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-08-26Fix tree node path generation for detached sub treesDavid Oberhollenzer
The function sqfs_tree_node_get_path is used in several places within rdsquashfs to produce a path for a tree node, either when describing the file system, or when unpacking it. Unpacking can be done on sub-trees as well as the entire tree, in which case the root of the sub-tree has its parent pointer removed, so the full path terminates at the new root. This works with directories, since they receive special case handling anyway, but fails if the sub-tree to unpack is only a single file because the sqfs_tree_node_get_path function assumes that we are at the tree root and returns "/" as a path, which gets normalized to "". This commit adds a workaround to the function to simply use the nodes name (if available) in that case instead. The describe case in rdsquashfs is unaffacted, since it always starts at the root. Likewise, the sqfs2tar case should also be unaffacted, since it already employs special case handling for the [sub] tree root node. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-08-20Add spec file to build RPM packages.Sébastien Gross
Signed-off-by: Sébastien Gross <seb•ɑƬ•chezwam•ɖɵʈ•org>
2020-08-16Add a libtar test case for a completely filled link target fieldDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-08-16Fix libtar treatment of link targets that fill the header fieldDavid Oberhollenzer
The tar header has a 100 byte field for symlink and hard link targets. If the target is longer than 100 bytes, an extension header has to be used. However, it is perfectly valid to fill all 100 bytes to the brim without adding a null terminator. In case of a symlink, this can result in garbage link targets, while for hard links it results in an immediate error since the target cannot be resolved later on. This commit attempts to fix the problem by replacing the strdup of the link target with an strndup that copies at most the size of the target header field. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>