aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2022-03-30Add a rudimentary unit test for sort filesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-30Add a sort-file option to gensquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-30Add sort-file implementationDavid Oberhollenzer
A `flags` field and `priority` are added to all file information structs. A news fstree function is introduced for parsing a "sort-file". Each line in the file is space separated, and has the following format: priority [flags] filename Priority is a 64 bit number, flags are optional and filename can be put in quotes if it is supposed to start or end with spaces. Single line comments can be used. The flags can be used to set block-processor flags (e.g. don't fragment, or don't compress), as well as instructing the parser to use file globbing to match the filename. After parsing the file, the list of file info structure is sorted according to the priority (default is 0) using a stable sort algorithm. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-30Add a unit test for the SOURCE_DATE_EPOCH logicDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-30Cleanup: remove node sorting from libfstreeDavid Oberhollenzer
Always insert the tree nodes in the correct oder and remove the post-process sorting step. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-30Cleanup: table driven pax header parsingDavid Oberhollenzer
Instead of having a long if-else-if chain, replace the PAX header field parsing with a table driven approach. Altough it is more code, it is hopefully more readable, maintainable, extensible and it dedupliates some of the value parsing code. The GNU.sparse parsing is left as is, because it requires maintaining state. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-30Cleanup: pax header parsingDavid Oberhollenzer
Split the key/value pairs right in the header and terminate the key name. This way, some of the magic numbers can be eliminated. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-30sqfs_dir_tree_destroy/sqfs_destroy: allow NULL inputLuca Boccassi
Many library destructor functions (like free()) allow a NULL pointer as input, and do nothing in that case. This allows easier cleanup patterns: initialize pointers to NULL and then always pass them to the destroyer functions, no need for verbose goto/if-else patterns. Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com>
2022-03-30Release version 1.1.4v1.1.4David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-30Update CHANGELOG.mdDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-30Update built-in zlib versionDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-19Fix mksrcrelease so version upgradeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-11More defensive programming in mem_pool_allocateDavid Oberhollenzer
Abort and retry in situations that should logically _never_ _ever_ happen. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-10Fix: guard against potential overflow in file size calculationDavid Oberhollenzer
The block_count is a size_t, so on 32 bit platforms the multiplication might be truncated before the comparison with filesz. On 64 bit platforms, it could potentially also overflow the 64 bit bounds of the data type. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-10Fix: use correct printf specifier for 64bit typesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-10Fix warning if __SIZEOF_INT128__ is not definedDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-10Cleanup libtar mkxattr, explicitly null-terminate stringsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-10Bump coverity version, add 32 bit targetDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-10Windows: redirect standard I/O and convert text to UTF-16David Oberhollenzer
Preprocessor magic is used to redirect putc/fputc/fputs/printf/fprintf to custom implementations. The custom implementations try to figure out if we are printing to the console and, if so, convert the resulting strings to UTF-16 and print them through ConsoleWriteW. If the output is redirected to a file or a pipe, the original (presummed) UTF-8 is kept. Simply setting the console output codepage to UTF-8 does not work, because the standard I/O facilities of MSVCRT either does not support unicode (in non-wchar mode), or has half-broken support through fputs, which can still break up multi-byte sequences through its internal buffering. Likewise, changing the codepage and using ConsoleWriteA, or trying to use fputws did not work in a test VM either. This approach is the one that worked most consistently among the ones tried, but also has problems. E.g. it breaks when setting the codepage to UTF-8 manually (using `chcp 65001`). Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-09Fix: Windows: libfstream: allocation size of stdout stream structDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-01-29Fix: libfstream: don't fail on Windows when reading from a pipeDavid Oberhollenzer
When piping the output of another program into tar2sqfs.exe, and the source program terminates, tar2sqfs.exe gets an ERROR_BROKEN_PIPE when the end is reached and it trys to pre-cache more data. This commit adds a work around, to propperly handle this as and end-of-file condition. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-12-14Fix Windows main wrapper after mingw upgradeDavid Oberhollenzer
Apparently, mingw implicitly included stdlib.h indirectly from either windows.h or shellapi.h. After an upgrade, the windows build now fails with EXIT_FAILURE being undefined. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-12-05Fix: unit test and sample program Windows buildDavid Oberhollenzer
Now that there is a wrapper for main() on Windows, all executable programs must use a common, cannonical signature for main(). Furthermore, the Windows version of the epoch test needs wrappers for setenv/unsetenv. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-12-05Fix: consistently use the widechar file API on WindowsDavid Oberhollenzer
When opening files on windows, use the widechar versions and convert from (assumed) UTF-8 to UTF-16 as needed. Since the broken, code-page-random API may acutall be intended in some use cases, leave that option in through an additional flag. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-12-05Add a wrapper for the main function on WindowsDavid Oberhollenzer
A macro and forward declaration are added to compat.h that rename the main() function programs using compat.h into sqfs_tools_main. An actual main() function is added to libcompat.a, that uses the shell API to get the UTF-16 command line arguments, convert them to UTF-8 and call sqfs_tools_main. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-11-24Fix: libcommon: Correctly restore prefix path in mkdir_p on WindowsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-11-24Fix: libfstream: Correctly handle FlushFileBuffers resturn statusDavid Oberhollenzer
The Windows port uses FlushFileBuffers in libfstream for the implmentation of the file flush method. Unlike other winapi functions, this function returns a boolean and not an error code. Previously, the error code path was executed on success, printing a rather confusing error message, that this file already exists. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-09-13Fix struct offset testing in ABI test caseDavid Oberhollenzer
The intention of the (severely incomplete) ABI test case is to detect changes to the ABI of libsquashfs. Currently it tries to blurt out if the layout of some structure is changed unintentionally. Unfortunately, the test uses some unportable assumptions. Among other things, it was assumed that a 64 bit field will always require 64 bit alignment. This is apparently no the case on 32 bit x86. This patch makes the check work on 32 bit and 64 bit x86, by adding an additional runtime check that relies on the __alignof__ extension offered by gcc and clang (the only 2 compilers that are really supported at the moment). Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-08-22Tighten bounds checks in sqfs_dir_reader_readerDavid Oberhollenzer
Use the same size check as sqfs_dir_reader_open_dir and report EOF, even if it is possible to read the header itself, but nothing beyond that. Also check if it should be possible to read an entry header before attempting and report EOF if not. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-08-22Fix half done initialization of sqfs_dir_reader_open_dirDavid Oberhollenzer
The sqfs_dir_reader_open_dir function tried to take a short-cut by returning early if the target directory is empty. However, this left some field unchanged from the previous directory. If iterating over a directory and then deciding to enter a sub-directory that happens to be empty, the directory reader will keep the settings for the current directory. After calling sqfs_dir_reader_rewind, the sub-directory will suddenly report the contents of the parent. A similar check is added to the rewind function to not track back on the meta data reader in that case. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-08-15Release version 1.1.3v1.1.3David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-08-15Fix explicit NULL derference for the extract-one exampleDavid Oberhollenzer
On the 1.1.x branch, the upstream "allow delete if NULL" patch is not applied, so this needs an explcit fix. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-08-15Update CHANGELOG.mdDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-08-15Fix checksums for the corpora testsDavid Oberhollenzer
Due to the change in directory size accounting, the checksums no longer matched. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-08-14Add a test case for the path traversal bugDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-08-12Fix symlink path traversal in rdsqaushfsDavid Oberhollenzer
If rdsquashfs unpacks a directory tree that contains a symlink, followed by something else with the exact same name, it will follow the symlink and can be tricked into writing to an arbitrary filesystem location controlled by the SquashFS image. Because there might actually be a reasonable use case, where an image is unpacked into an directory existing directory tree, with symlinks that should be followed, this is solved as follows: - Before unpacking, recursively sort the directory by filename. - FAIL if (after sorting) two consequtive entries at the same hierarchy level have the same name. This solution is more generic and prevents the unpacker from accessing the same thing twice in generall, thus also excluding the symlink issue. Hardlinks are already unfolded into duplicate tree nodes by the tree reader (with loop detection) so that should not prompt further issues. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-08-12Add an explicit license boiler plate to all the example programsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-08-12Add a simple example to show how to extract a single text fileLuca Boccassi
Licensed under 0BSD: https://opensource.org/licenses/0BSD Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com>
2021-08-12Replace format.txt with an asciidoc versionZachary Dremann
2021-08-12Strip space from first column of format.txtZachary Dremann
This is to allow git to recognize that format.txt "moves" to format.adoc in the next commit (with -M20, at least), which should allow easier comparison for what has changed (and more importantly, what hasn't) in converting to asciidoc. For instance, doing the diff with the following options: ``` -M20 --ignore-all-space --word-diff ```
2021-08-12Documentation: fix back reference to data block sectonDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-07-21Fix libsquashfs directory writer size accountingDavid Oberhollenzer
The squashfs readdir() implementation in the Linux kernel returns non-existing "." and ".." entries for offsets 0 and 1, and after that reads from disk. For convenience, it was decided to store an off-by-3 value on disk instead of doing complex primary school math to adjust for this. This didn't show up until now, because the kernel implementation trusts the value from the directory header more than the actual size in the inode and happily reads 3 more than the inode would allow it to. This only showed up with 7-zip which subtracts 3 from the size and expects the result to be exact and bails if the directory headers suggest otherwise. And yes, I did consider making a "Holy Hand Granade of Antioch" reference, but consciously decided not to. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-07-21Add a separate architecture/structure writeupDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-07-09ABI test: add sizes/layouts of structures likely to be expandedDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-07-09Bump package scriptsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-07-09tar2sqfs: Fix integer truncation issue when appending data to a tarballDavid Oberhollenzer
The data is processed 2G at a time to avoid this exact issue, but the check was skipped on 64 bit systems, since the function *used to* use size_t instead of sqfs_u32. This commit removes the second check. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-07-09Fix printf format specifiers used for generating tarballsDavid Oberhollenzer
When processing files > 4G, using "%o" truncates the result and the tarball is not readable. This should have been discovered when auto-patching the printf format specifiers, but a cast was added instead and the issue was overlooked. This commit replaces the down-cast and printf format specifiers. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25Release version 1.1.2v1.1.2David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25Update CHANGELOG.mdDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25Add a somewhat more extensive tar2sqfs test scriptDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>