aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-02-09Add utility function to fixup Windows file pathsDavid Oberhollenzer
The idea is to iterate over a (canonicalized) path with forward slashes by components, i.e. file and directory names. Each name is then looked at by iterating over components, i.e. everything between dots. If a component is an illegal name, like COM1 or AUX, we add an underscore. If it contains illegal characters, like : or \, we re-map that character into unicode private use area. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2024-02-09Remove check for OS specific stuff from is_filename_saneDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2024-01-29tar2sqfs: Backport exclude filtering logicDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2024-01-24tar2sqfs: Add option to exclude filesRichard Weinberger
Using --exclude or -E it is now possible to exclude files from the input tar stream. The options can be used multiple times. Signed-off-by: Richard Weinberger <richard@nod.at>
2024-01-24README.md: fixed trailing ] for url and one typoMarcel Petrick
2024-01-24libsqfs: add a threshold for extended directory inodes with indexDavid Oberhollenzer
mksquashfs generates extended inodes if a directory contains 256 entries. libsquashfs so far only generated extended inodes if there is no other way to encode it. Mimic the behaviour of mksquashfs by adding a threshold. For this to work, the "sqfs_inode_set_xattr_index" function has to be changed to not immediately try to demote inodes to basic types. The fstree serialization is modified to do that itself if the index is 0xFFFFFFFF and the target is not a directory inode. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-20Fix compressor ID enumerator in format documentationfixes-1.2.0David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-12-14rdsquashfs: improve unpacking error message on WindowsDavid Oberhollenzer
Print a proper error description instead of just an error code. If the error is ERROR_FILE_EXISTS, print out a hint that this might be caused by a case in-sensitivity issue. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-12-14Fix unix istream eof flagDavid Oberhollenzer
Propperly set the parent eof flag and not a local one that isn't accessed at all. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-12-03Update versioning information in README.mdDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-12-02Release version 1.2.0v1.2.0David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-12-02Re-run spell checker on the CHANGELOG.mdDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-12-01Update CHANGELOG.mdDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-29Add dummy hierarchy for fstree_from_dir testDavid Oberhollenzer
Instead of abusing the directory tree of another test case, add a proper dummy hierarchy. This also fixes issues with parallel tests, where another test generates squashfs images in the target path. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-27Continue cleanup of the test casesDavid Oberhollenzer
- Force all tests into their proper sub directory - Temporarily remove the corpora tests. They have been used for regression tests before releases and are disabled by default, so we should not ship them either. A script should be added for that, downloading what is needed. - The "pack a directory" test is also removed. It was rather hacky and there already is a test case for the fstree_from_dir function, which isn't ideal either. Something should be added to the regression test suite. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-27Improve thread pool testDavid Oberhollenzer
Instead of sleeping in the worker thread, busy-loop-wait on a ticket counter to try and serialize the workers in backward order. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-22Add a hacky script to sanity check symbols exported by libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-22Move gensquashfs specific code from libfstree to gensquashfsDavid Oberhollenzer
The "from dir" and from "from file" code, as well as the "sort file" code is specific to gensquashfs, so move them there and the test cases as well. The medium term idea is to reduce libfstree to a stub, merge it into the generic writer and ultimately hoist that into libsquashfs. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-22Move helper scripts to scripts/ sub directoryDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-22Get rid of the built-in copy of LZ4David Oberhollenzer
On Linux or BSD distributions we have a native version installed via package manager. On Windows, we can just build it from source like the other libraries. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-21Fix windows build for get_node_path testDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-21Make some string functions from libcompat available to libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-21filemap xattr: slightly improve error reportingDavid Oberhollenzer
- When printing an error, specify the filename and line number - Always print an error, including out of memory conditions - Fail if we encounter a line that we don't recognize Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-21filemap xattr: use sqfs_u8 data type for xattr value blobsDavid Oberhollenzer
The xattr values are not strings, they are arbitrary byte blobs. To be on the safe side tough, we should still allocate the space for the extra null byte and propperly initialize the buffer. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-19gensquashfs: Add example for the xattr filemap to the help textDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-19gensquashfs: split the help text up into smaller partsDavid Oberhollenzer
It's just too darn big, crossing the maximum guaranteed by C99. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-19gensquashfs: simplfy the help text a bitDavid Oberhollenzer
The help text should provide a short, concise explanation, for quickly lookup up how to use the programs. For the input formats, this is mostly covered by the examples themselves. The detailed description can remain in the man page. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-19Add a man-page section about the xattr file formatDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-18filemap xattr: use file istream from libio instead of stdio FILEDavid Oberhollenzer
The line-by-line reading function in the io library was specifically added for this kind of use case, so we don't have to handle trimming of lines, don't have to touch getline() and it's convoluted error handling (return value -1 could mean either EOF or error). The code that searches for '\n' and replaces it with '\0' can be removed as well and a memory leak involving "line" is removed along the way. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-18filemap xattr: simplify error handling, freeingDavid Oberhollenzer
- Some of the error/cleanup paths are merged. - Struct fields don't have to be set to NULL if the entire struct is free'd immediately after. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-18libsqfs: Fix an overzealous bounds check in the block processorDavid Oberhollenzer
When (during fragment deduplication) a fragment block is read back from disk and unpacked, it can happen that it is _exactly_ the given block size. The bounds check did '>=' instead of '>' and failed in that case with a "data corruption" error. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-18libsqfs: Initialize the return value in sqfs_compressor_createDavid Oberhollenzer
Initialize the output compressor pointer to NULL, so if the function fails, the value is propperly initialized to a NULL pointer instead of relying on the function user to initialize it. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-18Add a single, central base64 decoderDavid Oberhollenzer
Similar to the hex blob decoder, we need this once for tar and once for the filemap xattr parser. Simply add a single, central implementation to libutil, with a simple unit test, and then use it in both libtar and gensquashfs. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-18Add a single, central hex blob decoderDavid Oberhollenzer
Since we need it twice (once for tar, once for the filemap xattr parser), add a single, central implementation to libutil, add a unit test for that implementation and then use it in both libtar and gensquashfs. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-16filemap xattr: Add a test caseDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-16filemap xattr: move data structures to mkfs.hDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-07implement parser for xattr files as produced by getfattr --dump XXXEnno Boland
2022-11-04Only use available CPUsWessel Dankers
Not all CPUs may be available for the current process. Some CPUs may be offline, others may not be included in the process affinity mask. In such cases too many threads will be created, which will then compete unnecessarily for CPU time. Use sched_getaffinity() to determine the correct number of threads to create.
2022-11-04Fix typo in block count statisticsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-04Fix: update mempool accounting when freeing an objectDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-10-10block writer: further cleanup of the block writer logicDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-09-20block writer: move block comaprison to utility functionDavid Oberhollenzer
Slightly modify the byte-for-byte comparison function to compare an arbitrary range in a file and move it to libutil. Instead of calling it for each block in the block writer, simply let it check an entire range in the block writer and compute the range position/size of the reference ahead, before looking for potential matches. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-09-20block writer: remove open coded arrayDavid Oberhollenzer
Instead of open coding it, use the array_t type from libutil. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-08-31Hardlink search: report error if cannonicalization failsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-08-31Fix: documentation: directory header inode number signednessDavid Oberhollenzer
A directory listing starts with a header that specifies a reference inode number. Each entry then specifies a difference from that reference value. While the difference is _signed_, the reference value is _unsigned_. This is handled correctly in the code, but was pointent out wrongly in the documentation. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-08-23Cleanup gzip range clamping codeDavid Oberhollenzer
The zlib library uses a typedef'd uInt type for ranges which may be smaller than size_t. The complicated clamping to the uInt range is technically not needed, as the buffer size can grow at most to BUFSZ anyway, and it also confuses coverity. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-08-23Check sqfs_tree_node_get_path return value in rdsquashfs error pathDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-08-23Bump coverity versionDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-08-20Return an error number from sqfs_tree_find_hard_linksDavid Oberhollenzer
Instead of printing error messages to stderr, simply return an error number instead, that the caller then prints out using sqfs_perror. The underlying rbtree already uses sqfs error numbers, so little change is needed here. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-08-20Overhaul tar compression test script, add to corpora test setDavid Oberhollenzer
- Make the script faster by unpacking everything at once instead of file-by-file - Also compare symlinks and directories - More verbose output Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>