aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2023-04-01libsquashfs: remove the default block writer alignment featureDavid Oberhollenzer
The idea of the block align feature was to allow micro-managing that some files are forcefully aligned to 1k/4k ("device block") boundaries, hoping to improve access time at the cost of data density. The feature was not exposed in the tools for a long time and eventuall added to the sort file. Measurement and experimentation showed, that it in fact worsened the read performance on a test system with an old micro SD card as the bottle neck. The feature is removed, and if needed, can be brought back simply by wrapping/sub-classing the default block writer, if need be.. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-21libio: Add an xfrm stream wrapper testDavid Oberhollenzer
This basically re-uses the libxfrm pack/unpack tests, but runs the data through a stream wrapper. When de-compressing, we have a ridiculously tiny input buffer, to force the wrapper to snort up the data in several attempts until the it can decompress something. We read the data byte-by-byte to force the wraper to internally cache the uncompressed data and spoon feed it to us. It has to be completely transparent to us that it internaly decompresses and also reads transparently across concatenated streams. When compressing, we only require that the output is smaller than the input and not equal to it. We also require the wrapper to flush the wrapped stream when it is flushed. We then test if the compressed data can be unpacked again. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-21libxfr: Fix zstd test naming errorDavid Oberhollenzer
We were previously building pack for unpack and vice versa. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-21libtar: generate entire xattr header in a single bufferDavid Oberhollenzer
Having it all in one buffer allows us the re-use the "generat GNU record" function. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-20Remove ostream_printfDavid Oberhollenzer
By cobbling together the xattr lines manually in libtar, the need (and thus the function itself) are removed. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-20Fix compressor ID enumerator in format documentationDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-15Fix broken C++ guard in rbtree.hDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-12libtar: Add a test for the tar writing codeDavid Oberhollenzer
Generate a simple tarball and compare it with a reference. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-11libio: Add a test case for istream-to-ostream spliceDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-10libio: Add a test for istream precache/read/skipDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-08libtar: remove need for skip_padding functionDavid Oberhollenzer
In the istream implementation, automatically skip the padding when we reach end-of-file. Also skip file AND padding when we destroy the object. Replace the remaining instances with a simple istream_skip instead and remove the wrapper from libtar. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-08tar2sqfs: Use tar istream spliceDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-08libtar: Add an istream_t implementationDavid Oberhollenzer
The tar_istream_t reads the data from a tar file, having been given the header, and synthesizes zero bytes for sparse regions. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-05Update source tree hierarchy details in COPYINGDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-04libtar: internalize the declaration of read_octalDavid Oberhollenzer
Use read_number in the places that remain. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-04libtar: simplfy parsing of old GNU sparse formatDavid Oberhollenzer
There was some code duplication for extracting the sparse entry fields from the start record and the subsequent extended record. This commit introduces a data structure for both and unifies the parsing code paths. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-04libtar: some minor cleanupsDavid Oberhollenzer
- Use is_memory_zero from libutil - Move checksum update function to tar writer code - Move checksum verify function to tar reader code - Only export the function to compute the checksum Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-31Reintegrate test code with library codeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-31Move library source into src sub-directoryDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-30Remove without-tools feature switchDavid Oberhollenzer
This was originally added for the Windows port, which initially didn't have tool support. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-26libio: add a test case for append_sparse fallback implementationDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-26libio: move simple wrappers back into header as inline functionsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-19libtar: simplify padd_file functionDavid Oberhollenzer
We have an "append_sparse" function in libio, with a fallback, so use that. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-19Split stream compression out of libioDavid Oberhollenzer
Move it to a separate libxfrm library, where it can be independently tested as well. The bulk of the new code is also mainly test cases for the compressors. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-19Simplify cleanup on errorDavid Oberhollenzer
The drop/destroy functions support passing in a NULL pointer. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-19Implement rudimentary reference counting for sqfs_object_tDavid Oberhollenzer
Implement grab/drop functions to increase/decrease reference count and destroy the object if the count drops to 0. Make sure that all objects that maintain internal references actually grab that reference, duplicate it in the copy function, drop it in the destroy handler. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-19Add a helper function to initialize libsquashfs objectsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-19Drop backwards compatibillity note for sqfs_freeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-19libsqfs: add a threshold for extended directory inodes with indexDavid Oberhollenzer
mksquashfs generates extended inodes if a directory contains 256 entries. libsquashfs so far only generated extended inodes if there is no other way to encode it. Mimic the behaviour of mksquashfs by adding a threshold. For this to work, the "sqfs_inode_set_xattr_index" function has to be changed to not immediately try to demote inodes to basic types. The fstree serialization is modified to do that itself if the index is 0xFFFFFFFF and the target is not a directory inode. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-12-14rdsquashfs: improve unpacking error message on WindowsDavid Oberhollenzer
Print a proper error description instead of just an error code. If the error is ERROR_FILE_EXISTS, print out a hint that this might be caused by a case in-sensitivity issue. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-12-14Fix unix istream eof flagDavid Oberhollenzer
Propperly set the parent eof flag and not a local one that isn't accessed at all. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-12-03Update versioning information in README.mdDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-12-02Release version 1.2.0v1.2.0David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-12-02Re-run spell checker on the CHANGELOG.mdDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-12-01Update CHANGELOG.mdDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-29Add dummy hierarchy for fstree_from_dir testDavid Oberhollenzer
Instead of abusing the directory tree of another test case, add a proper dummy hierarchy. This also fixes issues with parallel tests, where another test generates squashfs images in the target path. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-27Continue cleanup of the test casesDavid Oberhollenzer
- Force all tests into their proper sub directory - Temporarily remove the corpora tests. They have been used for regression tests before releases and are disabled by default, so we should not ship them either. A script should be added for that, downloading what is needed. - The "pack a directory" test is also removed. It was rather hacky and there already is a test case for the fstree_from_dir function, which isn't ideal either. Something should be added to the regression test suite. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-27Improve thread pool testDavid Oberhollenzer
Instead of sleeping in the worker thread, busy-loop-wait on a ticket counter to try and serialize the workers in backward order. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-22Add a hacky script to sanity check symbols exported by libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-22Move gensquashfs specific code from libfstree to gensquashfsDavid Oberhollenzer
The "from dir" and from "from file" code, as well as the "sort file" code is specific to gensquashfs, so move them there and the test cases as well. The medium term idea is to reduce libfstree to a stub, merge it into the generic writer and ultimately hoist that into libsquashfs. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-22Move helper scripts to scripts/ sub directoryDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-22Get rid of the built-in copy of LZ4David Oberhollenzer
On Linux or BSD distributions we have a native version installed via package manager. On Windows, we can just build it from source like the other libraries. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-21Fix windows build for get_node_path testDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-21Make some string functions from libcompat available to libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-21filemap xattr: slightly improve error reportingDavid Oberhollenzer
- When printing an error, specify the filename and line number - Always print an error, including out of memory conditions - Fail if we encounter a line that we don't recognize Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-21filemap xattr: use sqfs_u8 data type for xattr value blobsDavid Oberhollenzer
The xattr values are not strings, they are arbitrary byte blobs. To be on the safe side tough, we should still allocate the space for the extra null byte and propperly initialize the buffer. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-19gensquashfs: Add example for the xattr filemap to the help textDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-19gensquashfs: split the help text up into smaller partsDavid Oberhollenzer
It's just too darn big, crossing the maximum guaranteed by C99. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-19gensquashfs: simplfy the help text a bitDavid Oberhollenzer
The help text should provide a short, concise explanation, for quickly lookup up how to use the programs. For the input formats, this is mostly covered by the examples themselves. The detailed description can remain in the man page. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-19Add a man-page section about the xattr file formatDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-27 06:08:23 +0000