aboutsummaryrefslogtreecommitdiff
path: root/lib/common
AgeCommit message (Collapse)Author
2020-09-16Remodel file extraction tools to use libfstreamDavid Oberhollenzer
This commit rewrites the libtar write paths to use libfstream insead of a FILE pointer. Also, the libcommon file extraction function is remodeled to use libfstream. In accordance, rdsquashfs, sqfs2tar and sqfsdiff have some minor adjustments made to work with the ported libtar and libcommon. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-16Implement a data writer based output streamDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-08-26Fix tree node path generation for detached sub treesDavid Oberhollenzer
The function sqfs_tree_node_get_path is used in several places within rdsquashfs to produce a path for a tree node, either when describing the file system, or when unpacking it. Unpacking can be done on sub-trees as well as the entire tree, in which case the root of the sub-tree has its parent pointer removed, so the full path terminates at the new root. This works with directories, since they receive special case handling anyway, but fails if the sub-tree to unpack is only a single file because the sqfs_tree_node_get_path function assumes that we are at the tree root and returns "/" as a path, which gets normalized to "". This commit adds a workaround to the function to simply use the nodes name (if available) in that case instead. The describe case in rdsquashfs is unaffacted, since it always starts at the root. Likewise, the sqfs2tar case should also be unaffacted, since it already employs special case handling for the [sub] tree root node. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-06-11Add flags to functions that might logically be expanded in the futureDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-06-04Cleanup: libcommon: use global LUTs for compressor optionsDavid Oberhollenzer
Instead of the convoluted logic, simply use a small number of LUTs that point to the available compressor flags for each compressor, the avaialble options and their ranges. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-06-04Cleanup: Pull compression level parameter out into compressor configDavid Oberhollenzer
Every compressor (except LC4) has a compression level parameter. This commit pulls the compression level field out into the generic configuration structure and applies some code clean ups as a result from this. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-06-04lzma compressor: support micro management optionsDavid Oberhollenzer
The LZMA compressor (through the xz-utils library) supports basically the same options for micro management as the XZ compressor. This commit enables support for those options in the compressor, the option parser and adds an option field to the configuration structure. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-06-04lzma compressor: add support for the "extreme" flagDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-05-29Support associating a user pointer with data blocksDavid Oberhollenzer
This commit modifies the block processor to support associating a user data pointer with data blocks that it forwards to the block writer, which is modified to accept an optional user data pointer. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-05-29Turn the sqfs_block_writer_t into an interfaceDavid Oberhollenzer
This way, everything that could be done through the hooks (and more) can be done by simply providign a custom implementation. The result is a lot clener that the previous hook based version. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-05-29cleanup: libsqfs: eliminate block writer statisticsDavid Oberhollenzer
- the "bytes submitted" can be moved over to the block processor - the number of blocks submitted are already there (implcitily, by adding the data block count to the fragment block count) - actual data bytes written can be computed from the super block - the remaining block count can be changed to simple counter that can be obtained through a function. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-05-04Expose more fine grained control values & flags on the XZ compressorDavid Oberhollenzer
This patch allows external users to fiddle with the XZ compressors compression strength, alignment and other values. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-04-16tar2sqfs & gensquashfs: Delete the output file on failureDavid Oberhollenzer
This commit changes the tar2sqfs & gensquashfs code to pass the exit status on to sqfs_writer_cleanup in libcommon. The function sqfs writer code in libcommon is changed to retain the output file name and delete it if the status passed to the cleanup function is anything other than EXIT_SUCCESS. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-04-01Fix missing header without LZOAlyssa Ross
lib/common/compress.c: In function 'compressor_get_default': lib/common/compress.c:39:2: warning: implicit declaration of function 'assert' [-Wimplicit-function-declaration] 39 | assert(0); | ^~~~~~ lib/common/compress.c:8:1: note: 'assert' is defined in header '<assert.h>'; did you forget to '#include <assert.h>'? 7 | #include "common.h" +++ |+#include <assert.h> 8 |
2020-03-19Fix compressor availability check in libcommonDavid Oberhollenzer
Initialize have_compressor to false before testing, to make the check work. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-03-05Get rid of sqfs_compressor_existsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-03-05Change the signature of sqfs_compressor_create to return an error codeDavid Oberhollenzer
Make sure the function has a way of telling the caller *why* it failed. This way, the function can convey whether it had an internal error, an allocation failure, whether the arguments are totaly nonsensical, or simply that the compressor *or specific configuration* is not supported. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-03-05Cleanup: Remove the E_ prefix from all libsquashfs enumeratorsDavid Oberhollenzer
Avoid namespace polution. Make sure all exportet symbols are prefixed with either sqfs_ or SQFS_. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-03-04Add a generic copying mechanism to sqfs_object_tDavid Oberhollenzer
This patch adds a deep-copy callback to sqfs_object_t and removes the copying mechanism from sqfs_compressor_t. This is also interesting for other types. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-03-01Fix printf format specifies for sqfs_u64David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-03-01Fix: Replace bit shifts in parse_size with SZ_MUL_OVDavid Oberhollenzer
On 32 bit systems, size_t is a 32 bit integer and doing 64 bit shifts won't do us any good. So instead, use the existing size_t multiply and catch overflow macros. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-27Add a function to the compressor interface to get the configurationDavid Oberhollenzer
This allows getting the compressor configuration back after creating it, for various purposes. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-23Turn file inode management completely over to the block processorDavid Oberhollenzer
If the block processor allocates and dynamically resizes inodes on the fly, we can add data indefinitely without knowing the size of the file ahead of time. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-23Unify the payload counters in the sqfs_inode_generic_tDavid Oberhollenzer
Instead of having seperate counters for blocks, dir index bytes and having to fiddle out the link target size, simply use a single value that stores the number of payload bytes used. A seperate "payload bytes available" is used for dynamically growing inodes during processing. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-22libcommon: stdin file: Fix size accounting for sparse filesDavid Oberhollenzer
The file has to report the "apparent size" for sparse files, but internally work with the actual size in the tar ball. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-22Move inode size accounting completely to the block processorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-22Make hard link detection scale betterDavid Oberhollenzer
Instead of doing a linear search, which scales quadratically, use a red-black tree with inode numbers as key for finding hard links. To reiterate: we can't just use a flat table, because the SquashFS file is potentially untrusted and the inode numbers can be anything, no matter what the super block says. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-15Add a "sequence error" to libsquashfs error codesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-12Implement a more explicit object systemDavid Oberhollenzer
Make every dynamically allocated, opaque data structure inherit from a common sqfs_object_t structure with common entry points (e.g. destroy). This removes tons of public API functions and replaces them with a simple sqfs_destroy instead. If semantics of the (until now implicit) object system need to be extended, it can be much more conveniantely done this way. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-12Remove usage of sqfs_block_t from block readerDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-10Cleanup statistics print outDavid Oberhollenzer
- Give a rounded input/output byte count. - Seperate groups by new lines. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-10Cleanup: Use stat structs instead of hooks in tar2sqfs/gensquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-09Move block writer and fragment table management out of block processorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-01-29Rename sqfs_data_writer_t back to sqfs_block_processor_tDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-01-26Fix memory leak in hard link detection codeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-01-20Add a flag field to the id table create functionDavid Oberhollenzer
Just to be safe in case there needs to be an extension in the future. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-01-19Cleanup: remove the payload pointers from sqfs_inode_generic_tDavid Oberhollenzer
There are 3 types of extra payload: - Directory index - File block sizes - Symlink target This commit removes the type specific pointers and modifies the code to use the payload area directly. To simplify the file block case and mitigate alignment issues, the type of the extra field is changed to sqfs_u32. For symlink target, the extra field can simply be cast to a character pointer (it had to be cast anyway for most uses). For block sizes, probably the most common usecase, it can be used as is. For directory indices, there is a helper function anyway. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-01-06Cleanup: use parse_size function to parse compressor optionsDavid Oberhollenzer
The XZ option parser had a similar function to parse_size. This commit removes the other implementation and extends parse_size with the one missing feature, i.e. allowing a '%' suffix for a relative value. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-01-02Support parsing [device] block size argument with SI suffixDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-23Add simple sqfs_tree_node_t hardlink detection to libcommonDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-23Bring back the flat list of inodes in libfstreeDavid Oberhollenzer
It makes further processing simpler and doesn't leak the abstraction into upper layers. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-22Add basic support for handling and serializing hard linksDavid Oberhollenzer
In libfstree, add a function to add a hard link to the fstree. The hard links stores the target in the data.target field, canonicalizes the target and sets a sentinel mode. A second function is used to resolve link, i.e. replacing it with a direct pointer, setting another sentinel mode and increasing the targets link count. The post process function tries to resolve unresolved hard links and only allocates inode numbers for nodes that aren't hard links. If the target node of a hard link does not have an inode number yet, the two need to be swapped, since this is also the order in which they are serialized. The serialization function in libcommon simply has to skip hard link nodes and when writing directory entries, use the inode num/ref of the target node. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-19Fix link count for file inodes in tree serialization.David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-18Add an explicit link count to the fstree nodesDavid Oberhollenzer
Gets initialized to 2 for directories, 1 for all other types. The count of the parent node is automatically incremented. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-18Rename fstree "slink_target" to "target"David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-18Move is_filename_sane to libfstree, add test casesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-18Cleanup: merge the fstree post processing functionsDavid Oberhollenzer
Instead of having 3 different functions for sorting the tree, numbering the nodes and generating a file list, that all have to be used in the right order, this commit merges them into a single "fstree_post_process" function. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-16Remove fstree inode tableDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-16Do the fstree serialization in a recursive tree walkDavid Oberhollenzer
Remove usage of the "inode table". Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-16Use the sqfs_dir_writer_t to create the NFS export tableDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-01 17:34:19 +0000