aboutsummaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)Author
2019-09-01Internalize the layout of the id_table_t structureDavid Oberhollenzer
As an opaque struct it has a chance to change its layout in the future without breaking ABI compatibiliy. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-01Move command line processing stuff out of compressor codeDavid Oberhollenzer
This commit moves stuff like printing help text, command line option processing and enumerating available processors on stdout out of the generic compressor code. The option string is replaced with a structure that directly exposese the tweakable parameters for all compressors. A function for parsing the command line arguments into this structure is added in sqfshelper. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-01API cleanup: Shuffle declarations aroundDavid Oberhollenzer
Move declarations for stuff that is defined in libsquashfs.so into the public headers and declarations for stuff that isn't, out of there. Also move the meta reader/writer helper functions to their respective headers. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-01Break up squashfs.h into topic related headersDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-01Install libsquashfs.so headers on the system in "sqfs" subdirectoryDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-01Move some application specific stuff out of libutilDavid Oberhollenzer
This commit does the following: - canonicalize_name is moved to libfstree - source_date_epoch is only used inside libfstree, so it's also moved over and can later be completely internalized - print_version is moved over to sqfshelper. Mainly so it doesn't end up in libsquashfs.so for no sane reason. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-31Turn libsquashfs.a into a shared libraryDavid Oberhollenzer
This of course entails turning the entire project over to libtool magic. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-31Split libsquashfs.a into low seperate librariesDavid Oberhollenzer
The idea is to make libsquashfs.a independend of libfstree.a, so it becomes a general purpose squashfs manipulation library. All the high level glue code for libfstree.a and utilites that are overly specific with to tools are moved to a seperate librarby. This commit makes the first step by moving the stuff with dependencies on libfstree to a seperate library. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-31Merge libcompress.a into libsquashfs.aDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-27Merge alloc_flex conditionals into oneDavid Oberhollenzer
It is shorter and less confusing for coverity. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-26Tune the paranoia down a bitDavid Oberhollenzer
size_t is guaranteed to be large enough to measure the size of things in memory, so when doing exactely that (e.g. strlen(a) + strlen(b)), checking for overflow is pointless since both objects are already in memory. If the addition would overflow, the two strings would occupy more memory than addressable. (Possible exception being some kind of harward style architecture with the two strings being in different kinds of memory of course.) Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-25Propperly set errno in read_inode_slink error pathDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-25Size accounting + alloc() overflow checking, round #3David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-23Check against format limits in meta_reader_read_dir_headerDavid Oberhollenzer
The SquashFS kernel implementation insists that a directory header is followed by no more than an upper bound of entries, way less than what the filed itself actually supports. This commit makes sure that the meta_reader_read_dir_header function also enforces that same limit. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-23Size accounting + alloc() overflow checking, round #2David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-23Do bounds checking in metadata readerDavid Oberhollenzer
In all cases where metadata blocks are read, we can roughly (in some cases even preciesly) say in what range those metadata blocks will be, so it makes sense to throw an error if an attempt is made to wander outside this range. Furthermore, when reading from an uncompressed block, it is more reasonable to check against the actual block bounds than to padd it with 0 bytes. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-23Some simple search/replace cases for allocationDavid Oberhollenzer
This commit exchanges some malloc(x + y * z) patterns that can be found with a simple git grep and are obvious for the new wrappers. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-23Add wrappers for calloc style functions with size overflow checkingDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-22deserialize_tree: filter out directory loopsDavid Oberhollenzer
The tree deserializer does a recursive depth-first search to populate the directory tree, moving back and forth between the directory listing containing the inode references and the inode table pointing to the list of child inodes. It is completely unaware of hard links and creates duplicate nodes instead. It is possible to create a malicious SquashFS image that contains a directory that contains as child a reference to its own inode. This can also be done transitively (i.e. directory contains its own parent or grand parent), leading to infinite recursion (actually finite, since it terminates once all stack memory is exhausted). This commit adds a simple check to see if a node has the same inode number as any of its would-be parents. If it does, the node is discarded and a warning message is emitted. Other cases with arbitrary layers of indirection could be constructed as well (e.g. dir 'a' contains hard link to 'b' and 'b' one back to 'a'), but the sub hierarchies are always expanded, this check should catch that too. Reported-by: Zachary Dremann <dremann@gmail.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-21Fix "no attributes" sentinel value for xattr readerDavid Oberhollenzer
An inode can be of extended type for reasons other than having extended attributes and simply set the xattr ID to 0xFFFFFFFF to indicate that it doesn't have extended attributes. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-19Fix file list generation: break any pre-existing connectionDavid Oberhollenzer
If the linked list pointer was already used before, break up the connection so we don't risk running into a loop or something when regenerating the list. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-19Minor improvements for parallel block processorDavid Oberhollenzer
- Fewer lock aquires in worker function - There is no point in locking/unlocking for inserting the completed block if we are going to lock again immediately in the next iteration -> Merge those two critical sections into one - Constant time queue insertion - Bypass queue entirely if there is nothing to do for a block Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-19Fix memory leak in data writer fragment deduplicationDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-19Fix memory leak in data writer error code pathsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-19Fix memory leak in dir-scan error code pathDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-18Replace update_crc32 helper function with crc32 from zlibDavid Oberhollenzer
It is optimized to the maximum and if we already use zlib anyway, why not use zlib crc32? This also makes zlib a hard dependency which also means the whole "do we have a compressor" sanity check in the build system can be removed. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-18Make data writer use block processorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-18Restructure data writer around passing block_t structuresDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-18Minor interface change to data writerDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-18cleanup: internalize deduplication list in data_writerDavid Oberhollenzer
This change removes the need for passing a list of files around for deduplication. Also the deduplication code no longer needs to worry about order, since the file being deduplicated is only added after deduplication is done. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-18Add pthread based, parallel block processor implementationDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-18Add block processor data structureDavid Oberhollenzer
The interface is designed for parallel, asynchronuous processing of data blocks with an I/O callback that handles the serialized result. The underlying implementation is currently still synchronuous. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-16Add deep-copy function to compressor interfaceDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-16Fix: don't try to read xattrs if there are noneDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-11Add gensquashfs option to read xattrs from input filesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-11Add --one-file-system option to gensquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-11Replace fstree_from_dir boolean with flag fieldDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-07Add pread(2) like function to data_readerDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-07Fix forward seek when unpacking sparse filesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-07Fix zero padding of extracted data blocksDavid Oberhollenzer
Only padd it if the *extracted* size is less then block size. Doing it with the compressed size results in garbled blocks. Especially because most of them are less than block size when compressed. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-05cleanup data readerDavid Oberhollenzer
- Split block reading code out from "dump_blocks" into precache_data_block, similar to precache_fragment_block - Merge the code paths for fragment/data block reading and uncompression Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-05cleanup: unify all the code that reads squashfs imagesDavid Oberhollenzer
This commit creates a new data structure called 'sqfs_reader_t' that takes care of all the repetetive tasks like opening the file, reading the super block, creating the compressor, deserializing an fstree and creating a data reader. This in turn makes it possible to remove all the duplicate code from rdsquashfs and sqfs2tar. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-04Improve file unpacking orderDavid Oberhollenzer
This commit moves the file unpacking order & job scheduling to a libfstree function. The ordering is improved by making sure fragment blocks are not extracted more than once and files with data blocks are extracted in order. This way, serial unpacking of a 2GiB Debian live image could be reduced from ~5' on my test machine to ~3.5', whereas parallel unpacking stays roughly the same (~3' for -j 4). Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-04Fix functions with side effect being used inside assertsDavid Oberhollenzer
If -DNDEBUG is set, the entire thing is omitted from the output. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-03Fix tar header error reporting on 32 bit systemsDavid Oberhollenzer
If an extension header is rejected because its too big, the error path would print the size as size_t, altough it is an uint64_t. On 64 bit systems, this works because size_t is a 64 bit unsigned integer, on 32 bit systems, not so much. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-03cleanup: remove left over atime/ctime codeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-02Fix explicit NULL dereference in deserialize_fstree failure pathDavid Oberhollenzer
If we failed to create the root node, we don't need to cleanup the fstree_t which would attempt to recursively cleanup the root node. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-02cleanup: merge error paths in xattr reader restore_kv_pairsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-02Fix potential double free of xattr reader id_block_startsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-02Implement support for SOURCE_DATE_EPOCH environment variableDavid Oberhollenzer
reproducible-builds.org suggests the use of an environment variable as a source for time stamps: https://reproducible-builds.org/specs/source-date-epoch/ This commit adds support for setting the default mtime from the variable, if it is set and only defaulting to 0 if not. The timestamp given by the command line switch takes precedence. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-18 16:47:36 +0000