aboutsummaryrefslogtreecommitdiff
path: root/tar/tar2sqfs.c
AgeCommit message (Collapse)Author
2020-02-22libcommon: stdin file: Fix size accounting for sparse filesDavid Oberhollenzer
The file has to report the "apparent size" for sparse files, but internally work with the actual size in the tar ball. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-22Move inode size accounting completely to the block processorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-12Implement a more explicit object systemDavid Oberhollenzer
Make every dynamically allocated, opaque data structure inherit from a common sqfs_object_t structure with common entry points (e.g. destroy). This removes tons of public API functions and replaces them with a simple sqfs_destroy instead. If semantics of the (until now implicit) object system need to be extended, it can be much more conveniantely done this way. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-10Cleanup: Use stat structs instead of hooks in tar2sqfs/gensquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-01-19Cleanup: remove the payload pointers from sqfs_inode_generic_tDavid Oberhollenzer
There are 3 types of extra payload: - Directory index - File block sizes - Symlink target This commit removes the type specific pointers and modifies the code to use the payload area directly. To simplify the file block case and mitigate alignment issues, the type of the extra field is changed to sqfs_u32. For symlink target, the extra field can simply be cast to a character pointer (it had to be cast anyway for most uses). For block sizes, probably the most common usecase, it can be used as is. For directory indices, there is a helper function anyway. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-01-06Cleanup: use parse_size function to parse compressor optionsDavid Oberhollenzer
The XZ option parser had a similar function to parse_size. This commit removes the other implementation and extends parse_size with the one missing feature, i.e. allowing a '%' suffix for a relative value. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-01-02Support parsing [device] block size argument with SI suffixDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-22Add hard link support to gensquashfs and tar2sqfsDavid Oberhollenzer
In libtar, set a special flag if the header is actually a hard link. In tar2sqfs, create a hard link node and skip the rest for hard links. Also refues to set the root attributes from a hard link, it may refere to a node that we have missed earlier, there is nothing else that we can do here. In fstree_from_file, add a "link" command for adding hard links. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-22Add basic support for handling and serializing hard linksDavid Oberhollenzer
In libfstree, add a function to add a hard link to the fstree. The hard links stores the target in the data.target field, canonicalizes the target and sets a sentinel mode. A second function is used to resolve link, i.e. replacing it with a direct pointer, setting another sentinel mode and increasing the targets link count. The post process function tries to resolve unresolved hard links and only allocates inode numbers for nodes that aren't hard links. If the target node of a hard link does not have an inode number yet, the two need to be swapped, since this is also the order in which they are serialized. The serialization function in libcommon simply has to skip hard link nodes and when writing directory entries, use the inode num/ref of the target node. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-19Fix spelling mistakes found by lintianBenjamin Drung
Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>
2019-12-18Cleanup: merge the fstree post processing functionsDavid Oberhollenzer
Instead of having 3 different functions for sorting the tree, numbering the nodes and generating a file list, that all have to be used in the right order, this commit merges them into a single "fstree_post_process" function. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-15Clamp mtime in tar2sqfs to the representable 32 bit range.David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-13Better support for reading/writing non-ASCII xattr values from/to tarDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-13Add --no-tail-packing option to gensquashfs, tar2sqfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-02Add a --root-becomes option to tar2sqfs and sqfs2tarDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-28Cleanup: Return combined return value from compressor id by nameDavid Oberhollenzer
Instead of returning the ID through a pointer and an error code as return status, return a single int that could be a compressor ID (positive values) or an error code (negative values). Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-25Cleanup: move overflow safe alloc code into libsquashfsDavid Oberhollenzer
There were only a hand full of instances outside libsquashfs that used the alloc code. In most cases, the thing allocated hat its size derived from something already in memory anyway, so it is safe to assume its size fits into a size_t. At the same time, the opencoded Windows path conversion functions are all unified into a single helper function. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-24Cleanup: remove the entirely redundant sqfs_has_xattr functionDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-24Fix: Move LZO compressor from libsquashfs to libcommonDavid Oberhollenzer
The liblzo2 library is licensed under GPLv2, so it is not possible to distribute binaries of libsquashfs that link against liblzo2 under LGPL. This commit moves the LZO compressor implementation to libcommon, where this isn't a problem, since the tools themselves are licensed under GPLv3. It removes the ability of libsquashfs to read or generate LZO compressed SquashFS images, but the tools still can. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-23Move some unix header inclusions to compat.hDavid Oberhollenzer
In most cases, including unistd.h and fcntl.h was a left over anyway. In the cases where it was not, move it to compat.h. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-22Ensure that tar2sqfs & sqfs2tar set stdin/out to binary modeDavid Oberhollenzer
As usual, Windows has things different and is the platform where the problem was actually discovered. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-06Cleanup: remove BSD style __prognameDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-06Remove raw file descriptors from tar read pathDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-10-23Fix tar2sqfs: check for empty names or './' entrtyDavid Oberhollenzer
This commit makes sure entries with empty file names are skipped by tar2sqfs. The case of a directory with the name './' is treated specially, because tar can generate in some common use cases. For instance: $ mkdir foo $ touch foo/bar $ tar -C ./foo -c . > out.tar $ tar tf out.tar will generate a tar file with an entry named './'. For the future it might be interesting to turn that into the root inode somehow. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-10-23Fix memory leak in tar2sqfs if entries are skippedDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-10-07Rename "hihglevel.h" to the more appropriate "common.h"David Oberhollenzer
It only contains helpers for _common_ stuff for all the utilities. The actual high level stuff has been moved to libsquashfs a while ago. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-10-06Make --keep-time the default for tar2sqfsDavid Oberhollenzer
It is IMO more intuitive to use the timestamps from the archive by default and only replacing them if explicitly specified. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-10-05Improve error reporting for tar2sqfs and gensquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-10-05Merge all the common code for generating imagesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-29Cleanup: rename "compress.h" to "compressor.h"David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-29Fix inode numbering: always start with 1, use 0 as parent for the rootDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-29Cleanup: fstree no longer has any use for the block sizeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-28Replace fstree/sqfshelper xattr code with sqfs_xattr_writer_tDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Add a header for platform compatibillity fluffDavid Oberhollenzer
- We don't have "endian.h" everywhere. On some BSDs its in sys and on some BSDs the macros have different names. - We definitely don't have sysmacros.h on non-Unix-like systems. - Likewise for sys/types.h, sys/stat.h and their contents. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Cleanup: replace fixed with data types with typedefsDavid Oberhollenzer
This is a fully automated search and replace, i.e. I ran this: git grep -l uint8_t | xargs sed -i 's/uint8_t/sqfs_u8/g' git grep -l uint16_t | xargs sed -i 's/uint16_t/sqfs_u16/g' git grep -l uint32_t | xargs sed -i 's/uint32_t/sqfs_u32/g' git grep -l uint64_t | xargs sed -i 's/uint64_t/sqfs_u64/g' git grep -l int8_t | xargs sed -i 's/int8_t/sqfs_s8/g' git grep -l int16_t | xargs sed -i 's/int16_t/sqfs_s16/g' git grep -l int32_t | xargs sed -i 's/int32_t/sqfs_s32/g' git grep -l int64_t | xargs sed -i 's/int64_t/sqfs_s64/g' and than added the appropriate definitions to sqfs/predef.h The whole point being better compatibillity with platforms that may not have an stdint.h with the propper definitions. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Cleanup: remove most of the payload pointer magic from libfstreeDavid Oberhollenzer
Now that dir_info_t and file_info_t have reasonably small, use them in tree_node_t directly instead of doing pointer arithmetic magic on the payload area. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Cleanup: merge data.h into block.hDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-26Remove old data writer flags, and unused parametersDavid Oberhollenzer
- The write_data_from_file function no longer needs the block size - Also remove the old data writer flags, pass the flags argument directly to the begin-file function. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-26Remove remnants of the old data writerDavid Oberhollenzer
- Move the statstics hooks to the rest of the statistics code - Used the new data writer directly in gensquashfs & tar2sqfs - Demote what is left to a helper function for processing an input file and submitting it to the new data writer Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Remove condensed sparse file handling from libsquashfsDavid Oberhollenzer
This only exists for tar2sqfs. Move the sparse file map to libtar and add the ability to do this into the stind sqfs_file_t abstraction, so it acts like a normal file but internally stitches the data together from the sparse implementation. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Remove no-longer-used cruft from libfstreeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-22Use inode structures in the data writer instead of fstree file infoDavid Oberhollenzer
Changes are minor, but needs some temporary hacks in fstree again. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-21Minor fstree related cleanupDavid Oberhollenzer
- file list is no longer needed for statistics - the size field in the directory info structure is no longer in use - sqfs2tar, rdsquashfs and sqfsdiff no longer depend on libfstree.a Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-15Move condensed-sparse-file handling to libsquashfsDavid Oberhollenzer
Add another helper function for getting a blocks from a condensed sparse file, then rewrite the data writer to use that instead, using an sqfs_file_t pointer instead of direct file i/o. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-15Move sparse_map_t to libsquashfs headers, rename it to sqfs_sparse_map_tDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-15Replace file descriptor IO in data_writer with sqfs_file_tDavid Oberhollenzer
First, this commit moves the create-blocks-from-fd function over to libsquashfs and ports it to work on an sqfs_file_t instead. Second, the function in the data_writer that reads from a file descriptor is adjusted to use an sqfs_file_t instead. Finally, the tools that use it have to be adjusted accordingly. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-14Remove fstree file flagsDavid Oberhollenzer
As a side effect, this requires the data writer to keep track of statistics. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-10Make the thread pool queue backlog user configurableDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-09Remove ad-hoc image size accountingDavid Oberhollenzer
Much simpler to just set the correct size in the end, before writing the super block. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-08Replace direct file I/O with abstraction layerDavid Oberhollenzer
This should make it easier to use libsquashfs with custom setups that embedd a squashfs image inside something else. Also, it should make it easier to port to non unix-like platforms. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>