aboutsummaryrefslogtreecommitdiff
path: root/unpack
AgeCommit message (Collapse)Author
2020-02-10Cleanup statistics print outDavid Oberhollenzer
- Give a rounded input/output byte count. - Seperate groups by new lines. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-01-20Add a flag field to the id table create functionDavid Oberhollenzer
Just to be safe in case there needs to be an extension in the future. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-01-19Cleanup: remove the payload pointers from sqfs_inode_generic_tDavid Oberhollenzer
There are 3 types of extra payload: - Directory index - File block sizes - Symlink target This commit removes the type specific pointers and modifies the code to use the payload area directly. To simplify the file block case and mitigate alignment issues, the type of the extra field is changed to sqfs_u32. For symlink target, the extra field can simply be cast to a character pointer (it had to be cast anyway for most uses). For block sizes, probably the most common usecase, it can be used as is. For directory indices, there is a helper function anyway. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-13rdsquashfs: print hex dump of xattrs if it isn't an ASCII/UTF-8 stringDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-12Fix: programs linking against libsquashfs also need pthreadDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-09Only check for OS specific bad filenames when unpackingDavid Oberhollenzer
When converting a SquashFS image to a tarball, it makes no sense to refuse conversion if the filename is considered evil by the OS. This patch adds an option to is_filename_sane to check if the OS has a problem with the given file name. sqfs2tar sets it to false and converts everything while rdsquashfs sets it to true when unpacking. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-25Cleanup: remove what is left of libutilDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-25Cleanup: move overflow safe alloc code into libsquashfsDavid Oberhollenzer
There were only a hand full of instances outside libsquashfs that used the alloc code. In most cases, the thing allocated hat its size derived from something already in memory anyway, so it is safe to assume its size fits into a size_t. At the same time, the opencoded Windows path conversion functions are all unified into a single helper function. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-24Cleanup: move canonicalize_name back to libfstree.aDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-24Fix: Move LZO compressor from libsquashfs to libcommonDavid Oberhollenzer
The liblzo2 library is licensed under GPLv2, so it is not possible to distribute binaries of libsquashfs that link against liblzo2 under LGPL. This commit moves the LZO compressor implementation to libcommon, where this isn't a problem, since the tools themselves are licensed under GPLv3. It removes the ability of libsquashfs to read or generate LZO compressed SquashFS images, but the tools still can. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-23Move some unix header inclusions to compat.hDavid Oberhollenzer
In most cases, including unistd.h and fcntl.h was a left over anyway. In the cases where it was not, move it to compat.h. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-22Cleanup: move all the compatibillity fluff to a dedicated "libcompat"David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-19Make rdsquashfs unpack work on WindowsDavid Oberhollenzer
AFAIK the only thing we can actually unpack on Windows is regular files and directories, so only do that. Furthermore, we have no way of setting Unix file attributes, so skip that. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-18Remove pushd/popd usage from rdsquashfs unpacking codeDavid Oberhollenzer
Instead, reconstruct the full path and use that instead. We do that anyway if --quiet is not set. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-06Cleanup: remove BSD style __prognameDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-06Remove raw file descriptors from unpack write pathsDavid Oberhollenzer
Instead, use stdio FILE pointers. On POSIX systems, use fileno to get the file descriptor and hopefully create sparase files. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-10-07Do an explicit "is filename sane" checkDavid Oberhollenzer
Until now, filenames containing '/' or being equal to '..' or '.' where not handled explicitly, because they are canonicalized later, which will then fail. This commit adds an explicit check to make those fail immediately with a clear, specific error message. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-10-07Cleanup: move libutil related headers to "util" sub directoryDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-10-07Rename libsqfshelper to libcommonDavid Oberhollenzer
That is IMO less confusing and express what it is (i.e. what it has become) more clearly, i.e. common code shared by the utilities. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-10-07Rename "hihglevel.h" to the more appropriate "common.h"David Oberhollenzer
It only contains helpers for _common_ stuff for all the utilities. The actual high level stuff has been moved to libsquashfs a while ago. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-10-06Fix missing error checking in diagnostic printingDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-10-06Improve error reporting for sqfs2tar, rdsquashfs, sqfsdiffDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-10-05Fix typoMatt Turner
Signed-off-by: Matt Turner <mattst88@gmail.com>
2019-09-29Cleanup: rename "compress.h" to "compressor.h"David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-28Move declarations for sqfs_xattr_reader_t to a seperate headerDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-28Fix absolute file paths in rdsquashfs unpackerDavid Oberhollenzer
Canonicalize the file names to make the diagnostics less scary. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Add a header for platform compatibillity fluffDavid Oberhollenzer
- We don't have "endian.h" everywhere. On some BSDs its in sys and on some BSDs the macros have different names. - We definitely don't have sysmacros.h on non-Unix-like systems. - Likewise for sys/types.h, sys/stat.h and their contents. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Cleanup: replace fixed with data types with typedefsDavid Oberhollenzer
This is a fully automated search and replace, i.e. I ran this: git grep -l uint8_t | xargs sed -i 's/uint8_t/sqfs_u8/g' git grep -l uint16_t | xargs sed -i 's/uint16_t/sqfs_u16/g' git grep -l uint32_t | xargs sed -i 's/uint32_t/sqfs_u32/g' git grep -l uint64_t | xargs sed -i 's/uint64_t/sqfs_u64/g' git grep -l int8_t | xargs sed -i 's/int8_t/sqfs_s8/g' git grep -l int16_t | xargs sed -i 's/int16_t/sqfs_s16/g' git grep -l int32_t | xargs sed -i 's/int32_t/sqfs_s32/g' git grep -l int64_t | xargs sed -i 's/int64_t/sqfs_s64/g' and than added the appropriate definitions to sqfs/predef.h The whole point being better compatibillity with platforms that may not have an stdint.h with the propper definitions. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Cleanup: merge data.h into block.hDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-22Add helper functions for working with inodesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-21Minor fstree related cleanupDavid Oberhollenzer
- file list is no longer needed for statistics - the size field in the directory info structure is no longer in use - sqfs2tar, rdsquashfs and sqfsdiff no longer depend on libfstree.a Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Integrate the data reader into libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Add sqfs_ prefix to data_reader_t & functionsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Split data reader constructor and fragment table loadingDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Move data_reader_dump function out of data readerDavid Oberhollenzer
This commit adds two new functions for getting a file block by index, or a files fragment. The data_reader_dump function is rewritten in terms of those two functions and moved to a seperate file. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Remove fstree code from rdsquashfsDavid Oberhollenzer
Use the directory reader from libsquashfs instead. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Move "optimize unpack order" to from fstree to rdsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Remove parallel unpackingDavid Oberhollenzer
Parallel unpacking didn't really improve the speed that much. Actually sorting the files for optimized unpack order improved speed much more than the parallel unpacker. Furthermore, the fork based parallel unpacker was actually pretty messy to begin with. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Remove sqfs reader from rdsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-01Break up squashfs.h into topic related headersDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-01Install libsquashfs.so headers on the system in "sqfs" subdirectoryDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-31Turn libsquashfs.a into a shared libraryDavid Oberhollenzer
This of course entails turning the entire project over to libtool magic. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-31Split libsquashfs.a into low seperate librariesDavid Oberhollenzer
The idea is to make libsquashfs.a independend of libfstree.a, so it becomes a general purpose squashfs manipulation library. All the high level glue code for libfstree.a and utilites that are overly specific with to tools are moved to a seperate librarby. This commit makes the first step by moving the stuff with dependencies on libfstree to a seperate library. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-31Merge libcompress.a into libsquashfs.aDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-19Fix chmod of symlinks in restore_fstreeDavid Oberhollenzer
We can't change the mod of symlinks and hence AT_SYMLINK_NOFOLLOW is not supported by chmod(2 -> RTFM!). The solution is to not use that flag and simply skip symlinks. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-19Fix directory traversal in update_tree_attribsDavid Oberhollenzer
Do not try to change the nameless root node which is mapped to the output directory. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-19Add rdsquashfs flag to restore file timestampsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-05cleanup: unify all the code that reads squashfs imagesDavid Oberhollenzer
This commit creates a new data structure called 'sqfs_reader_t' that takes care of all the repetetive tasks like opening the file, reading the super block, creating the compressor, deserializing an fstree and creating a data reader. This in turn makes it possible to remove all the duplicate code from rdsquashfs and sqfs2tar. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-04Improve file unpacking orderDavid Oberhollenzer
This commit moves the file unpacking order & job scheduling to a libfstree function. The ordering is improved by making sure fragment blocks are not extracted more than once and files with data blocks are extracted in order. This way, serial unpacking of a 2GiB Debian live image could be reduced from ~5' on my test machine to ~3.5', whereas parallel unpacking stays roughly the same (~3' for -j 4). Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-04Fix functions with side effect being used inside assertsDavid Oberhollenzer
If -DNDEBUG is set, the entire thing is omitted from the output. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>