aboutsummaryrefslogtreecommitdiff
path: root/lib/sqfs
AgeCommit message (Collapse)Author
2023-07-10Move sqfs_dir_entry_t code from libio into libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-07-03Refactor: rename dir_entry_t to sqfs_dir_entry_tDavid Oberhollenzer
Exact operation performed: git ls-files -z | \ xargs -0 sed -i -e 's/dir_entry_t/sqfs_dir_entry_t/g Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-07-03Refactor: rename sqfs_dir_entry_t to sqfs_dir_node_tDavid Oberhollenzer
Exact operation performed: git ls-files -z | \ xargs -0 sed -i -e 's/sqfs_dir_entry_t/sqfs_dir_node_t/g' Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-18Fix type flag decoding in win32 seek functionDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-18libsqfs: consistently rename the native file handle functionsDavid Oberhollenzer
Start with a common prefix "sqfs_native_file_*" Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-17Move data writer ostream into libsquashfsDavid Oberhollenzer
It is mainly a very thin wrapper on top of the block processor so far. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-16Cleanup: libsqfs: move the generic stream API to the rest of the I/O codeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-16libsquashfs: merge windows & unix file I/O primitivesDavid Oberhollenzer
By adding additional close/duplicate primitives, the remaining, mostly identical istream/ostream/file code is mostly identical between Windows and Unix and be merged, mostly without stitches. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-15libcompat: Add a helper to get/set OS error stateDavid Oberhollenzer
On Unix like OSes, this saves/restores errno, on Windows both errno and GetLastError state are saved/restored. This should make it simpler to preserve that across function calls. Additionally, while tracking down uses of GetLastError, some places where the error code was printed out directly where replaced with instances of w32_perror. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-15libsquashfs: cleanup the flag situation on istream/ostream functionsDavid Oberhollenzer
- The ostream creation functions already have flag arguments, but make them an sqfs_u32 instead of int. - Add flag arguments to the istream functions, sanitzie and forward them when opening the handle. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-15Migrate file istream/ostream from libio to libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-15libsquashfs: Add sqfs_open_native_file functionDavid Oberhollenzer
Having a function to interpret the flags and open a native file handle simplifies the istream/ostream/file code which shares that decoding part, particularly on windows where the character set needs to be transformed. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-15libsquashfs: Add a get_filename function to sqfs_file_tDavid Oberhollenzer
Similar to the sqfs_istream_t & sqfs_ostream_t interfaces. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-15Move sqfs_istream_t & sqfs_ostream_t into libsquashfsDavid Oberhollenzer
For now, only the interfaces and helper functions are moved, the concrete implementations remain in libio. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-15libio: remove ostream_append_sparse and fallback implementationDavid Oberhollenzer
Instead of a separate append-sparse function, simply accept NULL as an input for append. For both Unix and Win32, a fallback needs to be implemented. For XFRM, we can just memset the input buffer to zero, same for the libsquashfs data writer. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-07libsquashfs: Add utility functions to read xattrs into listDavid Oberhollenzer
The common pattern is used in rdsquashfs and sqfs2tar, move the code to libsquashfs. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-05libsqfs: Add an xattr writer function to add a combined key-value structDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-05Move dir_entry_xattr_t from libio to libsquashfsDavid Oberhollenzer
The structure and functions are renamed to sqfs_xattr_* instead, an additional helper is added to accept an encoded xattr. Documentation and unit test are added as well. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-01libsquashfs: remove the default block writer alignment featureDavid Oberhollenzer
The idea of the block align feature was to allow micro-managing that some files are forcefully aligned to 1k/4k ("device block") boundaries, hoping to improve access time at the cost of data density. The feature was not exposed in the tools for a long time and eventuall added to the sort file. Measurement and experimentation showed, that it in fact worsened the read performance on a test system with an old micro SD card as the bottle neck. The feature is removed, and if needed, can be brought back simply by wrapping/sub-classing the default block writer, if need be.. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-31Reintegrate test code with library codeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-31Move library source into src sub-directoryDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-19Implement rudimentary reference counting for sqfs_object_tDavid Oberhollenzer
Implement grab/drop functions to increase/decrease reference count and destroy the object if the count drops to 0. Make sure that all objects that maintain internal references actually grab that reference, duplicate it in the copy function, drop it in the destroy handler. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-19Add a helper function to initialize libsquashfs objectsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-19libsqfs: add a threshold for extended directory inodes with indexDavid Oberhollenzer
mksquashfs generates extended inodes if a directory contains 256 entries. libsquashfs so far only generated extended inodes if there is no other way to encode it. Mimic the behaviour of mksquashfs by adding a threshold. For this to work, the "sqfs_inode_set_xattr_index" function has to be changed to not immediately try to demote inodes to basic types. The fstree serialization is modified to do that itself if the index is 0xFFFFFFFF and the target is not a directory inode. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-22Get rid of the built-in copy of LZ4David Oberhollenzer
On Linux or BSD distributions we have a native version installed via package manager. On Windows, we can just build it from source like the other libraries. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-21Make some string functions from libcompat available to libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-18libsqfs: Fix an overzealous bounds check in the block processorDavid Oberhollenzer
When (during fragment deduplication) a fragment block is read back from disk and unpacked, it can happen that it is _exactly_ the given block size. The bounds check did '>=' instead of '>' and failed in that case with a "data corruption" error. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-18libsqfs: Initialize the return value in sqfs_compressor_createDavid Oberhollenzer
Initialize the output compressor pointer to NULL, so if the function fails, the value is propperly initialized to a NULL pointer instead of relying on the function user to initialize it. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-10-10block writer: further cleanup of the block writer logicDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-09-20block writer: move block comaprison to utility functionDavid Oberhollenzer
Slightly modify the byte-for-byte comparison function to compare an arbitrary range in a file and move it to libutil. Instead of calling it for each block in the block writer, simply let it check an entire range in the block writer and compute the range position/size of the reference ahead, before looking for potential matches. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-09-20block writer: remove open coded arrayDavid Oberhollenzer
Instead of open coding it, use the array_t type from libutil. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-07-08Make sqfs_tree_node_get_path more robustDavid Oberhollenzer
Test against various invariants: - Every non-root node must have a name - The root node muts not have a name - The name must not be ".." or "." - The name must not contain '/' - The loop that chases parent pointers must terminate, i.e. we must never reach the starting state again (link loop). Furthermore, make sure the sum of all path components plus separators does not overflow. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-07-08Move sqfs_tree_node_get_path to libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-07-08Cleanup: move libutil headers to sub directoryDavid Oberhollenzer
Move all the libutil stuff from the toplevel include/ to a util/ sub directory and fix up the includes that make use of them. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-06-02Cleanup: libsqfs: simplify state handling in dir readerDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-06-02Cleanup: libsqfs: sqfs_dir_reader_find_by_pathDavid Oberhollenzer
Split out several repated patterns into helper functions and move the rest of the code back into dir_reader.c Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-06-02Cleanup: libsqfs: merge dir cache code back into dir_reader.cDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-06-02Cleanup: libsqfs: move directory iteration out of the directory readerDavid Oberhollenzer
Add a simple directory state object to the meta data reader and use that to iterate directory entries. The code for reading the directory listing is movde to readdir.c Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-06-01Fix: libsqfs: do not report out of bounds positions from meta readerDavid Oberhollenzer
When asking the meta data reader for its current position and we *just* read to the end of a block, report the start of the next block as the current location. Otherwise, trying to *seek* to the resulting position immediately after reporting throws an out-of-bounds error. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-04-10Remove builtin copy of zlibDavid Oberhollenzer
On GNU/Linux, *BSD or MacOS we can simply use the system default library. The copy was primarily only there for the Windows build. The build script for Windows has now been adapted to download and compile a shared library from a tarball. This removes a huge chunk of code from the git tree as well as the release tarballs. Additionally it gets rid of iffy things like removing the Zlib copyright/version strings, so the libsquashfs DLL doesn't export it. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-04-09Add support for '.' and '..' entries in sqfs_dir_reader_tDavid Oberhollenzer
Two flags are added to the dir reader API, one for the create function that the dir reader should report those entries and one to the open function to suppress that if it was enabled. To implement the feature, a mapping of visited directory inodes is maintained internally, that mapps inode numbers to inode references. When opening a directory, state is maintained to generate the fake entries for '.' and '..'. Since all the other functions are based on the open/read/rewind API, no alterations need to be made. The tree scan function is modified, to use the suppress flag, so it does not accidentally catch those entries. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-04-05libsqfs: move dir reader code to sub directory, add internal headerDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-30sqfs_dir_tree_destroy/sqfs_destroy: allow NULL inputLuca Boccassi
Many library destructor functions (like free()) allow a NULL pointer as input, and do nothing in that case. This allows easier cleanup patterns: initialize pointers to NULL and then always pass them to the destroyer functions, no need for verbose goto/if-else patterns. Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com>
2022-03-10Fix: guard against potential overflow in file size calculationDavid Oberhollenzer
The block_count is a size_t, so on 32 bit platforms the multiplication might be truncated before the comparison with filesz. On 64 bit platforms, it could potentially also overflow the 64 bit bounds of the data type. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-12-05Fix: consistently use the widechar file API on WindowsDavid Oberhollenzer
When opening files on windows, use the widechar versions and convert from (assumed) UTF-8 to UTF-16 as needed. Since the broken, code-page-random API may acutall be intended in some use cases, leave that option in through an additional flag. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-08-22Tighten bounds checks in sqfs_dir_reader_readerDavid Oberhollenzer
Use the same size check as sqfs_dir_reader_open_dir and report EOF, even if it is possible to read the header itself, but nothing beyond that. Also check if it should be possible to read an entry header before attempting and report EOF if not. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-08-22Fix half done initialization of sqfs_dir_reader_open_dirDavid Oberhollenzer
The sqfs_dir_reader_open_dir function tried to take a short-cut by returning early if the target directory is empty. However, this left some field unchanged from the previous directory. If iterating over a directory and then deciding to enter a sub-directory that happens to be empty, the directory reader will keep the settings for the current directory. After calling sqfs_dir_reader_rewind, the sub-directory will suddenly report the contents of the parent. A similar check is added to the rewind function to not track back on the meta data reader in that case. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-07-21Fix libsquashfs directory writer size accountingDavid Oberhollenzer
The squashfs readdir() implementation in the Linux kernel returns non-existing "." and ".." entries for offsets 0 and 1, and after that reads from disk. For convenience, it was decided to store an off-by-3 value on disk instead of doing complex primary school math to adjust for this. This didn't show up until now, because the kernel implementation trusts the value from the directory header more than the actual size in the inode and happily reads 3 more than the inode would allow it to. This only showed up with 7-zip which subtracts 3 from the size and expects the result to be exact and bails if the directory headers suggest otherwise. And yes, I did consider making a "Holy Hand Granade of Antioch" reference, but consciously decided not to. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25Add default cases for every switch blockDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25Remove casual un-const casting in various placesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>