aboutsummaryrefslogtreecommitdiff
path: root/lib/sqfs
AgeCommit message (Collapse)Author
2023-10-11libsqfs: Rework the buffer advance logic for sqfs_istream_tDavid Oberhollenzer
Remove the asserts. If we eat more data than present, reset the offset and count to zero. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-09-19libsqfs: Add a helper function for turning an inode into a dir entryDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-09-19libsqfs: Make sure directory iterators always set the size for linksDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-09-19libsqfs: Expose the directory reader cache query functionDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-09-19libsqfs: Add a SquashFS implementation of the dir_iterator_tDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-09-15libsqfs: Cleanup readdir state structDavid Oberhollenzer
There is no need to keep a copy of the state data internally. If the external user wants to reset/rewind, they can simply keep a copy of the old state, now that it is completely transparent. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-09-15libsqfs: externalize the dir reader stateDavid Oberhollenzer
Instead of holding the state internally, move it outside and let the user pass it in. This allows processing multiple directories and going back and forth between them, while also simplifying the dir reader internals. The sqfs_dir_reader_get_inode function can also be made more generic to accept a reference instead of being fully state dependent. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-09-15libsqfs: Remove sqfs_dir_reader_t functions unused in the code baseDavid Oberhollenzer
Those functions are only used in the example code, we remove them for now to make restructuring easier and fix the examples later. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-09-15libsqfs: dir reader: encapsulate internal state in a separate objectDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-09-15libsqfs: Internalize sqfs_dir_reader_find functionDavid Oberhollenzer
The only user in the code base is sqfs_dir_reader_find_by_path. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-09-15libsqfs: remove dir reader internal.h, sub hierarchyDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-09-15Move directory tree related code from libsquashfs to libcommonDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-09-08Add a hard link detecting/filtering directory iteratorDavid Oberhollenzer
The reason this is implemented separately, instead of roling it into the recursive iterator, is so that we can do additional filtering in between. For instance, we can rewrite the path and the hard link path will match up, or if we remove nodes from the hierarchy, we won't end up with a hard link pointing outside. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-09-08libsqfs: bring sqfs_file_t in line with stream APIDavid Oberhollenzer
Rename the open function to sqfs_file_open, use an argument for the return pointer and pass back and error number on failure. Also add an inermediate function to open an sqfs_file_t using a handle, similar to the stream API. The get_file_size function is moved to the native wrappers and some of the implementation is cleaned up a little. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-08-18Add an inode fieled to sqfs_dir_entry_tDavid Oberhollenzer
This might be needed externally for things like hard link detection. For the unix implementation, store the actual inode number here. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-08-10Add a data reader based sqfs_istream_t implementationDavid Oberhollenzer
To the sqfs_data_reader_t is added, an sqfs_istream_t implementation that internally reads through the data reader. The uses of the data_reader_dump function are removed and the function is subsequently removed from libcommon. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-08-10Split recursive directory iteratorDavid Oberhollenzer
The recursive part and the filter part are split up, the recursive iterator wrapper is moved into libsquashfs and the libio iterator is modified to use that internally instead of implementig the recursion step. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-08-10Move dir_iterator_t from libio into libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-07-10Move sqfs_dir_entry_t code from libio into libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-07-03Refactor: rename dir_entry_t to sqfs_dir_entry_tDavid Oberhollenzer
Exact operation performed: git ls-files -z | \ xargs -0 sed -i -e 's/dir_entry_t/sqfs_dir_entry_t/g Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-07-03Refactor: rename sqfs_dir_entry_t to sqfs_dir_node_tDavid Oberhollenzer
Exact operation performed: git ls-files -z | \ xargs -0 sed -i -e 's/sqfs_dir_entry_t/sqfs_dir_node_t/g' Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-18Fix type flag decoding in win32 seek functionDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-18libsqfs: consistently rename the native file handle functionsDavid Oberhollenzer
Start with a common prefix "sqfs_native_file_*" Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-17Move data writer ostream into libsquashfsDavid Oberhollenzer
It is mainly a very thin wrapper on top of the block processor so far. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-16Cleanup: libsqfs: move the generic stream API to the rest of the I/O codeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-16libsquashfs: merge windows & unix file I/O primitivesDavid Oberhollenzer
By adding additional close/duplicate primitives, the remaining, mostly identical istream/ostream/file code is mostly identical between Windows and Unix and be merged, mostly without stitches. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-15libcompat: Add a helper to get/set OS error stateDavid Oberhollenzer
On Unix like OSes, this saves/restores errno, on Windows both errno and GetLastError state are saved/restored. This should make it simpler to preserve that across function calls. Additionally, while tracking down uses of GetLastError, some places where the error code was printed out directly where replaced with instances of w32_perror. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-15libsquashfs: cleanup the flag situation on istream/ostream functionsDavid Oberhollenzer
- The ostream creation functions already have flag arguments, but make them an sqfs_u32 instead of int. - Add flag arguments to the istream functions, sanitzie and forward them when opening the handle. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-15Migrate file istream/ostream from libio to libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-15libsquashfs: Add sqfs_open_native_file functionDavid Oberhollenzer
Having a function to interpret the flags and open a native file handle simplifies the istream/ostream/file code which shares that decoding part, particularly on windows where the character set needs to be transformed. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-15libsquashfs: Add a get_filename function to sqfs_file_tDavid Oberhollenzer
Similar to the sqfs_istream_t & sqfs_ostream_t interfaces. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-15Move sqfs_istream_t & sqfs_ostream_t into libsquashfsDavid Oberhollenzer
For now, only the interfaces and helper functions are moved, the concrete implementations remain in libio. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-15libio: remove ostream_append_sparse and fallback implementationDavid Oberhollenzer
Instead of a separate append-sparse function, simply accept NULL as an input for append. For both Unix and Win32, a fallback needs to be implemented. For XFRM, we can just memset the input buffer to zero, same for the libsquashfs data writer. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-07libsquashfs: Add utility functions to read xattrs into listDavid Oberhollenzer
The common pattern is used in rdsquashfs and sqfs2tar, move the code to libsquashfs. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-05libsqfs: Add an xattr writer function to add a combined key-value structDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-05Move dir_entry_xattr_t from libio to libsquashfsDavid Oberhollenzer
The structure and functions are renamed to sqfs_xattr_* instead, an additional helper is added to accept an encoded xattr. Documentation and unit test are added as well. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-01libsquashfs: remove the default block writer alignment featureDavid Oberhollenzer
The idea of the block align feature was to allow micro-managing that some files are forcefully aligned to 1k/4k ("device block") boundaries, hoping to improve access time at the cost of data density. The feature was not exposed in the tools for a long time and eventuall added to the sort file. Measurement and experimentation showed, that it in fact worsened the read performance on a test system with an old micro SD card as the bottle neck. The feature is removed, and if needed, can be brought back simply by wrapping/sub-classing the default block writer, if need be.. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-31Reintegrate test code with library codeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-31Move library source into src sub-directoryDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-19Implement rudimentary reference counting for sqfs_object_tDavid Oberhollenzer
Implement grab/drop functions to increase/decrease reference count and destroy the object if the count drops to 0. Make sure that all objects that maintain internal references actually grab that reference, duplicate it in the copy function, drop it in the destroy handler. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-19Add a helper function to initialize libsquashfs objectsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-19libsqfs: add a threshold for extended directory inodes with indexDavid Oberhollenzer
mksquashfs generates extended inodes if a directory contains 256 entries. libsquashfs so far only generated extended inodes if there is no other way to encode it. Mimic the behaviour of mksquashfs by adding a threshold. For this to work, the "sqfs_inode_set_xattr_index" function has to be changed to not immediately try to demote inodes to basic types. The fstree serialization is modified to do that itself if the index is 0xFFFFFFFF and the target is not a directory inode. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-22Get rid of the built-in copy of LZ4David Oberhollenzer
On Linux or BSD distributions we have a native version installed via package manager. On Windows, we can just build it from source like the other libraries. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-21Make some string functions from libcompat available to libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-18libsqfs: Fix an overzealous bounds check in the block processorDavid Oberhollenzer
When (during fragment deduplication) a fragment block is read back from disk and unpacked, it can happen that it is _exactly_ the given block size. The bounds check did '>=' instead of '>' and failed in that case with a "data corruption" error. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-18libsqfs: Initialize the return value in sqfs_compressor_createDavid Oberhollenzer
Initialize the output compressor pointer to NULL, so if the function fails, the value is propperly initialized to a NULL pointer instead of relying on the function user to initialize it. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-10-10block writer: further cleanup of the block writer logicDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-09-20block writer: move block comaprison to utility functionDavid Oberhollenzer
Slightly modify the byte-for-byte comparison function to compare an arbitrary range in a file and move it to libutil. Instead of calling it for each block in the block writer, simply let it check an entire range in the block writer and compute the range position/size of the reference ahead, before looking for potential matches. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-09-20block writer: remove open coded arrayDavid Oberhollenzer
Instead of open coding it, use the array_t type from libutil. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-07-08Make sqfs_tree_node_get_path more robustDavid Oberhollenzer
Test against various invariants: - Every non-root node must have a name - The root node muts not have a name - The name must not be ".." or "." - The name must not contain '/' - The loop that chases parent pointers must terminate, i.e. we must never reach the starting state again (link loop). Furthermore, make sure the sum of all path components plus separators does not overflow. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>