summaryrefslogtreecommitdiff
path: root/lib/sqfs
AgeCommit message (Collapse)Author
2019-09-28Fix out of bounds check in lzo_compressor_createDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-28Fix lzma2 VLI filter testingDavid Oberhollenzer
Starting out with 0, makes the loop condition evaluate to false and none of the filters are tested. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-28Fix memory leak in meta reader read inode functionDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-28Fix unwanted sign extension in lzma decompressorDavid Oberhollenzer
First extend the uint8_t to size_t, then do the arithmetic, otherwise there is an addtional implicit conversion to int, possibly resulting in unwanted sign extension. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-28Make coverity happyDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Add a header for platform compatibillity fluffDavid Oberhollenzer
- We don't have "endian.h" everywhere. On some BSDs its in sys and on some BSDs the macros have different names. - We definitely don't have sysmacros.h on non-Unix-like systems. - Likewise for sys/types.h, sys/stat.h and their contents. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Replace usage of mode_t in libsquashfs with sqfs_u16David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Remove use of ssize_t from library APIDavid Oberhollenzer
ssize_t is only available on POSIX platforms and even there it is only defined to hold at least -1 in the range of negative numbers. This commit replaces ssize_t return types with sqfs_s32 and the coresponding function arguments with sqfs_u32. Because the range of positiv numbers for a signed 32 bit number is only half that of the unsigned version, additional checks have to be added. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Cleanup: replace fixed with data types with typedefsDavid Oberhollenzer
This is a fully automated search and replace, i.e. I ran this: git grep -l uint8_t | xargs sed -i 's/uint8_t/sqfs_u8/g' git grep -l uint16_t | xargs sed -i 's/uint16_t/sqfs_u16/g' git grep -l uint32_t | xargs sed -i 's/uint32_t/sqfs_u32/g' git grep -l uint64_t | xargs sed -i 's/uint64_t/sqfs_u64/g' git grep -l int8_t | xargs sed -i 's/int8_t/sqfs_s8/g' git grep -l int16_t | xargs sed -i 's/int16_t/sqfs_s16/g' git grep -l int32_t | xargs sed -i 's/int32_t/sqfs_s32/g' git grep -l int64_t | xargs sed -i 's/int64_t/sqfs_s64/g' and than added the appropriate definitions to sqfs/predef.h The whole point being better compatibillity with platforms that may not have an stdint.h with the propper definitions. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Add a pkg-config file for libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Fix typo in error enumeratorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Cleanup: merge data.h into block.hDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Fix lzma compressor: test if compression actually shrunk the dataDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Add xz-utils based lzma compressor implementationDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-26Remove the create I/O block from sqfs_file_t functionDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-26Replace the data writer enqueue with "append buffer to current file"David Oberhollenzer
This commit turns the file interface into an actual, generic file interface and does away with having to move around blocks outside the data writer. Instead the data writer takes over full control and responsibility of dividing the input data up into blocks propperly. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-26Add file API stub to sqfs data writerDavid Oberhollenzer
Basically move the state tracking from the old data writer over to the new one. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Remove condensed sparse file handling from libsquashfsDavid Oberhollenzer
This only exists for tar2sqfs. Move the sparse file map to libtar and add the ability to do this into the stind sqfs_file_t abstraction, so it acts like a normal file but internally stitches the data together from the sparse implementation. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Cleanup: move the stdin sqfs_file_t wrapper out of libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Add the ability to hook into the data writer block writingDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Move sqfs_block_t to its own headerDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Rename block processor to sqfs_data_writer_tDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25More refactoring of the block processorDavid Oberhollenzer
Basically just moving functions around and renaming things. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Cleanup "signature" misnomer. It's actually a hash value.David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Minor cleanup on fragment handling codeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Minor fixup for internal block processor headerDavid Oberhollenzer
- Add copyright header - Remove redundant header inclusions - Remove unused macros Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Refactor out common code of the block processorsDavid Oberhollenzer
Code that already is shared between the pthread and the serial processor as well as code that can be re-used by other threading API implementations. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Cleanup pthread queue processing codeDavid Oberhollenzer
This commit removes duplicated code paths from the block processor finish function, i.e. reimplementing it using the existing queue processing function and merges cleanup code paths in the later. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Fix zero size stack array allocation in write_inodeDavid Oberhollenzer
Simply skip copying of file blocks if there are none. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Fix memory and pointer leaks in data reader error pathsDavid Oberhollenzer
Make sure that the block cache pointers are reset to NULL after freeing them, the get_block function does not update them on failure. Also, make sure all error paths in the get_block function actually clean up the allocated memory. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Bring back batched dequeuingDavid Oberhollenzer
Instead of doing piece-wise block dequeuing, take as many consequitive blocks as we can and in the comparetively rare* case that we produce a fragment block, put everything back for the next run. *rare because a lot of fragments are eaten by deduplication and even after that, a number of them are required to fill a block. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Clenaup error handlingDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Do the fragment checksumming in the workter threadDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Move entire fragment processing from data writer to block processorDavid Oberhollenzer
So far, this is mostly a direct port from the block processor. The actual fragment checksumming is not done through the thread pool. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Breake some of the helper functions out of process_block.cDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Cleanup pthread based block processing codeDavid Oberhollenzer
Break convoluted, long functions up into smaller ones where the control flow (especially locking and signalling) is more easily readable and remove some copy and paste clean up code. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Process completed blocks one-by-oneDavid Oberhollenzer
This commit makes sure the block processor enqueue function dequeues at most a single block which it dispatches. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-23Cleanup block processor error handlingDavid Oberhollenzer
If an error happpens, immediately stop everything and report only the first error that happened. This also fixes a memory leak in the error paths of the serial and pthread based block processors. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-23Cleanup block processor flag handlingDavid Oberhollenzer
First, remove the "Don't checksum" flag, fragment blocks also need to be checksumed and are also subject to deduplication. For sentinel blocks, instead check if the size is zero. Second, cleanly seperate the user settable flags from the non-user settable flags and reject any block that has non-user settable or unknown flags set. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-23Move the bulk of the work from the data writer to the block processorDavid Oberhollenzer
Instead of calling a callback, the block processor now takes care of writing the data blocks to the file in the correct order, keeping track of fragment blocks and deduplicating blocks. Some cleanup work remains to be done and the statistics have to be repaired (yet again). Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-23Replace user pointer in sqfs_block_t with explicit inode pointerDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-22Add helper functions for working with inodesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-21Merge some of serial & pthread block processor code paths/declarationsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Integrate the data reader into libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Cleanup: split "dir.h" into "dir_reader.h" and "dir_writer.h"David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Add fstree like tree deserialization function to dir readerDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-19Add directory reader data structureDavid Oberhollenzer
This moves a lot of the stuff that is done manually in the tree deserializer to a generic helper in libsquashfs. Due to how the fstree is implemented, as a work around, the inode needs to be temporarily stored in the tree node, but some of the directory details could be removed. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-15Fix: work out the *current* file size when opening a fileDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-15Move condensed-sparse-file handling to libsquashfsDavid Oberhollenzer
Add another helper function for getting a blocks from a condensed sparse file, then rewrite the data writer to use that instead, using an sqfs_file_t pointer instead of direct file i/o. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-15Replace file descriptor IO in data_writer with sqfs_file_tDavid Oberhollenzer
First, this commit moves the create-blocks-from-fd function over to libsquashfs and ports it to work on an sqfs_file_t instead. Second, the function in the data_writer that reads from a file descriptor is adjusted to use an sqfs_file_t instead. Finally, the tools that use it have to be adjusted accordingly. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-09 02:43:43 +0000