aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2019-09-27Remove use of ssize_t from library APIDavid Oberhollenzer
ssize_t is only available on POSIX platforms and even there it is only defined to hold at least -1 in the range of negative numbers. This commit replaces ssize_t return types with sqfs_s32 and the coresponding function arguments with sqfs_u32. Because the range of positiv numbers for a signed 32 bit number is only half that of the unsigned version, additional checks have to be added. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Cleanup: replace fixed with data types with typedefsDavid Oberhollenzer
This is a fully automated search and replace, i.e. I ran this: git grep -l uint8_t | xargs sed -i 's/uint8_t/sqfs_u8/g' git grep -l uint16_t | xargs sed -i 's/uint16_t/sqfs_u16/g' git grep -l uint32_t | xargs sed -i 's/uint32_t/sqfs_u32/g' git grep -l uint64_t | xargs sed -i 's/uint64_t/sqfs_u64/g' git grep -l int8_t | xargs sed -i 's/int8_t/sqfs_s8/g' git grep -l int16_t | xargs sed -i 's/int16_t/sqfs_s16/g' git grep -l int32_t | xargs sed -i 's/int32_t/sqfs_s32/g' git grep -l int64_t | xargs sed -i 's/int64_t/sqfs_s64/g' and than added the appropriate definitions to sqfs/predef.h The whole point being better compatibillity with platforms that may not have an stdint.h with the propper definitions. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Add a pkg-config file for libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Fix typo in LZO_LIBS variableDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Cleanup: remove most of the payload pointer magic from libfstreeDavid Oberhollenzer
Now that dir_info_t and file_info_t have reasonably small, use them in tree_node_t directly instead of doing pointer arithmetic magic on the payload area. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Fix typo in error enumeratorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Add missing comments on error enumeratorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Cleanup: merge data.h into block.hDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Fix lzma compressor: test if compression actually shrunk the dataDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Add xz-utils based lzma compressor implementationDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-26Update documentation on sqfs_data_writer_tDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-26Remove old data writer flags, and unused parametersDavid Oberhollenzer
- The write_data_from_file function no longer needs the block size - Also remove the old data writer flags, pass the flags argument directly to the begin-file function. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-26Remove the create I/O block from sqfs_file_t functionDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-26Replace the data writer enqueue with "append buffer to current file"David Oberhollenzer
This commit turns the file interface into an actual, generic file interface and does away with having to move around blocks outside the data writer. Instead the data writer takes over full control and responsibility of dividing the input data up into blocks propperly. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-26Add file API stub to sqfs data writerDavid Oberhollenzer
Basically move the state tracking from the old data writer over to the new one. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-26Remove remnants of the old data writerDavid Oberhollenzer
- Move the statstics hooks to the rest of the statistics code - Used the new data writer directly in gensquashfs & tar2sqfs - Demote what is left to a helper function for processing an input file and submitting it to the new data writer Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Fix API changes in unit testsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Remove condensed sparse file handling from libsquashfsDavid Oberhollenzer
This only exists for tar2sqfs. Move the sparse file map to libtar and add the ability to do this into the stind sqfs_file_t abstraction, so it acts like a normal file but internally stitches the data together from the sparse implementation. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Cleanup: move the stdin sqfs_file_t wrapper out of libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Do the statistics acocunting through data writer hooksDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Add the ability to hook into the data writer block writingDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Move sqfs_block_t to its own headerDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Remove no-longer-used cruft from libfstreeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Rename block processor to sqfs_data_writer_tDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25More refactoring of the block processorDavid Oberhollenzer
Basically just moving functions around and renaming things. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Cleanup "signature" misnomer. It's actually a hash value.David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Minor cleanup on fragment handling codeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Minor fixup for internal block processor headerDavid Oberhollenzer
- Add copyright header - Remove redundant header inclusions - Remove unused macros Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Refactor out common code of the block processorsDavid Oberhollenzer
Code that already is shared between the pthread and the serial processor as well as code that can be re-used by other threading API implementations. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Cleanup pthread queue processing codeDavid Oberhollenzer
This commit removes duplicated code paths from the block processor finish function, i.e. reimplementing it using the existing queue processing function and merges cleanup code paths in the later. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Fix zero size stack array allocation in write_inodeDavid Oberhollenzer
Simply skip copying of file blocks if there are none. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Fix memory and pointer leaks in data reader error pathsDavid Oberhollenzer
Make sure that the block cache pointers are reset to NULL after freeing them, the get_block function does not update them on failure. Also, make sure all error paths in the get_block function actually clean up the allocated memory. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Bring back batched dequeuingDavid Oberhollenzer
Instead of doing piece-wise block dequeuing, take as many consequitive blocks as we can and in the comparetively rare* case that we produce a fragment block, put everything back for the next run. *rare because a lot of fragments are eaten by deduplication and even after that, a number of them are required to fill a block. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Clenaup error handlingDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Do the fragment checksumming in the workter threadDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Move entire fragment processing from data writer to block processorDavid Oberhollenzer
So far, this is mostly a direct port from the block processor. The actual fragment checksumming is not done through the thread pool. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Breake some of the helper functions out of process_block.cDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Fix trivial use-after-free bugDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Cleanup pthread based block processing codeDavid Oberhollenzer
Break convoluted, long functions up into smaller ones where the control flow (especially locking and signalling) is more easily readable and remove some copy and paste clean up code. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Process completed blocks one-by-oneDavid Oberhollenzer
This commit makes sure the block processor enqueue function dequeues at most a single block which it dispatches. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-23Cleanup block processor error handlingDavid Oberhollenzer
If an error happpens, immediately stop everything and report only the first error that happened. This also fixes a memory leak in the error paths of the serial and pthread based block processors. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-23Cleanup fragment handling in the data reader a littleDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-23Cleanup block processor flag handlingDavid Oberhollenzer
First, remove the "Don't checksum" flag, fragment blocks also need to be checksumed and are also subject to deduplication. For sentinel blocks, instead check if the size is zero. Second, cleanly seperate the user settable flags from the non-user settable flags and reject any block that has non-user settable or unknown flags set. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-23Move the bulk of the work from the data writer to the block processorDavid Oberhollenzer
Instead of calling a callback, the block processor now takes care of writing the data blocks to the file in the correct order, keeping track of fragment blocks and deduplicating blocks. Some cleanup work remains to be done and the statistics have to be repaired (yet again). Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-23Move fragment table resizing into block processor callbackDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-23Replace user pointer in sqfs_block_t with explicit inode pointerDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-22Use inode structures in the data writer instead of fstree file infoDavid Oberhollenzer
Changes are minor, but needs some temporary hacks in fstree again. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-22Move data writer custom flags over to block processorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-22Remove block processor ordering testDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-22Add helper functions for working with inodesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-27 07:52:37 +0000