aboutsummaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)Author
2019-09-24Fix memory and pointer leaks in data reader error pathsDavid Oberhollenzer
Make sure that the block cache pointers are reset to NULL after freeing them, the get_block function does not update them on failure. Also, make sure all error paths in the get_block function actually clean up the allocated memory. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Bring back batched dequeuingDavid Oberhollenzer
Instead of doing piece-wise block dequeuing, take as many consequitive blocks as we can and in the comparetively rare* case that we produce a fragment block, put everything back for the next run. *rare because a lot of fragments are eaten by deduplication and even after that, a number of them are required to fill a block. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Clenaup error handlingDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Do the fragment checksumming in the workter threadDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Move entire fragment processing from data writer to block processorDavid Oberhollenzer
So far, this is mostly a direct port from the block processor. The actual fragment checksumming is not done through the thread pool. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Breake some of the helper functions out of process_block.cDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Fix trivial use-after-free bugDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Cleanup pthread based block processing codeDavid Oberhollenzer
Break convoluted, long functions up into smaller ones where the control flow (especially locking and signalling) is more easily readable and remove some copy and paste clean up code. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-24Process completed blocks one-by-oneDavid Oberhollenzer
This commit makes sure the block processor enqueue function dequeues at most a single block which it dispatches. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-23Cleanup block processor error handlingDavid Oberhollenzer
If an error happpens, immediately stop everything and report only the first error that happened. This also fixes a memory leak in the error paths of the serial and pthread based block processors. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-23Cleanup fragment handling in the data reader a littleDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-23Cleanup block processor flag handlingDavid Oberhollenzer
First, remove the "Don't checksum" flag, fragment blocks also need to be checksumed and are also subject to deduplication. For sentinel blocks, instead check if the size is zero. Second, cleanly seperate the user settable flags from the non-user settable flags and reject any block that has non-user settable or unknown flags set. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-23Move the bulk of the work from the data writer to the block processorDavid Oberhollenzer
Instead of calling a callback, the block processor now takes care of writing the data blocks to the file in the correct order, keeping track of fragment blocks and deduplicating blocks. Some cleanup work remains to be done and the statistics have to be repaired (yet again). Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-23Move fragment table resizing into block processor callbackDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-23Replace user pointer in sqfs_block_t with explicit inode pointerDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-22Use inode structures in the data writer instead of fstree file infoDavid Oberhollenzer
Changes are minor, but needs some temporary hacks in fstree again. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-22Move data writer custom flags over to block processorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-22Add helper functions for working with inodesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-21Merge some of serial & pthread block processor code paths/declarationsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Integrate the data reader into libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Add sqfs_ prefix to data_reader_t & functionsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Split data reader constructor and fragment table loadingDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Internal cleanup of data readerDavid Oberhollenzer
Make use of sqfs_block_t pointers for internal fragment and data block cache and reuse the get_block() for fetching them. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Cleanup: split "dir.h" into "dir_reader.h" and "dir_writer.h"David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Move data_reader_dump function out of data readerDavid Oberhollenzer
This commit adds two new functions for getting a file block by index, or a files fragment. The data_reader_dump function is rewritten in terms of those two functions and moved to a seperate file. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Dead code removal, round #2David Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Remove sqfs reader & fstree usage from sqfsdiffDavid Oberhollenzer
Replace with direct usage of the dir reader and lower level data structures. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Large round of dead code removalDavid Oberhollenzer
Remove all the library functions that no longer have any users. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Remove xattr restore & rest of rdtree flagsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Remove defunct rdtree flagsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Move canonicalize_name back to libutilDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Move "optimize unpack order" to from fstree to rdsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Port fstree get_path function to dir reader treeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Add data reader function to dump contents by inodeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Remove parallel unpackingDavid Oberhollenzer
Parallel unpacking didn't really improve the speed that much. Actually sorting the files for optimized unpack order improved speed much more than the parallel unpacker. Furthermore, the fork based parallel unpacker was actually pretty messy to begin with. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Add helper function to turn sqfs inode into struct statDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-20Add fstree like tree deserialization function to dir readerDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-19Add directory reader data structureDavid Oberhollenzer
This moves a lot of the stuff that is done manually in the tree deserializer to a generic helper in libsquashfs. Due to how the fstree is implemented, as a work around, the inode needs to be temporarily stored in the tree node, but some of the directory details could be removed. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-16Remove dead code from sqfshelperDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-15Fix data block deduplicationDavid Oberhollenzer
Because deduplication can now cross file boundaries, we have to include the fragment blocks in-between. Also, allow self-overlapping deduplication. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-15Merge data writer file processing functionsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-15Fix: work out the *current* file size when opening a fileDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-15Move condensed-sparse-file handling to libsquashfsDavid Oberhollenzer
Add another helper function for getting a blocks from a condensed sparse file, then rewrite the data writer to use that instead, using an sqfs_file_t pointer instead of direct file i/o. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-15Move sparse_map_t to libsquashfs headers, rename it to sqfs_sparse_map_tDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-15Replace file descriptor IO in data_writer with sqfs_file_tDavid Oberhollenzer
First, this commit moves the create-blocks-from-fd function over to libsquashfs and ports it to work on an sqfs_file_t instead. Second, the function in the data_writer that reads from a file descriptor is adjusted to use an sqfs_file_t instead. Finally, the tools that use it have to be adjusted accordingly. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-15Add an sqfs_file_t implementation for reading from STDINDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-15Add helper function to directory writer to create a basic inodeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-14Remove fstree file flagsDavid Oberhollenzer
As a side effect, this requires the data writer to keep track of statistics. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-14Move data deduplication from fstree code to data writerDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-13Make block process helper function internalDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-02 22:35:08 +0000