squashfs-tools-ng.git - A new set of tools and libraries for working with SquashFS images

Age	Commit message (Collapse)	Author
2020-05-23	block processor: recycle blocks to reduce allocation pressure	David Oberhollenzer
	Instead of freeing/allocating blocks all the time in the locked, serial path, use a free list to "recycle" blocks. Once a block is no longer used, throw it onto the free list. If a new block is, needed try to get one from the free list before calling malloc. After a few iterations, the block processor should stop allocating new blocks and only re-use the ones it already has. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-03-18	Cleanup: Move xxhash32 code to libutil	David Oberhollenzer
	Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-23	Turn file inode management completely over to the block processor	David Oberhollenzer
	If the block processor allocates and dynamically resizes inodes on the fly, we can add data indefinitely without knowing the size of the file ahead of time. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-21	Cleanup: move utilities back out of libsquashfs	David Oberhollenzer
	This commit removes the allocation helpers and string table functions out of libsquashfs back into a "libutil.a". The problem of libsquashfs exporting stuff that it shouldn't is resolved by retaining the internal attributes and directly adding the source to libsquashfs instead of trying to somehow link against libutil.la. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-20	Restructure thread pool block processor	David Oberhollenzer
	Implement the io-queue based design as outline in doc/parallelism.txt Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-16	block processor: move the internals to the respective implementations	David Oberhollenzer
	Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-16	Replace crc32 with xxhash32	David Oberhollenzer
	On the one hand, benchmarking and profiling determined xxhash32 to be faster than the zlib implementation of crc32, on the other hand profiling determined that crc32 computation contributed signifficantly to the overall runtime. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-16	Move all the queue-waiting logic to the thread pool implemenation	David Oberhollenzer
	Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-15	Move block block accounting to the other end of the block pipeline	David Oberhollenzer
	This commit moves all of the fragment/block accounting in the block processor over to the writing end of the pipeline. In order to do this, the sparse blocks are allowed to bubble through the pipeline. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-15	Cleanup: block processor: move init/cleanup functions into implemenations	David Oberhollenzer
	Again, the generic init/cleanup functions do way too many things that are specific to the thread pool implementation. Thanks to the splitting up of the block processor, they also have become quite trivial. This commit moves those functions into their respective implementations, allowing even further simplificiation. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-15	Cleanup: block processor: remove test_and_set_status	David Oberhollenzer
	Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-15	Cleanup: block processor: remove delayed thread notification	David Oberhollenzer
	Under the assumption that block processing is CPU bound and not I/O bound, this is entirely useless. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-12	Implement a more explicit object system	David Oberhollenzer
	Make every dynamically allocated, opaque data structure inherit from a common sqfs_object_t structure with common entry points (e.g. destroy). This removes tons of public API functions and replaces them with a simple sqfs_destroy instead. If semantics of the (until now implicit) object system need to be extended, it can be much more conveniantely done this way. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-12	Cleanup: Move sqfs_block_t to block processor internals	David Oberhollenzer
	It was basically built around the block processor and exposed way too many internals. Removing it from other places was mostly trivial. This commit completely removes it from the public API and even most of the libsquashfs internals. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-10	Cleanup: remove block hooks entirely from block processor	David Oberhollenzer
	Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-10	Add run time statistics to the block writer and processor	David Oberhollenzer
	Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-09	Move block writer and fragment table management out of block processor	David Oberhollenzer
	Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-01-31	Split the block writing/deduplication away from the block processor	David Oberhollenzer
	This commit moves the entire block writing and deduplication of data blocks over to a different data type named "block writer". For simplicity, the interfaces of the block processor are left as is and are turned into warppers. Likewise, most of the code in the block writer is just verbatim from the block processor, to be cleaned up in subsequent commits. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-01-29	Rename sqfs_data_writer_t back to sqfs_block_processor_t	David Oberhollenzer
	Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>