summaryrefslogtreecommitdiff
path: root/lib/sqfs/block_processor/serial.c
AgeCommit message (Collapse)Author
2021-01-19libsqfs: Implement exact matching of fragmentsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-01-19libsqfs: Add a sqfs_block_processor_create_ex functionDavid Oberhollenzer
This function creates a block processor from a structure describing it. A stub implementation for the old sqfs_block_processor_create is added that simply sets up such a struct and forwards the call. The current version of the description struct only contains the exact same parameters and a size field at the beginning. This approach is supposed to make extending the range of parameters easier without breaking ABI compatibillity. Currently already planned are: - Adding a sqfs_file_t pointer to double-check when deduplicating fragments. - When the scanning code reaches a usable state, add the abillity to pass scanned fragment data, so the block processor can be used for appending to an existing image. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-01-19Fix: Move fragment consolidation back to block processor serial partDavid Oberhollenzer
Keeping a list of fragments stored away in the current fragment block and consolidating them in the thread pool takes them out of circulation. If we have a lot of tiny fragments, this can lead to a situation where all the limit is reached, but we cannot do anything, because we are waiting for a block to complete, but they are all attached to the current fragment block and the queue is empty. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-06-07block processor: add an internal common cleanup functionDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-05-29Block processor: merge finish & sync functionsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-05-29Block processor: turn internal functions into interface entry pointsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-05-23block processor: move the block consolidation to the worker threadDavid Oberhollenzer
Instead of merging fragments into the fragment block inside the process_completed_fragment function, store a linked list of fragments in the fragment block and do the actual merging (several memcpy calls totaling of up to 1M of data in worst case) in the worker thread instead of the locked, serial path. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-05-23block processor: recycle blocks to reduce allocation pressureDavid Oberhollenzer
Instead of freeing/allocating blocks all the time in the locked, serial path, use a free list to "recycle" blocks. Once a block is no longer used, throw it onto the free list. If a new block is, needed try to get one from the free list before calling malloc. After a few iterations, the block processor should stop allocating new blocks and only re-use the ones it already has. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-22Add a seperate sqfs_block_processor_sync functionDavid Oberhollenzer
This function waits for all pending blocks to be written to disk, but doesn't flush the fragment block, so processing can continue afterwards as if nothing happened. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-16block processor: move the internals to the respective implementationsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-16Move all the queue-waiting logic to the thread pool implemenationDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-16block processor: move sparse block detection into worker threadDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-15Cleanup: block processor: move init/cleanup functions into implemenationsDavid Oberhollenzer
Again, the generic init/cleanup functions do way too many things that are specific to the thread pool implementation. Thanks to the splitting up of the block processor, they also have become quite trivial. This commit moves those functions into their respective implementations, allowing even further simplificiation. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-15Cleanup: block processor: move finish function back into implementationsDavid Oberhollenzer
The "generic" version actually uses specific internals of the thread pool implementation. Move it back into the thread pool based implementation and simplify the serial processor. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-15Cleanup: block processor: remove test_and_set_statusDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-15Cleanup: block processor: remove delayed thread notificationDavid Oberhollenzer
Under the assumption that block processing is CPU bound and not I/O bound, this is entirely useless. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-12Implement a more explicit object systemDavid Oberhollenzer
Make every dynamically allocated, opaque data structure inherit from a common sqfs_object_t structure with common entry points (e.g. destroy). This removes tons of public API functions and replaces them with a simple sqfs_destroy instead. If semantics of the (until now implicit) object system need to be extended, it can be much more conveniantely done this way. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-09Move block writer and fragment table management out of block processorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-01-29Rename sqfs_data_writer_t back to sqfs_block_processor_tDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>