summaryrefslogtreecommitdiff
path: root/lib/comp
AgeCommit message (Collapse)Author
2019-08-19Minor improvements for parallel block processorDavid Oberhollenzer
- Fewer lock aquires in worker function - There is no point in locking/unlocking for inserting the completed block if we are going to lock again immediately in the next iteration -> Merge those two critical sections into one - Constant time queue insertion - Bypass queue entirely if there is nothing to do for a block Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-18Replace update_crc32 helper function with crc32 from zlibDavid Oberhollenzer
It is optimized to the maximum and if we already use zlib anyway, why not use zlib crc32? This also makes zlib a hard dependency which also means the whole "do we have a compressor" sanity check in the build system can be removed. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-18Add pthread based, parallel block processor implementationDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-18Add block processor data structureDavid Oberhollenzer
The interface is designed for parallel, asynchronuous processing of data blocks with an I/O callback that handles the serialized result. The underlying implementation is currently still synchronuous. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-08-16Add deep-copy function to compressor interfaceDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-07-30Add propper copyright headers to all source filesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-07-25Replace reads in squashfs with positional readsDavid Oberhollenzer
In most cases, we know exactely where the data that we want to read is on disk, so instead of using read() on the squashfs (or lseek + read), the code can in many places be cleaned up to use the pread wrapper read_data_at instead. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-07-24Include the reason ZSTD gave us for the errorMatt Turner
Without it you're left guessing or using a debugger to figure out what's wrong. Signed-off-by: Matt Turner <mattst88@gmail.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-07-24Enable largefile supportMatt Turner
Requires that config.h be included before other headers, since the macro _FILE_OFFSET_BITS changes the definitions of things like 'struct stat'. I chose to simply include it at the top of every C file and at immediately after the double-inclusion guards of every header. Signed-off-by: Matt Turner <mattst88@gmail.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-07-16cleanup: move error handling into read_retryDavid Oberhollenzer
If read_retry fails to read the expected amount of data (EOF or otherwise), it is almost always an error. This commit renames read_retry to read_data and moves error handling into the function, making a lot of error handling code redundant. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-07-16cleanup: move error handling into write_retryDavid Oberhollenzer
If write_retry fails to write everything, it is *always* an error. This commit renames write_retry to write_data and moves error handling into the function, making a lot of error handling code redundant. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-07-04Fix: possible out of bounds read in libcompress.aDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-07-01Fix memory leak in gzip compressorDavid Oberhollenzer
Don't allocate the compressor structure twice. Bug found using scan-build. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-06-23Move all handling of compressor names to libcompress.aDavid Oberhollenzer
This commit removes handling of compressor names from gensquashfs. Instead, functions are added to libcompress to obtain name from ID, ID from name and to print out defaults. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-06-11Cleanup dependency handlingDavid Oberhollenzer
Always try to gather all compressor libraries available, but only complain about missing a one if it has been *explicitly* selected. If a compressor has been explicityl disabled, we can still turn it off after checking. Also, rework gensquashfs to set the default compressor based on what's available. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-05-25Add options for lzma2 compressorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-05-23Add options for deflate compressorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-05-21Add options for lz4 compressorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-05-21Add options for lzo compressorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-05-21Add options for zstd compressorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-05-21Add command line flag for compressor options, pass them to compressorsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-05-19Add ZSTD compressor implementationDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-05-06Add lz4 compressor implementationDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-05-06Implement reading and writing of compressor optionsDavid Oberhollenzer
- gensquashfs simply asks the backend compressor to write its options to the file and does accounting - rdsquasfs simply asks the backend compressor to transparentyl snort the options from the file - not implemented in any compressor backend yet Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-05-06Add LZO compressor implementationDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-05-04Rename lzma compressor to xz, zlib compressor to gzipDavid Oberhollenzer
Stick to the terminology that SquashFS uses. SquashFS uses "lzma" to refere to lzma1, "xz" to refere to lzma2 and confusingly "gzip" to refere to raw zlib deflate. We can avoid a lot of confusion by _consistently_ using the same terminology. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-05-04Remove compressor internal buffersDavid Oberhollenzer
Pass in an external destination buffer + size and allow for propper bounds checking (especially when unpacking). Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-05-02Add compressor libraryDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-01 08:25:14 +0000