aboutsummaryrefslogtreecommitdiff
path: root/lib/fstream
AgeCommit message (Collapse)Author
2021-06-25libfstream: sanity check the buffer size in the gzip stream compressorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25libfstream: guard against potential integer overflowsDavid Oberhollenzer
The differen compressor libraries use differnt integer types to tally the buffer sizes. The libfstream library uses size_t, which may be bigger than the actualy types, potentially causing an overflow if trying to compress to much at once. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-04Fix: allow concatenated Bzip2 streamsDavid Oberhollenzer
This is a followup to dd4e6ead142e58568aec89d76b0b2e867ee983f2. Basically the same problem occours with Bzip2, but it so far it wasn't possible to find a sampel that reproduces it. Unlike libxz, the libbz2 API does not support concatenated streams by itself and will choke when trying to decompress after the stream end, so this commit adds a workaround to simply initialize the decompressor on-the-fly and tear it down again when and end-of-stream is returned. The end-of-file condition is only set when there actually is no more data to read. Otherwise, the decompressor will be re-initialized in the next round. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-05-06Fix: allow concatenated xz streamsDavid Oberhollenzer
Some xz compressed tarballs (e.g. from kernel.org) are not made up of a single xz stream, but rather contain several, independendly compressed streams. In that case, the xz decompressor hits an LZMA_STREAM_END early on and reports EOF. If you are lucky, the tar reader bails (premature end-of-file). If you are unlucky, it happens exactely between two records and is interpeted as regular end-of-file. As this seems to be a normal use case for xz, it has a flag to just read across the seams and only report end-of-stream if the action is set to finish. This commit adds the flag to the initialization propperly sets the lzma_action depending on whether the underlying stream hit EOF or not. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-03-30Fix: don't throw an error if fsync() returns EINVALDavid Oberhollenzer
This indicates that sync isn't possible on the underlying file descriptor (e.g. a pipe), which currently causes sqfs2tar to err if the output isn't written directly to a file. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-03-20Fix: add missing include path to libfstream if using builtin zlibDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-29Add a more usefull getline-like function to libfstreamDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-29libfstream: Rework unix sparse file writingDavid Oberhollenzer
Instead of immediately seeking forward, record the sparse bytes "written" and then do the seek the next time we write data, or do an ftruncate if we do a sync. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-17Add a check whether libzstd supports stream compressionDavid Oberhollenzer
Streaming compression was added fairly recently to zstd (and then the API was changed a few times). Rather than rely on libzstd versioning macros, this commit adds an m4 script to test at configure time if the enums/functions we need are available by trying to compile a small sample. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-17Fix: add missing compiler/linker flagsDavid Oberhollenzer
If tar2sqfs and sqfs2tar want to use comperssion via libfstream, they need to link against the compressor libraries. Also, libfstream may need the apropriate cflags to access the headers on some platforms. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-16Add bzip2 stream compression supportDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-16Add zstd stream compressor implementation to libfstreamDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-16Implement istream decompression supportDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-16Implement ostream compressor wrapperDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-16Add stream I/O abstraction libraryDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>