aboutsummaryrefslogtreecommitdiff
path: root/lib/tar
AgeCommit message (Collapse)Author
2023-02-21libtar: generate entire xattr header in a single bufferDavid Oberhollenzer
Having it all in one buffer allows us the re-use the "generat GNU record" function. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-20Remove ostream_printfDavid Oberhollenzer
By cobbling together the xattr lines manually in libtar, the need (and thus the function itself) are removed. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-12libtar: Add a test for the tar writing codeDavid Oberhollenzer
Generate a simple tarball and compare it with a reference. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-08libtar: remove need for skip_padding functionDavid Oberhollenzer
In the istream implementation, automatically skip the padding when we reach end-of-file. Also skip file AND padding when we destroy the object. Replace the remaining instances with a simple istream_skip instead and remove the wrapper from libtar. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-08libtar: Add an istream_t implementationDavid Oberhollenzer
The tar_istream_t reads the data from a tar file, having been given the header, and synthesizes zero bytes for sparse regions. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-04libtar: internalize the declaration of read_octalDavid Oberhollenzer
Use read_number in the places that remain. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-04libtar: simplfy parsing of old GNU sparse formatDavid Oberhollenzer
There was some code duplication for extracting the sparse entry fields from the start record and the subsequent extended record. This commit introduces a data structure for both and unifies the parsing code paths. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-04libtar: some minor cleanupsDavid Oberhollenzer
- Use is_memory_zero from libutil - Move checksum update function to tar writer code - Move checksum verify function to tar reader code - Only export the function to compute the checksum Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-31Reintegrate test code with library codeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-31Move library source into src sub-directoryDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-19libtar: simplify padd_file functionDavid Oberhollenzer
We have an "append_sparse" function in libio, with a fallback, so use that. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-18Add a single, central base64 decoderDavid Oberhollenzer
Similar to the hex blob decoder, we need this once for tar and once for the filemap xattr parser. Simply add a single, central implementation to libutil, with a simple unit test, and then use it in both libtar and gensquashfs. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-18Add a single, central hex blob decoderDavid Oberhollenzer
Since we need it twice (once for tar, once for the filemap xattr parser), add a single, central implementation to libutil, add a unit test for that implementation and then use it in both libtar and gensquashfs. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-07-08Cleanup: split libtar header, move to sub directoryDavid Oberhollenzer
Some of the on-disk format internals are moved to a separate header and some of the stuff from internal.h is moved to that format header. C++ guards are added in addtion. Everything PAX related is moved to pax_header.c, some internal functions are marked as static. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-30Cleanup: remove struct stat from libtarDavid Oberhollenzer
The idea was originally to use struct stat in the libfstree code, so we can simply hose data read from a directory into the fstree_t. The struct was then also used with libtar, for simpler interoperation, but it turned out to introduce a lot of platform quirks and causes more trouble than it's worth. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-30Cleanup: table driven pax header parsingDavid Oberhollenzer
Instead of having a long if-else-if chain, replace the PAX header field parsing with a table driven approach. Altough it is more code, it is hopefully more readable, maintainable, extensible and it dedupliates some of the value parsing code. The GNU.sparse parsing is left as is, because it requires maintaining state. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-30Cleanup: pax header parsingDavid Oberhollenzer
Split the key/value pairs right in the header and terminate the key name. This way, some of the magic numbers can be eliminated. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-10Cleanup libtar mkxattr, explicitly null-terminate stringsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-07-09Fix printf format specifiers used for generating tarballsDavid Oberhollenzer
When processing files > 4G, using "%o" truncates the result and the tarball is not readable. This should have been discovered when auto-patching the printf format specifiers, but a cast was added instead and the issue was overlooked. This commit replaces the down-cast and printf format specifiers. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25Add default cases for every switch blockDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25Remove casual un-const casting in various placesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-16Remodel libtar/tar2sqfs to read data from an istream_tDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-16Remodel file extraction tools to use libfstreamDavid Oberhollenzer
This commit rewrites the libtar write paths to use libfstream insead of a FILE pointer. Also, the libcommon file extraction function is remodeled to use libfstream. In accordance, rdsquashfs, sqfs2tar and sqfsdiff have some minor adjustments made to work with the ported libtar and libcommon. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-03Fix integer bounds checking in GNU tar sparse format 1.0 parserDavid Oberhollenzer
- Make sure the file actually has that many records before trying to read one and fail if not. - Use the helper macros for size_t overflow checking instead of assuming size_t == uint64_t. - Impose a "reasonable" upper bound on the number of data segments and insist that there is at least one entry. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-02Fix nonexistant gnu tar sparse format 1.0 supportDavid Oberhollenzer
Contrary to previous claims, support for the GNU tar sparse format 1.0 was missing entirely (the newest of their 3 different sparse mapping formats). This oversight wasn't caught, because the unit test was compiling the wrong source file and tar2sqfs had no problem processing the test file because it is still a valid POSIX-ish tar archive (but the sparse part was missing and the mapping embedded in the file). Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-08-16Fix libtar treatment of link targets that fill the header fieldDavid Oberhollenzer
The tar header has a 100 byte field for symlink and hard link targets. If the target is longer than 100 bytes, an extension header has to be used. However, it is perfectly valid to fill all 100 bytes to the brim without adding a null terminator. In case of a symlink, this can result in garbage link targets, while for hard links it results in an immediate error since the target cannot be resolved later on. This commit attempts to fix the problem by replacing the strdup of the link target with an strndup that copies at most the size of the target header field. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-05-30Cleanup: sqfs2tar: break up and simplify the repacking codeDavid Oberhollenzer
- Move the xattr extraction and repacking to xattr.c - Don't on-the-fly delete the tar xattr list, use the function from libtar.a - Split minor tasks into static helper functions - creating a libtar xattr struct from libsqfs xattr data - finding a hard link entry from current path and inode number Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-05-18libtar: fix size computation of PAX line lengthDavid Oberhollenzer
This commit attempts to fix the following two problems: - The number of digits computation returning an off-by-one result if the number is 10, or the resulting digit string starts with "10". This results in one-too-many padding bytes, corrupting the rest of the archive since the headers now don't start at multiples of 512 anymore. - Adding the line length prefix affects the line length (duh). If it grows far enough to require more digits, the result is a similar problem. This is a converging series that we need to compute the limit of. Unit tests for this still need to be added. Or maybe I can convince a bored undergrad student to provide an induction proof. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-04-22Skip PAX global headersDavid Oberhollenzer
Tar archives can contain set two kinds of PAX headers: - local headers that modify the attributes of the next file - global headers that set defaults for all files The later is used "... not widely used", according to tar(5) and has been deliberately not implemented. Some programs (e.g. git-archive) *do* generate them (in the case of git, it stores the commit hash). This commit adds a code path that skips a PAX global header entirely and resumes tar parsing, instead of erroneusly reporting it as an entry. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-04-17Remove some configure time sizeof checksDavid Oberhollenzer
In libtar, the sizeof time_t checked when trying to store a time value. It is pointless using the preprocessor here, as we can simply do an if (sizeof(time_t) < ...) check and the compiler will take care optimizing away one or the other branch. After changing the libtar check and the corresponding unit tests, the sizeof check can be removed from configure.ac, along with other unused sizeof checks. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-04-17Cleanup: split read_header.c in libtar.aDavid Oberhollenzer
Simply moving the pax header decoding to a separate file and splitting out the common helper functions should be a good start. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-02-28Cleanup pax header parser a littleDavid Oberhollenzer
This commit tries to untangle the logic of parsing and sanitizing the pax header length field and the associated bounds checks. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-23Add libtar.a function to create hardlink recordsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-22Add hard link support to gensquashfs and tar2sqfsDavid Oberhollenzer
In libtar, set a special flag if the header is actually a hard link. In tar2sqfs, create a hard link node and skip the rest for hard links. Also refues to set the root attributes from a hard link, it may refere to a node that we have missed earlier, there is nothing else that we can do here. In fstree_from_file, add a "link" command for adding hard links. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-15Fix tar GNU sparse header parsingDavid Oberhollenzer
Extract the filename correctly. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-13Better support for reading/writing non-ASCII xattr values from/to tarDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-12-13Make the PAX header parser more strictDavid Oberhollenzer
Actually parse the length field and insist it matches the line length, then use the length field to advance to the next line. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-25Cleanup: remove what is left of libutilDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-23Move some unix header inclusions to compat.hDavid Oberhollenzer
In most cases, including unistd.h and fcntl.h was a left over anyway. In the cases where it was not, move it to compat.h. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-06Fix: use size_t printf macro in tar PAX header writerDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-06Remove raw file descriptors from tar read pathDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-11-06Remove raw file descriptors from unpack write pathsDavid Oberhollenzer
Instead, use stdio FILE pointers. On POSIX systems, use fileno to get the file descriptor and hopefully create sparase files. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-10-07Cleanup: move libutil related headers to "util" sub directoryDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-10-07Cleanup: move write_data to libtarDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-10-07Cleanup: move read_data function to libtarDavid Oberhollenzer
Its the only user. The other code doesn't touch raw file descriptors anymore. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-10-07Cleanup: Move padd_file function to libtarDavid Oberhollenzer
It's only ever used for padding tarballs. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Add a header for platform compatibillity fluffDavid Oberhollenzer
- We don't have "endian.h" everywhere. On some BSDs its in sys and on some BSDs the macros have different names. - We definitely don't have sysmacros.h on non-Unix-like systems. - Likewise for sys/types.h, sys/stat.h and their contents. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-27Cleanup: replace fixed with data types with typedefsDavid Oberhollenzer
This is a fully automated search and replace, i.e. I ran this: git grep -l uint8_t | xargs sed -i 's/uint8_t/sqfs_u8/g' git grep -l uint16_t | xargs sed -i 's/uint16_t/sqfs_u16/g' git grep -l uint32_t | xargs sed -i 's/uint32_t/sqfs_u32/g' git grep -l uint64_t | xargs sed -i 's/uint64_t/sqfs_u64/g' git grep -l int8_t | xargs sed -i 's/int8_t/sqfs_s8/g' git grep -l int16_t | xargs sed -i 's/int16_t/sqfs_s16/g' git grep -l int32_t | xargs sed -i 's/int32_t/sqfs_s32/g' git grep -l int64_t | xargs sed -i 's/int64_t/sqfs_s64/g' and than added the appropriate definitions to sqfs/predef.h The whole point being better compatibillity with platforms that may not have an stdint.h with the propper definitions. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-25Remove condensed sparse file handling from libsquashfsDavid Oberhollenzer
This only exists for tar2sqfs. Move the sparse file map to libtar and add the ability to do this into the stind sqfs_file_t abstraction, so it acts like a normal file but internally stitches the data together from the sparse implementation. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2019-09-15Move sparse_map_t to libsquashfs headers, rename it to sqfs_sparse_map_tDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>