| Age | Commit message (Collapse) | Author |
|---|
|
Since the user has to call istream_get_buffered_data afterwards anyway,
we can do the precache lazily.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This properly maps to all of our use cases and makes istream_precache
obsolete.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
As soon as we no longer have any data to read, unlock/drop the
parent iterator_t object. Also, make sure we get the buffer
count right, not all data might have been consumed yet when
precache is called. Remove the precache/read loop in the non-sparse
case, we have already established that there is data available. If
it is insufficient, the user will simply call precache again once
it's used up, which istream_get_buffered_data forwards to a precache
call in the underlying stream.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead, go through helper functions, which in a next step can be
moved inside the implementation.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The structure and functions are renamed to sqfs_xattr_* instead,
an additional helper is added to accept an encoded xattr.
Documentation and unit test are added as well.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead, make the buffer const, let the user adjust the pointer and
size. The offset can then be inferred in precache.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The end goal is to remove direct buffer access from the istream_t
interfaces and make that opaque. For the tar implementation, this
already safes us needless buffer copying, as we essentially allow
the user to read-through from the underlying stream.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The existing istream_t wrapper is mered into this one as well, we
can open the files via the iterators open_file_ro function. Unit
tests and tar2sqfs are modified accordingly.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Having it all in one buffer allows us the re-use the "generat GNU
record" function.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
By cobbling together the xattr lines manually in libtar, the
need (and thus the function itself) are removed.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Generate a simple tarball and compare it with a reference.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
In the istream implementation, automatically skip the padding when we
reach end-of-file. Also skip file AND padding when we destroy the
object. Replace the remaining instances with a simple istream_skip
instead and remove the wrapper from libtar.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The tar_istream_t reads the data from a tar file, having been given
the header, and synthesizes zero bytes for sparse regions.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Use read_number in the places that remain.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
There was some code duplication for extracting the sparse entry
fields from the start record and the subsequent extended record.
This commit introduces a data structure for both and unifies
the parsing code paths.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- Use is_memory_zero from libutil
- Move checksum update function to tar writer code
- Move checksum verify function to tar reader code
- Only export the function to compute the checksum
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
We have an "append_sparse" function in libio, with a fallback,
so use that.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Similar to the hex blob decoder, we need this once for tar and
once for the filemap xattr parser.
Simply add a single, central implementation to libutil, with a
simple unit test, and then use it in both libtar and gensquashfs.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Since we need it twice (once for tar, once for the filemap xattr
parser), add a single, central implementation to libutil, add a
unit test for that implementation and then use it in both libtar
and gensquashfs.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Some of the on-disk format internals are moved to a separate header
and some of the stuff from internal.h is moved to that format header.
C++ guards are added in addtion.
Everything PAX related is moved to pax_header.c, some internal
functions are marked as static.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The idea was originally to use struct stat in the libfstree code, so
we can simply hose data read from a directory into the fstree_t. The
struct was then also used with libtar, for simpler interoperation,
but it turned out to introduce a lot of platform quirks and causes
more trouble than it's worth.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of having a long if-else-if chain, replace the PAX header
field parsing with a table driven approach.
Altough it is more code, it is hopefully more readable, maintainable,
extensible and it dedupliates some of the value parsing code.
The GNU.sparse parsing is left as is, because it requires
maintaining state.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Split the key/value pairs right in the header and terminate the key
name. This way, some of the magic numbers can be eliminated.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
When processing files > 4G, using "%o" truncates the result and the
tarball is not readable. This should have been discovered when
auto-patching the printf format specifiers, but a cast was added
instead and the issue was overlooked.
This commit replaces the down-cast and printf format specifiers.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit rewrites the libtar write paths to use libfstream insead of
a FILE pointer. Also, the libcommon file extraction function is remodeled
to use libfstream.
In accordance, rdsquashfs, sqfs2tar and sqfsdiff have some minor
adjustments made to work with the ported libtar and libcommon.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- Make sure the file actually has that many records before trying
to read one and fail if not.
- Use the helper macros for size_t overflow checking instead of
assuming size_t == uint64_t.
- Impose a "reasonable" upper bound on the number of data segments
and insist that there is at least one entry.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Contrary to previous claims, support for the GNU tar sparse format 1.0
was missing entirely (the newest of their 3 different sparse mapping
formats). This oversight wasn't caught, because the unit test was
compiling the wrong source file and tar2sqfs had no problem processing
the test file because it is still a valid POSIX-ish tar archive (but
the sparse part was missing and the mapping embedded in the file).
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The tar header has a 100 byte field for symlink and hard link targets.
If the target is longer than 100 bytes, an extension header has to be
used.
However, it is perfectly valid to fill all 100 bytes to the brim
without adding a null terminator. In case of a symlink, this can
result in garbage link targets, while for hard links it results in
an immediate error since the target cannot be resolved later on.
This commit attempts to fix the problem by replacing the strdup of
the link target with an strndup that copies at most the size of the
target header field.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- Move the xattr extraction and repacking to xattr.c
- Don't on-the-fly delete the tar xattr list, use the function
from libtar.a
- Split minor tasks into static helper functions
- creating a libtar xattr struct from libsqfs xattr data
- finding a hard link entry from current path and inode number
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit attempts to fix the following two problems:
- The number of digits computation returning an off-by-one result
if the number is 10, or the resulting digit string starts
with "10". This results in one-too-many padding bytes, corrupting
the rest of the archive since the headers now don't start at
multiples of 512 anymore.
- Adding the line length prefix affects the line length (duh). If it
grows far enough to require more digits, the result is a similar
problem. This is a converging series that we need to compute the
limit of.
Unit tests for this still need to be added. Or maybe I can convince a
bored undergrad student to provide an induction proof.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Tar archives can contain set two kinds of PAX headers:
- local headers that modify the attributes of the next file
- global headers that set defaults for all files
The later is used "... not widely used", according to tar(5)
and has been deliberately not implemented.
Some programs (e.g. git-archive) *do* generate them (in the case
of git, it stores the commit hash).
This commit adds a code path that skips a PAX global header entirely
and resumes tar parsing, instead of erroneusly reporting it as an
entry.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
In libtar, the sizeof time_t checked when trying to store a time value.
It is pointless using the preprocessor here, as we can simply do an
if (sizeof(time_t) < ...) check and the compiler will take care
optimizing away one or the other branch.
After changing the libtar check and the corresponding unit tests, the
sizeof check can be removed from configure.ac, along with other unused
sizeof checks.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Simply moving the pax header decoding to a separate file and splitting
out the common helper functions should be a good start.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit tries to untangle the logic of parsing and sanitizing the
pax header length field and the associated bounds checks.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
In libtar, set a special flag if the header is actually a hard link.
In tar2sqfs, create a hard link node and skip the rest for hard links.
Also refues to set the root attributes from a hard link, it may refere
to a node that we have missed earlier, there is nothing else that we
can do here.
In fstree_from_file, add a "link" command for adding hard links.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Extract the filename correctly.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Actually parse the length field and insist it matches the line length,
then use the length field to advance to the next line.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
