| Age | Commit message (Collapse) | Author |
|---|
|
There was some code duplication for extracting the sparse entry
fields from the start record and the subsequent extended record.
This commit introduces a data structure for both and unifies
the parsing code paths.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- Use is_memory_zero from libutil
- Move checksum update function to tar writer code
- Move checksum verify function to tar reader code
- Only export the function to compute the checksum
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
We have an "append_sparse" function in libio, with a fallback,
so use that.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Move it to a separate libxfrm library, where it can be independently
tested as well. The bulk of the new code is also mainly test cases
for the compressors.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Implement grab/drop functions to increase/decrease reference count
and destroy the object if the count drops to 0.
Make sure that all objects that maintain internal references actually
grab that reference, duplicate it in the copy function, drop it in
the destroy handler.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
mksquashfs generates extended inodes if a directory contains 256
entries. libsquashfs so far only generated extended inodes if there
is no other way to encode it. Mimic the behaviour of mksquashfs by
adding a threshold.
For this to work, the "sqfs_inode_set_xattr_index" function has to
be changed to not immediately try to demote inodes to basic types.
The fstree serialization is modified to do that itself if the index
is 0xFFFFFFFF and the target is not a directory inode.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Propperly set the parent eof flag and not a local one that isn't
accessed at all.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The "from dir" and from "from file" code, as well as the "sort file"
code is specific to gensquashfs, so move them there and the test
cases as well.
The medium term idea is to reduce libfstree to a stub, merge it into
the generic writer and ultimately hoist that into libsquashfs.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
On Linux or BSD distributions we have a native version installed
via package manager. On Windows, we can just build it from source
like the other libraries.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
When (during fragment deduplication) a fragment block is read back
from disk and unpacked, it can happen that it is _exactly_ the
given block size. The bounds check did '>=' instead of '>' and
failed in that case with a "data corruption" error.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Initialize the output compressor pointer to NULL, so if the function
fails, the value is propperly initialized to a NULL pointer instead
of relying on the function user to initialize it.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Similar to the hex blob decoder, we need this once for tar and
once for the filemap xattr parser.
Simply add a single, central implementation to libutil, with a
simple unit test, and then use it in both libtar and gensquashfs.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Since we need it twice (once for tar, once for the filemap xattr
parser), add a single, central implementation to libutil, add a
unit test for that implementation and then use it in both libtar
and gensquashfs.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Not all CPUs may be available for the current process. Some CPUs
may be offline, others may not be included in the process affinity
mask. In such cases too many threads will be created, which will
then compete unnecessarily for CPU time.
Use sched_getaffinity() to determine the correct number of threads
to create.
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Slightly modify the byte-for-byte comparison function to compare an
arbitrary range in a file and move it to libutil. Instead of calling
it for each block in the block writer, simply let it check an entire
range in the block writer and compute the range position/size of the
reference ahead, before looking for potential matches.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of open coding it, use the array_t type from libutil.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The zlib library uses a typedef'd uInt type for ranges which may be
smaller than size_t.
The complicated clamping to the uInt range is technically not needed,
as the buffer size can grow at most to BUFSZ anyway, and it also
confuses coverity.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of printing error messages to stderr, simply return an error
number instead, that the caller then prints out using sqfs_perror.
The underlying rbtree already uses sqfs error numbers, so little
change is needed here.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Test against various invariants:
- Every non-root node must have a name
- The root node muts not have a name
- The name must not be ".." or "."
- The name must not contain '/'
- The loop that chases parent pointers must terminate, i.e. we must
never reach the starting state again (link loop).
Furthermore, make sure the sum of all path components plus separators
does not overflow.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
After trying removing many instances of `struct stat`, there is only
one user of inode_stat left, so move the function there.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Some of the on-disk format internals are moved to a separate header
and some of the stuff from internal.h is moved to that format header.
C++ guards are added in addtion.
Everything PAX related is moved to pax_header.c, some internal
functions are marked as static.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Move all the libutil stuff from the toplevel include/ to a util/
sub directory and fix up the includes that make use of them.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The squashfs on-disk format uses 32 bit link counters, but the fstree
used 16 bit ones. Because the link count also includes child nodes,
this artificially limited the number of entries in a directory to ~64k
files.
This patch removes the limit by switching libfstree to 32 bit counters.
Reported-by: Marvin Renich <mrvn@renich.org>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
If fstree_mknode fails, because the parent link count would overflow,
the function fails and cleans up behind it. The problem arises because
the function does this check *after* inserting the node in the parent
node, so it is later free'd again, when destroying the rest of the
tree.
This patch moves the insertion after the check to mitigate the problem.
Reported-by: Marvin Renich <mrvn@renich.org>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Split out several repated patterns into helper functions and move the
rest of the code back into dir_reader.c
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Add a simple directory state object to the meta data reader and use
that to iterate directory entries. The code for reading the directory
listing is movde to readdir.c
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
When asking the meta data reader for its current position and
we *just* read to the end of a block, report the start of the
next block as the current location.
Otherwise, trying to *seek* to the resulting position immediately
after reporting throws an out-of-bounds error.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
On GNU/Linux, *BSD or MacOS we can simply use the system default
library. The copy was primarily only there for the Windows build.
The build script for Windows has now been adapted to download and
compile a shared library from a tarball.
This removes a huge chunk of code from the git tree as well as
the release tarballs. Additionally it gets rid of iffy things like
removing the Zlib copyright/version strings, so the libsquashfs DLL
doesn't export it.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Two flags are added to the dir reader API, one for the create function
that the dir reader should report those entries and one to the open
function to suppress that if it was enabled.
To implement the feature, a mapping of visited directory inodes is
maintained internally, that mapps inode numbers to inode references.
When opening a directory, state is maintained to generate the fake
entries for '.' and '..'. Since all the other functions are based on
the open/read/rewind API, no alterations need to be made. The tree
scan function is modified, to use the suppress flag, so it does not
accidentally catch those entries.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
