| Age | Commit message (Collapse) | Author |
|---|
|
By storing the blocks in a tree, the de-duplication can lookup
existing blocks in logartihmic instead of linear time.
The linked list is still maintained, because we need to iterate
over the blocks in creation order during serialization.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
If we use the rb-tree in libsquashfs objects, we need to be able
top copy an entire tree as part of the object.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
|
|
Since the canonicalize_name function only fails if the path
contains ".." and the one we are constructing from the scanned
fstree (built using canonicalized names), it should NEVER fail.
However, coverity does get concerned, because we are checking the
return value elesewhere. So do what we do at other, similar locations
and add an assert().
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The base path is passed to the fstree_from_file function and in turn
to the individual callbacks.
The line parsing function is modified to allow '*' as mode, uid and gid
for specifically marked callbacks.
A glob callback is added that internally uses the fstree_from_dir scanning
functions in combination with a filter callback.
Directory scanning flags are parsed from the extra arguments before
interpreting it as a path fragment.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
So we can scan a sub-directory within a the base directory without
having to do string operations first.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit mainly serves the static analysis tooling.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This function creates a block processor from a structure describing it.
A stub implementation for the old sqfs_block_processor_create is added
that simply sets up such a struct and forwards the call.
The current version of the description struct only contains the exact
same parameters and a size field at the beginning.
This approach is supposed to make extending the range of parameters
easier without breaking ABI compatibillity.
Currently already planned are:
- Adding a sqfs_file_t pointer to double-check when deduplicating
fragments.
- When the scanning code reaches a usable state, add the abillity
to pass scanned fragment data, so the block processor can be used
for appending to an existing image.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Keeping a list of fragments stored away in the current fragment block
and consolidating them in the thread pool takes them out of circulation.
If we have a lot of tiny fragments, this can lead to a situation where
all the limit is reached, but we cannot do anything, because we are
waiting for a block to complete, but they are all attached to the
current fragment block and the queue is empty.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
It looks like the last commit missed a couple more occurences
where '\' was treated incorrectly.
Fixes were still needed in sqfs_dir_reader_find_by_path and
sqfs_dir_reader_get_full_hierarchy.
This path is used in extras/browse.c.
|
|
All paths were canonicalized internally, which includes filtering
sequences of slashes and converting backslashes to slashes.
Furthermore, when unpacking files, filenames are sanity checked
and rejected if they contain forward OR backward slashes.
This is a problem on Unix-like systems, where files containing
backslashes are a legitimate use case (*cough* SystemD *cough*).
This patch removes the backslash conversion from the canonicalization
and modifies the sanity check to reject backslashes only on Windows.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- Instead of using the fstree root, let the caller specify it.
- Add a flag to prevent recursion into sub directories.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of comparing (compresed, disk-size, checksum) tuples to find
block matches, do an exact, byte-for-byte comparison of the data
stored on disk to avoid the possibility of a spurious colision.
Since this is the desired behaviour, make it the default, optionally
overrideable through a flag.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of dereferencing the NULL pointer and crashing.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The sqsf_writer is moved to a separate header, as well as the compressor
related stuff. The statistics function is moved into the writer code, as
this is the only place that actually uses it. The writer code itself is
split up into a hand full of file in their own subdirectory.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Modify the man pages and help texts to mention compression support, make
the help option display a list of supported compressors.
Also clarify the squashfs compressor list in the help text to state that
the list shows block compressors supported for SquashFS compression.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of immediately seeking forward, record the sparse bytes "written"
and then do the seek the next time we write data, or do an ftruncate if
we do a sync.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Streaming compression was added fairly recently to zstd (and then
the API was changed a few times).
Rather than rely on libzstd versioning macros, this commit adds an
m4 script to test at configure time if the enums/functions we need
are available by trying to compile a small sample.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
If tar2sqfs and sqfs2tar want to use comperssion via libfstream, they
need to link against the compressor libraries.
Also, libfstream may need the apropriate cflags to access the headers
on some platforms.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit rewrites the libtar write paths to use libfstream insead of
a FILE pointer. Also, the libcommon file extraction function is remodeled
to use libfstream.
In accordance, rdsquashfs, sqfs2tar and sqfsdiff have some minor
adjustments made to work with the ported libtar and libcommon.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- Make sure the file actually has that many records before trying
to read one and fail if not.
- Use the helper macros for size_t overflow checking instead of
assuming size_t == uint64_t.
- Impose a "reasonable" upper bound on the number of data segments
and insist that there is at least one entry.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Contrary to previous claims, support for the GNU tar sparse format 1.0
was missing entirely (the newest of their 3 different sparse mapping
formats). This oversight wasn't caught, because the unit test was
compiling the wrong source file and tar2sqfs had no problem processing
the test file because it is still a valid POSIX-ish tar archive (but
the sparse part was missing and the mapping embedded in the file).
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The function sqfs_tree_node_get_path is used in several places within
rdsquashfs to produce a path for a tree node, either when describing
the file system, or when unpacking it. Unpacking can be done on
sub-trees as well as the entire tree, in which case the root of the
sub-tree has its parent pointer removed, so the full path terminates
at the new root.
This works with directories, since they receive special case handling
anyway, but fails if the sub-tree to unpack is only a single file
because the sqfs_tree_node_get_path function assumes that we are at the
tree root and returns "/" as a path, which gets normalized to "".
This commit adds a workaround to the function to simply use the nodes
name (if available) in that case instead.
The describe case in rdsquashfs is unaffacted, since it always starts
at the root. Likewise, the sqfs2tar case should also be unaffacted,
since it already employs special case handling for the [sub] tree root
node.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The tar header has a 100 byte field for symlink and hard link targets.
If the target is longer than 100 bytes, an extension header has to be
used.
However, it is perfectly valid to fill all 100 bytes to the brim
without adding a null terminator. In case of a symlink, this can
result in garbage link targets, while for hard links it results in
an immediate error since the target cannot be resolved later on.
This commit attempts to fix the problem by replacing the strdup of
the link target with an strndup that copies at most the size of the
target header field.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit fixes a bug where the block processor state machine would
not add the "last block" flag if there is only one not entirely filled
block and the "don't fragment" flag is set. If the flag isn't set, the
inode start block position is not updated and points to the beginning
of the image instead.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The source code of a modified liblz4 and zlib are included with the
option to compile them into libsquashfs if they are not available on
the system.
So far, the source code was included directly in the compressor sub
directory within libsqsuashfs. This commit moves the libraries out
into the lib directory.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
