| Age | Commit message (Collapse) | Author |
|---|
|
This commit adds a reference count functionality to the string table
implementation and uses this functionality in the fstree code to
count how often each key and value is referenced by the deduplicated
Xattr blocks. This is needed to support deduplication through
out-of-band storage of xattrs.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
If read_retry fails to read the expected amount of data (EOF or otherwise),
it is almost always an error.
This commit renames read_retry to read_data and moves error handling
into the function, making a lot of error handling code redundant.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
If write_retry fails to write everything, it is *always* an error.
This commit renames write_retry to write_data and moves error handling
into the function, making a lot of error handling code redundant.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Digging around in kernel internals and mksquashfs reveals that it is
actually a buffer offset into the raw directory buffer. The error
hasn't been noted until now because of the bug fixed by a5428e0.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The added flags allow controlling the following on a per file level:
- forcing a file to be written uncompressed
- forcing a file to not have a fragment, i.e. the last truncated block
actually being written as a block
- padding a file to be alligned to device block size
The flags are not yet exposed to anything user controllable (such as
command line flags).
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of writing meta data blocks directly to disk, the writer can
now alternatively keep the blocks in memory until explicitly told to
write to disk.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The directory listing stores a signed difference of the inode number.
Actually treating it as signed saves emitting extra headers if hard
links or file deduplication are finally implemented.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Some experiments seem to indicate that the various GNU extensions are
more widely supported than their POSIX equivalents[1]. Possibly because
they are easier to implement and possibly because of the wide spread
use of GNU tar.
This commit replaces the PAX writer in the write_tar_header implementation
with a GNU extension based writer.
The writer is also cleaned up by removing all global state. The record
counter is moved outside into the tar2sqfs program and passed in as
function argument.
[1] https://dev.gentoo.org/~mgorny/articles/portability-of-tar-features.html
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit attempts to split some of the monolitic tar parsing code up
into multiple functions in seperate files. Also, some code duplication
(like reading a record into memory which was implemented twice) is
removed.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit broadly does the following things:
- Rename and move the sparse mapping structure to libutil
- Add a function to the data writer for writing condensed versions
of sparse files, given the mapping.
- This shares code with the already existing function for regular
files. The shared code is moved to a common helper function.
- Add support to tar2sqfs for repacking sparse files.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit adds support for packing sparse files into squashfs images
as follows:
- In the data writer: simply detect zero blocks and write a zero to the
block size field and don't emit any data. Record the number of bytes
saved this way. For fragments, set the fragment offset to invalid.
- In the inode writer: write out the number of bytes saved for sparse
files. If there should be a fragment but there is none, append a block
count of 0.
- In the data reader: if the block size is 0, read nothing from disk and
emit an empty block. Do the same if the fragment is missing.
- In the inode reader: restore the number of bytes saved for sparse files.
The sparse files can be packed and unpacked, but the unpacking will not
create sparse files for now.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of decomposing a default string in gensquashfs option processing,
move that to fstree_init instead and pass the option string directly to
fstree_init.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit removes handling of compressor names from gensquashfs. Instead,
functions are added to libcompress to obtain name from ID, ID from name
and to print out defaults.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit removes the packdir/packfile based directory setup magic
from fstree_from_file and moves it to gensquashfs. Over there, the
common parts are deduplicated.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Make tree node list sort and recursive variant available and independend
of the fstree_t.
This is considered cleaner, since the fstree_t actually isn't needed for
any of this and we can just call the recusvie sort on the root instead,
and we can use the sort implementation directly for things like the
upcoming unit test.
Also this commit splits up the merge/sort implementation into a seperate
split and merge functions to make the code somewhat more readable.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- Replace scan and memmove with simple automaton
- Convert forward to back slashes, lots of file systems treat them
equivalenty, this saves us a few more checks down the road
- Remove './' path components. We can saveley remove them in a string
processing step instead of throwing an error. Also they actually appear
often in tar balls, possibly not under user control.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Replace default_$FIELD with a struct stat. Merge the rest of
add_node/add_file into mknode and add by path+stat.
Expose the mknode function and replace all the duplicated node pointer
arithmetic magic with a call to mknode.
Plus a generic cleanup by utlizing new util functions.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Always try to gather all compressor libraries available, but only
complain about missing a one if it has been *explicitly* selected.
If a compressor has been explicityl disabled, we can still turn it
off after checking.
Also, rework gensquashfs to set the default compressor based on
what's available.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Don't need that, we already know the index when we write add
the fragment.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The idea is to move various higher level helper functions there.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of allocating inode numbers as we go, generate and populat an
inode table from the fstree ahead of time. This makes processing nodes
a little bit simpler and we will need that table anyway for NFS export
support later on.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit replaces the chdir to the input directory with pushd/popd
when building the fstree and again when packing files.
This simplifies handling of other file paths given on the command line
that have to be accessed and are relative to the original working
directories.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
If packdir and packfile are both specified, use packdir as
alternate root.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
