| Age | Commit message (Collapse) | Author |
|---|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit updates the text issued by print_version() to reflect in
some way that the software contains contributions from co-authors.
The original text was based on the sterotypical --version output of
GNU coreutils programs. It may have to be rewritten eventually.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Data blocks need to be deduplicated before attempting to write a fragment.
In the current attempt if the data blocks are found to be duplicates but
the fragment isn't, the flushed fragments are purged as well, possibly
damaging other files.
Also, when the deduplication happens, the HAS_FRAGMENT flag needs to be
set, otherwise the deduplication code thinks that there is one more block
than there actually is.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Since it is actually completely independend of libsqfs and only works
on file_info_t lists, it can be safely moved over to libfstree and
the data writer becomes less cluttered as a result.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
For merging, the use of a pointer to a pointer can simplify linked list
operations
For sorting, find the half-way point of the list in a single iteration
over the list
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Despite having a flag for that now, they still need to be initialized
because they are written straight to disk.
Fixes: d4d1854aaed867d28ebfc97afb3518254ab6fd4b
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
A file is a complete duplicate if:
- It has no blocks, only a single fragment and that is a duplicate
- It has blocks but no fragment and the blocks are duplicate
- It has blocks and a fragment and both are duplicate
The previous version only counted the last one.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
If an entire file is eliminated, we need to reset the "used_bytes" counter,
otherwise, ALL the table positions are way off.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
We didn't allocate the ID table, so we don't need to free() it when
reading from disk fails.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Cut & paste misshap after mergining with fragment reader: If there are
no fragments, data_reader_create should return the data reader, not 0!
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Simplifies some task if we can just add a flag that a file has a framgent
or that it has already been detected as a duplicate.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The strategy is as follows:
- At the beginning of every file, remember the current position
- Once a file is done scan the list of existing files for the following:
- Look for an existing file that has a block with the same size and
checksum as the first non-sparse block of the current file
- After that, every block in the current file has to match in size and
checksum the ones in the file that we found, from that point onward
- sparse blocks in either file are skipped
- If we found a match, we update the current file to point to the first
matching block and rewind the squashfs image to remove the newly written
data
This strategy should in theory be able to find an existing file where the
on-disk data *contains* the on-disk data of the current file.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The strategy is simple:
- The data writer function that write data/fragment blocks get
access to the list files.
- When writing a fragment, we look for an already written file that has
a fragment with the same size and checksum.
- If we find one, we throw away the fragment and reuse the existing one.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
After the table read unification, there wasn't much left of the fragment
reader and the remains could easily be moved over to the data reader.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Since they are read directly into memory, blindly allocating the size
from the tar ball is probably a bad idea.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Make sure range is checked when reading a block and that the check is
made correctly. Also make the block log check a little more strict.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The names generated by fstree are always in the correct format, but this
is still needed for the sake of documentation and to guide static
analysis tools.
Should canonicalize_name fail on an fstree generated name, something is
serverly broken.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Should the main process exit (e.g. the user presses CTRL+C) or explcitily
sends it a signal, the desired behaviour is for the children to stop
unpacking and exit.
This commit adds a line to configure the kernel to send SIGKILL to the
children if their parent dies.
The same option also exists on various BSDs (not all of them) but with a
different name which has to be checked for and adjusted should the program
be required to run on one of those.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of doing DFS on the fly in gensquashfs, churn out a linked list
of all files in an archive.
Future improvements in packing strategies can go into this file.
This can also be usefull for other purposes in the future, such as file
deduplication or as a work queue for the unpacker.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
If some library isn't present, the variable $(FOO_LIBS) simply evaluates
to empty string, so it can be simply be added to LDADD without any checks.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The utility programs pretty much all have the same structure of including
one central header per C file that includes all required library headers,
so we can simply include config.h which we need for large file support
from there and remove it from the C files.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit attempts to make the generic table writer more readable.
A few changes are made, including heap allocation of the block list.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit fixes a bug in the fragment table reader where the reader
tries to read data into an out of bounds location due to an oversight
in size calculation.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
In most cases, we know exactely where the data that we want to read is
on disk, so instead of using read() on the squashfs (or lseek + read),
the code can in many places be cleaned up to use the pread wrapper
read_data_at instead.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
struct stat uses time_t to store time values. On some 32 bit systems,
this may be a 32 bit integer.
This patch adds a broken-out 64 bit time value to tar_header_decoded_t
and makes sure to clamp the value to +/- (2^32 - 1) if required when
writing it back to a struct stat.
Reported-by: Matt Turner <mattst88@gmail.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit removes all the code for parsing and processing atime/ctime
and values and related test code.
Caring about those is kind of pointless because squashfs can only store
mtime in inodes. The only relevant place is when generating a struct
stat from a squashfs inode or an fstree node.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Would cause the build to fail if zstd was enable and lz4 was disabled.
Signed-off-by: Matt Turner <mattst88@gmail.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Without it you're left guessing or using a debugger to figure out what's
wrong.
Signed-off-by: Matt Turner <mattst88@gmail.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Requires that config.h be included before other headers, since the macro
_FILE_OFFSET_BITS changes the definitions of things like 'struct stat'.
I chose to simply include it at the top of every C file and at
immediately after the double-inclusion guards of every header.
Signed-off-by: Matt Turner <mattst88@gmail.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Until now, the tar checksum verification simply copied the header,
recomputed the checksum and compared it byte-for-byte with the
original.
However, not all implementations store the checksum the same way. For
instance, git-archive generates tar balls that use the same format as
for other octal numbers.
This patch makes the checksum verification more lenient by parsing the
checksum from the header and comparing it with the computed value
instead of copying the entire block and insisting on byte-for-byte
equivalence.
The result is better interoperabillity with existing tools and perhaps
slightly faster processing since the block doesn't have to be copied.
Reported-by: Matt Turner <mattst88@gmail.com>
Suggested-by: René Scharfe <l.s.r@web.de>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
