| Age | Commit message (Collapse) | Author |
|---|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of having 3 different functions for sorting the tree, numbering
the nodes and generating a file list, that all have to be used in the
right order, this commit merges them into a single "fstree_post_process"
function.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Same rational as for the dir-scanner code: It's actually the only user and
it is going to get a lot closer integerated with libsquashfs.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
It's actually the only user and the dir-scanner xattr code is going
to get a lot closer integerated with libsquashfs.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- We don't have "endian.h" everywhere. On some BSDs its in sys and
on some BSDs the macros have different names.
- We definitely don't have sysmacros.h on non-Unix-like systems.
- Likewise for sys/types.h, sys/stat.h and their contents.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This is a fully automated search and replace, i.e. I ran this:
git grep -l uint8_t | xargs sed -i 's/uint8_t/sqfs_u8/g'
git grep -l uint16_t | xargs sed -i 's/uint16_t/sqfs_u16/g'
git grep -l uint32_t | xargs sed -i 's/uint32_t/sqfs_u32/g'
git grep -l uint64_t | xargs sed -i 's/uint64_t/sqfs_u64/g'
git grep -l int8_t | xargs sed -i 's/int8_t/sqfs_s8/g'
git grep -l int16_t | xargs sed -i 's/int16_t/sqfs_s16/g'
git grep -l int32_t | xargs sed -i 's/int32_t/sqfs_s32/g'
git grep -l int64_t | xargs sed -i 's/int64_t/sqfs_s64/g'
and than added the appropriate definitions to sqfs/predef.h
The whole point being better compatibillity with platforms that may
not have an stdint.h with the propper definitions.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Now that dir_info_t and file_info_t have reasonably small, use them in
tree_node_t directly instead of doing pointer arithmetic magic on the
payload area.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Changes are minor, but needs some temporary hacks in fstree again.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- file list is no longer needed for statistics
- the size field in the directory info structure is no longer in use
- sqfs2tar, rdsquashfs and sqfsdiff no longer depend on libfstree.a
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Remove all the library functions that no longer have any users.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Parallel unpacking didn't really improve the speed that much. Actually
sorting the files for optimized unpack order improved speed much more
than the parallel unpacker.
Furthermore, the fork based parallel unpacker was actually pretty messy
to begin with.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This moves a lot of the stuff that is done manually in the tree
deserializer to a generic helper in libsquashfs.
Due to how the fstree is implemented, as a work around, the inode needs
to be temporarily stored in the tree node, but some of the directory
details could be removed.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
As a side effect, this requires the data writer to keep track of
statistics.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit does the following:
- canonicalize_name is moved to libfstree
- source_date_epoch is only used inside libfstree, so it's also moved
over and can later be completely internalized
- print_version is moved over to sqfshelper. Mainly so it doesn't end
up in libsquashfs.so for no sane reason.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This change removes the need for passing a list of files around for
deduplication. Also the deduplication code no longer needs to worry
about order, since the file being deduplicated is only added after
deduplication is done.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit moves the file unpacking order & job scheduling to a libfstree
function. The ordering is improved by making sure fragment blocks are not
extracted more than once and files with data blocks are extracted in order.
This way, serial unpacking of a 2GiB Debian live image could be reduced
from ~5' on my test machine to ~3.5', whereas parallel unpacking stays
roughly the same (~3' for -j 4).
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Since it is actually completely independend of libsqfs and only works
on file_info_t lists, it can be safely moved over to libfstree and
the data writer becomes less cluttered as a result.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Simplifies some task if we can just add a flag that a file has a framgent
or that it has already been detected as a duplicate.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of doing DFS on the fly in gensquashfs, churn out a linked list
of all files in an archive.
Future improvements in packing strategies can go into this file.
This can also be usefull for other purposes in the future, such as file
deduplication or as a work queue for the unpacker.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Requires that config.h be included before other headers, since the macro
_FILE_OFFSET_BITS changes the definitions of things like 'struct stat'.
I chose to simply include it at the top of every C file and at
immediately after the double-inclusion guards of every header.
Signed-off-by: Matt Turner <mattst88@gmail.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
First of all, this commit adds a mod_time field to a tree node. When
creating the tree node, the field is set from the struct stat. When
scanning a directory, the time stamps from the input are used if set.
Second, the libsqfs code that reads inodes is modified to store the
mod_time from the inode in the fstree node and to write the tree node
into a generated inode.
Finally, tar2sqfs is modified to optionally keep the timestamps from
the tar archive instead of setting defaults. gensquashfs is similarly
modified to keep the input timestamps if specified.
The result is as follows:
- sqfs2tar will always carry the timestamps from the squashfs over
to the tar ball.
- tar2sqfs will set defaults, unless explicitly asked to preserve
the mtime from the tar ball.
- gensquashfs can optionally preserve the mtime from the input
hierarchy it processes if only --pack-dir is specified.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- Store them in a struct instead of a hacky uint64_t with magic shifts
- Split up key/value pair write function to write_key and write_value
- Move the size accounting into those functions respectively
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit adds support for packing sparse files into squashfs images
as follows:
- In the data writer: simply detect zero blocks and write a zero to the
block size field and don't emit any data. Record the number of bytes
saved this way. For fragments, set the fragment offset to invalid.
- In the inode writer: write out the number of bytes saved for sparse
files. If there should be a fragment but there is none, append a block
count of 0.
- In the data reader: if the block size is 0, read nothing from disk and
emit an empty block. Do the same if the fragment is missing.
- In the inode reader: restore the number of bytes saved for sparse files.
The sparse files can be packed and unpacked, but the unpacking will not
create sparse files for now.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of decomposing a default string in gensquashfs option processing,
move that to fstree_init instead and pass the option string directly to
fstree_init.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit removes the packdir/packfile based directory setup magic
from fstree_from_file and moves it to gensquashfs. Over there, the
common parts are deduplicated.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Make tree node list sort and recursive variant available and independend
of the fstree_t.
This is considered cleaner, since the fstree_t actually isn't needed for
any of this and we can just call the recusvie sort on the root instead,
and we can use the sort implementation directly for things like the
upcoming unit test.
Also this commit splits up the merge/sort implementation into a seperate
split and merge functions to make the code somewhat more readable.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Replace default_$FIELD with a struct stat. Merge the rest of
add_node/add_file into mknode and add by path+stat.
Expose the mknode function and replace all the duplicated node pointer
arithmetic magic with a call to mknode.
Plus a generic cleanup by utlizing new util functions.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Don't need that, we already know the index when we write add
the fragment.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of allocating inode numbers as we go, generate and populat an
inode table from the fstree ahead of time. This makes processing nodes
a little bit simpler and we will need that table anyway for NFS export
support later on.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
