| Age | Commit message (Collapse) | Author |
|---|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
There is no need to keep a copy of the state data internally. If the
external user wants to reset/rewind, they can simply keep a copy of
the old state, now that it is completely transparent.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of holding the state internally, move it outside and let the
user pass it in. This allows processing multiple directories and going
back and forth between them, while also simplifying the dir reader
internals.
The sqfs_dir_reader_get_inode function can also be made more generic
to accept a reference instead of being fully state dependent.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Those functions are only used in the example code, we remove them for
now to make restructuring easier and fix the examples later.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The only user in the code base is sqfs_dir_reader_find_by_path.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Always set the reference count to 1 after creating a copy
of an object.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The reason this is implemented separately, instead of roling it into
the recursive iterator, is so that we can do additional filtering
in between. For instance, we can rewrite the path and the hard link
path will match up, or if we remove nodes from the hierarchy, we
won't end up with a hard link pointing outside.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Rename the open function to sqfs_file_open, use an argument for the
return pointer and pass back and error number on failure. Also add
an inermediate function to open an sqfs_file_t using a handle, similar
to the stream API. The get_file_size function is moved to the native
wrappers and some of the implementation is cleaned up a little.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This might be needed externally for things like hard link detection.
For the unix implementation, store the actual inode number here.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
To the sqfs_data_reader_t is added, an sqfs_istream_t implementation
that internally reads through the data reader. The uses of the
data_reader_dump function are removed and the function is subsequently
removed from libcommon.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The recursive part and the filter part are split up, the recursive
iterator wrapper is moved into libsquashfs and the libio iterator
is modified to use that internally instead of implementig the
recursion step.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Exact operation performed:
git ls-files -z | \
xargs -0 sed -i -e 's/sqfs_dir_entry_t/sqfs_dir_node_t/g'
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Start with a common prefix "sqfs_native_file_*"
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
It is mainly a very thin wrapper on top of the block processor so far.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
By adding additional close/duplicate primitives, the remaining, mostly
identical istream/ostream/file code is mostly identical between Windows
and Unix and be merged, mostly without stitches.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- The ostream creation functions already have flag arguments,
but make them an sqfs_u32 instead of int.
- Add flag arguments to the istream functions, sanitzie and forward
them when opening the handle.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Having a function to interpret the flags and open a native file handle
simplifies the istream/ostream/file code which shares that decoding
part, particularly on windows where the character set needs to be
transformed.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Similar to the sqfs_istream_t & sqfs_ostream_t interfaces.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
For now, only the interfaces and helper functions are moved, the
concrete implementations remain in libio.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The common pattern is used in rdsquashfs and sqfs2tar, move the code
to libsquashfs.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The structure and functions are renamed to sqfs_xattr_* instead,
an additional helper is added to accept an encoded xattr.
Documentation and unit test are added as well.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The idea of the block align feature was to allow micro-managing that
some files are forcefully aligned to 1k/4k ("device block") boundaries,
hoping to improve access time at the cost of data density. The feature
was not exposed in the tools for a long time and eventuall added to the
sort file. Measurement and experimentation showed, that it in fact
worsened the read performance on a test system with an old micro SD
card as the bottle neck.
The feature is removed, and if needed, can be brought back simply by
wrapping/sub-classing the default block writer, if need be..
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Implement grab/drop functions to increase/decrease reference count
and destroy the object if the count drops to 0.
Make sure that all objects that maintain internal references actually
grab that reference, duplicate it in the copy function, drop it in
the destroy handler.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
mksquashfs generates extended inodes if a directory contains 256
entries. libsquashfs so far only generated extended inodes if there
is no other way to encode it. Mimic the behaviour of mksquashfs by
adding a threshold.
For this to work, the "sqfs_inode_set_xattr_index" function has to
be changed to not immediately try to demote inodes to basic types.
The fstree serialization is modified to do that itself if the index
is 0xFFFFFFFF and the target is not a directory inode.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Test against various invariants:
- Every non-root node must have a name
- The root node muts not have a name
- The name must not be ".." or "."
- The name must not contain '/'
- The loop that chases parent pointers must terminate, i.e. we must
never reach the starting state again (link loop).
Furthermore, make sure the sum of all path components plus separators
does not overflow.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Add a simple directory state object to the meta data reader and use
that to iterate directory entries. The code for reading the directory
listing is movde to readdir.c
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Two flags are added to the dir reader API, one for the create function
that the dir reader should report those entries and one to the open
function to suppress that if it was enabled.
To implement the feature, a mapping of visited directory inodes is
maintained internally, that mapps inode numbers to inode references.
When opening a directory, state is maintained to generate the fake
entries for '.' and '..'. Since all the other functions are based on
the open/read/rewind API, no alterations need to be made. The tree
scan function is modified, to use the suppress flag, so it does not
accidentally catch those entries.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Many library destructor functions (like free()) allow a NULL
pointer as input, and do nothing in that case.
This allows easier cleanup patterns: initialize pointers to NULL
and then always pass them to the destroyer functions, no need for
verbose goto/if-else patterns.
Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com>
|
|
When opening files on windows, use the widechar versions and convert
from (assumed) UTF-8 to UTF-16 as needed.
Since the broken, code-page-random API may acutall be intended in some
use cases, leave that option in through an additional flag.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The squashfs readdir() implementation in the Linux kernel returns
non-existing "." and ".." entries for offsets 0 and 1, and after
that reads from disk. For convenience, it was decided to store an
off-by-3 value on disk instead of doing complex primary school math
to adjust for this. This didn't show up until now, because the kernel
implementation trusts the value from the directory header more than
the actual size in the inode and happily reads 3 more than the inode
would allow it to. This only showed up with 7-zip which subtracts 3
from the size and expects the result to be exact and bails if the
directory headers suggest otherwise.
And yes, I did consider making a "Holy Hand Granade of Antioch"
reference, but consciously decided not to.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
If SQFS_STATIC is defined, dummy out the SQFS_API definition, so we
don't try to pull stuff from a (in this case) non-existant DLL or
try to export functions.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
On systems like Windows, the dynamic library and applications can
easily end up being linked against different runtime libraries, so
applications cannot be expected to be able to free() any malloc'd
pointer that the library returns.
This commit adds an sqfs_free function so the application can pass
pointers back to the library to call the correct free() implementation.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This function creates a block processor from a structure describing it.
A stub implementation for the old sqfs_block_processor_create is added
that simply sets up such a struct and forwards the call.
The current version of the description struct only contains the exact
same parameters and a size field at the beginning.
This approach is supposed to make extending the range of parameters
easier without breaking ABI compatibillity.
Currently already planned are:
- Adding a sqfs_file_t pointer to double-check when deduplicating
fragments.
- When the scanning code reaches a usable state, add the abillity
to pass scanned fragment data, so the block processor can be used
for appending to an existing image.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of comparing (compresed, disk-size, checksum) tuples to find
block matches, do an exact, byte-for-byte comparison of the data
stored on disk to avoid the possibility of a spurious colision.
Since this is the desired behaviour, make it the default, optionally
overrideable through a flag.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
It does not make any changes to the writer itself, so mark it as
const. This also requires some similar changes to the string table.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Fragment deduplication really doesn't belong into the public API of
the fragment table.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
