| Age | Commit message (Collapse) | Author |
|---|
|
The structure and functions are renamed to sqfs_xattr_* instead,
an additional helper is added to accept an encoded xattr.
Documentation and unit test are added as well.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The idea of the block align feature was to allow micro-managing that
some files are forcefully aligned to 1k/4k ("device block") boundaries,
hoping to improve access time at the cost of data density. The feature
was not exposed in the tools for a long time and eventuall added to the
sort file. Measurement and experimentation showed, that it in fact
worsened the read performance on a test system with an old micro SD
card as the bottle neck.
The feature is removed, and if needed, can be brought back simply by
wrapping/sub-classing the default block writer, if need be..
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Implement grab/drop functions to increase/decrease reference count
and destroy the object if the count drops to 0.
Make sure that all objects that maintain internal references actually
grab that reference, duplicate it in the copy function, drop it in
the destroy handler.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
mksquashfs generates extended inodes if a directory contains 256
entries. libsquashfs so far only generated extended inodes if there
is no other way to encode it. Mimic the behaviour of mksquashfs by
adding a threshold.
For this to work, the "sqfs_inode_set_xattr_index" function has to
be changed to not immediately try to demote inodes to basic types.
The fstree serialization is modified to do that itself if the index
is 0xFFFFFFFF and the target is not a directory inode.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Test against various invariants:
- Every non-root node must have a name
- The root node muts not have a name
- The name must not be ".." or "."
- The name must not contain '/'
- The loop that chases parent pointers must terminate, i.e. we must
never reach the starting state again (link loop).
Furthermore, make sure the sum of all path components plus separators
does not overflow.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Add a simple directory state object to the meta data reader and use
that to iterate directory entries. The code for reading the directory
listing is movde to readdir.c
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Two flags are added to the dir reader API, one for the create function
that the dir reader should report those entries and one to the open
function to suppress that if it was enabled.
To implement the feature, a mapping of visited directory inodes is
maintained internally, that mapps inode numbers to inode references.
When opening a directory, state is maintained to generate the fake
entries for '.' and '..'. Since all the other functions are based on
the open/read/rewind API, no alterations need to be made. The tree
scan function is modified, to use the suppress flag, so it does not
accidentally catch those entries.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Many library destructor functions (like free()) allow a NULL
pointer as input, and do nothing in that case.
This allows easier cleanup patterns: initialize pointers to NULL
and then always pass them to the destroyer functions, no need for
verbose goto/if-else patterns.
Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com>
|
|
When opening files on windows, use the widechar versions and convert
from (assumed) UTF-8 to UTF-16 as needed.
Since the broken, code-page-random API may acutall be intended in some
use cases, leave that option in through an additional flag.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The squashfs readdir() implementation in the Linux kernel returns
non-existing "." and ".." entries for offsets 0 and 1, and after
that reads from disk. For convenience, it was decided to store an
off-by-3 value on disk instead of doing complex primary school math
to adjust for this. This didn't show up until now, because the kernel
implementation trusts the value from the directory header more than
the actual size in the inode and happily reads 3 more than the inode
would allow it to. This only showed up with 7-zip which subtracts 3
from the size and expects the result to be exact and bails if the
directory headers suggest otherwise.
And yes, I did consider making a "Holy Hand Granade of Antioch"
reference, but consciously decided not to.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
If SQFS_STATIC is defined, dummy out the SQFS_API definition, so we
don't try to pull stuff from a (in this case) non-existant DLL or
try to export functions.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
On systems like Windows, the dynamic library and applications can
easily end up being linked against different runtime libraries, so
applications cannot be expected to be able to free() any malloc'd
pointer that the library returns.
This commit adds an sqfs_free function so the application can pass
pointers back to the library to call the correct free() implementation.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This function creates a block processor from a structure describing it.
A stub implementation for the old sqfs_block_processor_create is added
that simply sets up such a struct and forwards the call.
The current version of the description struct only contains the exact
same parameters and a size field at the beginning.
This approach is supposed to make extending the range of parameters
easier without breaking ABI compatibillity.
Currently already planned are:
- Adding a sqfs_file_t pointer to double-check when deduplicating
fragments.
- When the scanning code reaches a usable state, add the abillity
to pass scanned fragment data, so the block processor can be used
for appending to an existing image.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of comparing (compresed, disk-size, checksum) tuples to find
block matches, do an exact, byte-for-byte comparison of the data
stored on disk to avoid the possibility of a spurious colision.
Since this is the desired behaviour, make it the default, optionally
overrideable through a flag.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
It does not make any changes to the writer itself, so mark it as
const. This also requires some similar changes to the string table.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Fragment deduplication really doesn't belong into the public API of
the fragment table.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Every compressor (except LC4) has a compression level parameter. This
commit pulls the compression level field out into the generic
configuration structure and applies some code clean ups as a result
from this.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The LZMA compressor (through the xz-utils library) supports basically
the same options for micro management as the XZ compressor.
This commit enables support for those options in the compressor, the
option parser and adds an option field to the configuration structure.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit adds propper defines in the super block header and removes
some of the hard coded constants.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Previously, the XZ compressor was called LZMA at some points in the
code, because it actually referres to LZMAv2 and the older LZMAv1
wasn't implemented. Since LZMAv1 is supported for some time now,
simply calling it LZMA is confusing.
This commit tries to replace the last bits where XZ is referred to
as LZMA, which turns out to be only a hand full of comments.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This function allows submission of raw blocks to the block processor,
completely bypassing the file API.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit adds 2 new user settable flags to the block processor:
- A flag to ignore sparse blocks and treat them like normal
data blocks.
- A flag to disable checksum computation altogether.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit modifies the block processor to support associating a user
data pointer with data blocks that it forwards to the block writer,
which is modified to accept an optional user data pointer.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This way, everything that could be done through the hooks (and more)
can be done by simply providign a custom implementation. The result is
a lot clener that the previous hook based version.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- the "bytes submitted" can be moved over to the block processor
- the number of blocks submitted are already there (implcitily, by
adding the data block count to the fragment block count)
- actual data bytes written can be computed from the super block
- the remaining block count can be changed to simple counter that
can be obtained through a function.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Its purely informational, but make sure other programs don't print
out scary messages that imply the data has been ineficiently.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This patch allows external users to fiddle with the XZ compressors
compression strength, alignment and other values.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Otherwise, C++ compilers will scream.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Make sure the function has a way of telling the caller *why* it failed.
This way, the function can convey whether it had an internal error, an
allocation failure, whether the arguments are totaly nonsensical, or
simply that the compressor *or specific configuration* is not supported.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Avoid namespace polution. Make sure all exportet symbols are prefixed
with either sqfs_ or SQFS_.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of creating everything in the "create" function, cleanup and
create/initialize stuff in a "load" function. This allows the xattr
reader to be reset/re-used and adds the benefit of not having to
lug around references to the super block, compressor and file (altough
the later two are hidden inside the meta reader).
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This patch adds a deep-copy callback to sqfs_object_t and removes the
copying mechanism from sqfs_compressor_t. This is also interesting for
other types.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This allows getting the compressor configuration back after
creating it, for various purposes.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
With unified payload size counters, copying an inode is now trivial.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
If the block processor allocates and dynamically resizes inodes on
the fly, we can add data indefinitely without knowing the size of
the file ahead of time.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of having seperate counters for blocks, dir index bytes
and having to fiddle out the link target size, simply use a single
value that stores the number of payload bytes used.
A seperate "payload bytes available" is used for dynamically
growing inodes during processing.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This function waits for all pending blocks to be written to disk, but
doesn't flush the fragment block, so processing can continue afterwards
as if nothing happened.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
