| Age | Commit message (Collapse) | Author |
|---|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The idea is as follows: Initially let the user submit blocks until the
queue is filled, then kick of the threads. Every thread will end up
getting a block without any waits until they completely deplete the
queue. Assuming the threads take longer to process the data than it
takes the main thread to do I/O and submit new blocks, the queue should
stay mostly filled with minimal wait times.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- ONLY manipulate the back log counter in the main thread.
- Fix the order of operations when submitting blocks.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The zstd compress function returns an error code if it cannot fit the
compressed data into the given destination buffer.
This commit adds a check for this error and reports that the
libsquashfs compressor implementation was unable to shrink the input
instead of claiming a fatal error happened.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
It's cleaner, more stable and works pretty much the same way as the
pthread version. The downside is that the minimum target for the
library is now Windows Vista, or Server 2008. But both are over a
decade old anyway, so this shouldn't be an issue.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Move "do block" function over to the rest of the block related
code and internalizie the pthread worker structure.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
If a signal is sent to a process, the POSIX spec says that ANY thread
could be picked ARBITRARILY to handle the signal. In our case this could
be one of the internal worker threads.
The problem here is that an unsuspecting user of the library might
suddenly have their signal handler run in parallel to their main thread
and run into weird concurrency issues, because they didn't expect that
to happen.
In fact, the libsquashfs API tries to be transparent about whether or
not it uses a thread pool internally and does everything other than
number crunching (e.g. I/O that may happen through user supplied
callbacks) in the same thread as the one that submitts the blocks.
Unfortunately, pthread doesn't have a way to set a signal mask for the
new thread and setting it inside the thread is racy (i.e. a signal might
be delivered before the worker thread sets the mask).
The only portable and non-racy way to do this, is to disable all signals
in the calling thread that sets up the data writer, create the threads
(which will inherit the mask) and then resetore the previous signal mask,
hoping for the best.
The downside to this is that signals may be lost in that short time.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of returning the ID through a pointer and an error code as
return status, return a single int that could be a compressor ID
(positive values) or an error code (negative values).
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
There were only a hand full of instances outside libsquashfs that used
the alloc code. In most cases, the thing allocated hat its size derived
from something already in memory anyway, so it is safe to assume its
size fits into a size_t.
At the same time, the opencoded Windows path conversion functions are
all unified into a single helper function.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The check needs to include 1M, which is still a valid block size.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- Add an explicit "you're holding it wrong" error code.
- Consistently return error codes and not have some special places
where -1 is returned.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The liblzo2 library is licensed under GPLv2, so it is not possible to
distribute binaries of libsquashfs that link against liblzo2 under
LGPL.
This commit moves the LZO compressor implementation to libcommon,
where this isn't a problem, since the tools themselves are licensed
under GPLv3.
It removes the ability of libsquashfs to read or generate LZO compressed
SquashFS images, but the tools still can.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
In most cases, including unistd.h and fcntl.h was a left over anyway.
In the cases where it was not, move it to compat.h.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The -static-libgcc flag has to be passed through the compiler with
a "-Wc," prefix, because libtool tries to be clever about linker
flags. If added directly to LDFLAGS, libtool removes it.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
SquashFS cannot represent empty string file names in the directory
entry structure, so reject them and report an error.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit moves the generic unix implementation into a "unix"
subdirectory and adds a "win32" subdirectory with a winapi based
implementation.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
After the last modification, the directory index size counter
no longer counts the size, but the number of index entries. To
avoid future confusion, remove the thing entirely and compute
the size on the fly.
Furthermore, eliminate unaligned memory writes and fix the
darn counter once again.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Apparently mksquashfs writes an actual usage count (1 = 1 entry,
2 = 2 entries; i.e. NOT off by one).
Also, if it does happen to be garbage, guard against an overflow.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of using int or unsigned int for generic function flag
arguments, consistently use an unsigned, fixed size type.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Before the misspelled version has a chance to become stable API.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- Padd the compressor config union
- 128 bytes aught to be enough for everyone, i.e. future compressors.
- Insist that the padding space is initialized to 0. If a field gets
added to an existing compressor, it can test for 0 as a sentinel
value.
- Add a size field to the hook structure, aka "the Microsoft way".
- The explanation is in the comment.
- Don't make the Microsoft mistake of checking for >=, insist on *exact*
size match. Future users will need a fallback if their hooks are
rejected. But at least they will be rejected instead of silently not
being used.
- Add an unsupported flag check to the dir tree reader.
- Add a basic abi unit test that, for now, checks the size of the compressor
config struct fields.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Combines all the type depenend attribute setting and inode type
promotion into a single function.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit modifies the dir writer to store the directory index directly
in the payload area of the inode it creates. The size of the index in bytes
is stored in a seperate field. Storing the index is moved to the write
inode function.
The read inode function is in turn also modified to read the directory
index from disk and actually store it in the inode structure in memory
for later use.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
SquashFS restores the file mode from the type, so those bits are
completely redundant.
On a side note: this also reduces entropy, allowing the compressor
to *maybe* squeeze a few more bytes out.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
If we don't store the padding location in the deduplication list,
the block deduplication might try to deduplicate across two files
with padding in between, which can't be done.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
If there were no key-value pairs recoreded during the lifetime of the
xattr writer, take a short cut out of the flush function and set the
super block values to indicate that.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
If begin/end was called without recording *any* attributes, simply
set the resulting index to the 0xFFFFFFFF sentinel.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Count the *corrent* entry structure size, i.e. the on-disk one, not the
in-memory one.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Mostly straight copy-pasta from the sqfshelper code, except it doesn't
use the fstree functions.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit implements the part of the API responsible for recoding
and deduplicating xattr key-value blocks.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
