| Age | Commit message (Collapse) | Author |
|---|
|
This removes further clutter from the data writer. Any future efforts
on making fragment by hash lookup faster can focus on that area only
and don't clutter the block processor.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit moves the single use helper functions that are called from
worker thread context into the worker thread function.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
There are 3 types of extra payload:
- Directory index
- File block sizes
- Symlink target
This commit removes the type specific pointers and modifies the code
to use the payload area directly. To simplify the file block case and
mitigate alignment issues, the type of the extra field is changed to
sqfs_u32.
For symlink target, the extra field can simply be cast to a character
pointer (it had to be cast anyway for most uses). For block sizes,
probably the most common usecase, it can be used as is. For directory
indices, there is a helper function anyway.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Since they are both structured the same way using condition variables,
they are only a few defines away from removing code duplication.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The idea is as follows: Initially let the user submit blocks until the
queue is filled, then kick of the threads. Every thread will end up
getting a block without any waits until they completely deplete the
queue. Assuming the threads take longer to process the data than it
takes the main thread to do I/O and submit new blocks, the queue should
stay mostly filled with minimal wait times.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- ONLY manipulate the back log counter in the main thread.
- Fix the order of operations when submitting blocks.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
It's cleaner, more stable and works pretty much the same way as the
pthread version. The downside is that the minimum target for the
library is now Windows Vista, or Server 2008. But both are over a
decade old anyway, so this shouldn't be an issue.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Move "do block" function over to the rest of the block related
code and internalizie the pthread worker structure.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
If a signal is sent to a process, the POSIX spec says that ANY thread
could be picked ARBITRARILY to handle the signal. In our case this could
be one of the internal worker threads.
The problem here is that an unsuspecting user of the library might
suddenly have their signal handler run in parallel to their main thread
and run into weird concurrency issues, because they didn't expect that
to happen.
In fact, the libsquashfs API tries to be transparent about whether or
not it uses a thread pool internally and does everything other than
number crunching (e.g. I/O that may happen through user supplied
callbacks) in the same thread as the one that submitts the blocks.
Unfortunately, pthread doesn't have a way to set a signal mask for the
new thread and setting it inside the thread is racy (i.e. a signal might
be delivered before the worker thread sets the mask).
The only portable and non-racy way to do this, is to disable all signals
in the calling thread that sets up the data writer, create the threads
(which will inherit the mask) and then resetore the previous signal mask,
hoping for the best.
The downside to this is that signals may be lost in that short time.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
There were only a hand full of instances outside libsquashfs that used
the alloc code. In most cases, the thing allocated hat its size derived
from something already in memory anyway, so it is safe to assume its
size fits into a size_t.
At the same time, the opencoded Windows path conversion functions are
all unified into a single helper function.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Before the misspelled version has a chance to become stable API.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- Padd the compressor config union
- 128 bytes aught to be enough for everyone, i.e. future compressors.
- Insist that the padding space is initialized to 0. If a field gets
added to an existing compressor, it can test for 0 as a sentinel
value.
- Add a size field to the hook structure, aka "the Microsoft way".
- The explanation is in the comment.
- Don't make the Microsoft mistake of checking for >=, insist on *exact*
size match. Future users will need a fallback if their hooks are
rejected. But at least they will be rejected instead of silently not
being used.
- Add an unsupported flag check to the dir tree reader.
- Add a basic abi unit test that, for now, checks the size of the compressor
config struct fields.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
If we don't store the padding location in the deduplication list,
the block deduplication might try to deduplicate across two files
with padding in between, which can't be done.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This is a fully automated search and replace, i.e. I ran this:
git grep -l uint8_t | xargs sed -i 's/uint8_t/sqfs_u8/g'
git grep -l uint16_t | xargs sed -i 's/uint16_t/sqfs_u16/g'
git grep -l uint32_t | xargs sed -i 's/uint32_t/sqfs_u32/g'
git grep -l uint64_t | xargs sed -i 's/uint64_t/sqfs_u64/g'
git grep -l int8_t | xargs sed -i 's/int8_t/sqfs_s8/g'
git grep -l int16_t | xargs sed -i 's/int16_t/sqfs_s16/g'
git grep -l int32_t | xargs sed -i 's/int32_t/sqfs_s32/g'
git grep -l int64_t | xargs sed -i 's/int64_t/sqfs_s64/g'
and than added the appropriate definitions to sqfs/predef.h
The whole point being better compatibillity with platforms that may
not have an stdint.h with the propper definitions.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit turns the file interface into an actual, generic file
interface and does away with having to move around blocks outside
the data writer. Instead the data writer takes over full control
and responsibility of dividing the input data up into blocks
propperly.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Basically move the state tracking from the old data writer over to
the new one.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
