squashfs-tools-ng.git/lib/sqfs/data_writer, branch v0.8

Merge windows and pthread thread pool implementations

2019-12-13T01:27:17+00:00

Since they are both structured the same way using condition variables, they are only a few defines away from removing code duplication. Signed-off-by: David Oberhollenzer

Cleanup data writer

2019-12-13T01:05:42+00:00

Signed-off-by: David Oberhollenzer

Don't kick off the threads until the queue is full

2019-12-12T15:43:01+00:00

The idea is as follows: Initially let the user submit blocks until the queue is filled, then kick of the threads. Every thread will end up getting a block without any waits until they completely deplete the queue. Assuming the threads take longer to process the data than it takes the main thread to do I/O and submit new blocks, the queue should stay mostly filled with minimal wait times. Signed-off-by: David Oberhollenzer

Fix thread pool queue accounting

2019-12-12T11:48:33+00:00

- ONLY manipulate the back log counter in the main thread. - Fix the order of operations when submitting blocks. Signed-off-by: David Oberhollenzer

Replace SetEvent synchronization with condition variables

2019-12-08T15:48:59+00:00

It's cleaner, more stable and works pretty much the same way as the pthread version. The downside is that the minimum target for the library is now Windows Vista, or Server 2008. But both are over a decade old anyway, so this shouldn't be an issue. Signed-off-by: David Oberhollenzer

Add native Windows port of the multi-threaded data writer

2019-12-08T13:17:53+00:00

Signed-off-by: David Oberhollenzer

Minor data writer cleanup

2019-12-05T14:44:50+00:00

Move "do block" function over to the rest of the block related code and internalizie the pthread worker structure. Signed-off-by: David Oberhollenzer

Fix pthread data writer interfering with signal handling

2019-12-05T11:32:00+00:00

If a signal is sent to a process, the POSIX spec says that ANY thread could be picked ARBITRARILY to handle the signal. In our case this could be one of the internal worker threads. The problem here is that an unsuspecting user of the library might suddenly have their signal handler run in parallel to their main thread and run into weird concurrency issues, because they didn't expect that to happen. In fact, the libsquashfs API tries to be transparent about whether or not it uses a thread pool internally and does everything other than number crunching (e.g. I/O that may happen through user supplied callbacks) in the same thread as the one that submitts the blocks. Unfortunately, pthread doesn't have a way to set a signal mask for the new thread and setting it inside the thread is racy (i.e. a signal might be delivered before the worker thread sets the mask). The only portable and non-racy way to do this, is to disable all signals in the calling thread that sets up the data writer, create the threads (which will inherit the mask) and then resetore the previous signal mask, hoping for the best. The downside to this is that signals may be lost in that short time. Signed-off-by: David Oberhollenzer

Cleanup: move overflow safe alloc code into libsquashfs

2019-11-25T12:20:08+00:00

There were only a hand full of instances outside libsquashfs that used the alloc code. In most cases, the thing allocated hat its size derived from something already in memory anyway, so it is safe to assume its size fits into a size_t. At the same time, the opencoded Windows path conversion functions are all unified into a single helper function. Signed-off-by: David Oberhollenzer

Cleanup: move libutil related headers to "util" sub directory

2019-10-07T13:34:22+00:00

Signed-off-by: David Oberhollenzer