| Age | Commit message (Collapse) | Author |
|---|
|
This commit moves the entire block writing and deduplication of data
blocks over to a different data type named "block writer".
For simplicity, the interfaces of the block processor are left as is
and are turned into warppers. Likewise, most of the code in the block
writer is just verbatim from the block processor, to be cleaned up in
subsequent commits.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This removes further clutter from the data writer. Any future efforts
on making fragment by hash lookup faster can focus on that area only
and don't clutter the block processor.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Just to be safe in case there needs to be an extension
in the future.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
There are 3 types of extra payload:
- Directory index
- File block sizes
- Symlink target
This commit removes the type specific pointers and modifies the code
to use the payload area directly. To simplify the file block case and
mitigate alignment issues, the type of the extra field is changed to
sqfs_u32.
For symlink target, the extra field can simply be cast to a character
pointer (it had to be cast anyway for most uses). For block sizes,
probably the most common usecase, it can be used as is. For directory
indices, there is a helper function anyway.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Tests with the Debian image (which is generated with squashfs-tools,
so should be interpreted as ground truth) have showed that the count
is not stored off-by-one. The code was already doing the right thing,
but the documentation was wrong.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
It optionally allows code that does tree traversal to start at an
inode that it obtained previously and makes it easier to keep state
externally.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>
|
|
If the dir writer is used to create the directory table, it neccessarily
sees every single inode number and coresponding location for all inodes
that are referenced by the filesystem tree. This means it can easily
collect that information internally to create an export table later on.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of returning the ID through a pointer and an error code as
return status, return a single int that could be a compressor ID
(positive values) or an error code (negative values).
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- Add an explicit "you're holding it wrong" error code.
- Consistently return error codes and not have some special places
where -1 is returned.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of using int or unsigned int for generic function flag
arguments, consistently use an unsigned, fixed size type.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Before the misspelled version has a chance to become stable API.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- Padd the compressor config union
- 128 bytes aught to be enough for everyone, i.e. future compressors.
- Insist that the padding space is initialized to 0. If a field gets
added to an existing compressor, it can test for 0 as a sentinel
value.
- Add a size field to the hook structure, aka "the Microsoft way".
- The explanation is in the comment.
- Don't make the Microsoft mistake of checking for >=, insist on *exact*
size match. Future users will need a fallback if their hooks are
rejected. But at least they will be rejected instead of silently not
being used.
- Add an unsupported flag check to the dir tree reader.
- Add a basic abi unit test that, for now, checks the size of the compressor
config struct fields.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Combines all the type depenend attribute setting and inode type
promotion into a single function.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit modifies the dir writer to store the directory index directly
in the payload area of the inode it creates. The size of the index in bytes
is stored in a seperate field. Storing the index is moved to the write
inode function.
The read inode function is in turn also modified to read the directory
index from disk and actually store it in the inode structure in memory
for later use.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
- We don't have "endian.h" everywhere. On some BSDs its in sys and
on some BSDs the macros have different names.
- We definitely don't have sysmacros.h on non-Unix-like systems.
- Likewise for sys/types.h, sys/stat.h and their contents.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
ssize_t is only available on POSIX platforms and even there it is
only defined to hold at least -1 in the range of negative numbers.
This commit replaces ssize_t return types with sqfs_s32 and the
coresponding function arguments with sqfs_u32. Because the range
of positiv numbers for a signed 32 bit number is only half that of
the unsigned version, additional checks have to be added.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This is a fully automated search and replace, i.e. I ran this:
git grep -l uint8_t | xargs sed -i 's/uint8_t/sqfs_u8/g'
git grep -l uint16_t | xargs sed -i 's/uint16_t/sqfs_u16/g'
git grep -l uint32_t | xargs sed -i 's/uint32_t/sqfs_u32/g'
git grep -l uint64_t | xargs sed -i 's/uint64_t/sqfs_u64/g'
git grep -l int8_t | xargs sed -i 's/int8_t/sqfs_s8/g'
git grep -l int16_t | xargs sed -i 's/int16_t/sqfs_s16/g'
git grep -l int32_t | xargs sed -i 's/int32_t/sqfs_s32/g'
git grep -l int64_t | xargs sed -i 's/int64_t/sqfs_s64/g'
and than added the appropriate definitions to sqfs/predef.h
The whole point being better compatibillity with platforms that may
not have an stdint.h with the propper definitions.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This commit turns the file interface into an actual, generic file
interface and does away with having to move around blocks outside
the data writer. Instead the data writer takes over full control
and responsibility of dividing the input data up into blocks
propperly.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Basically move the state tracking from the old data writer over to
the new one.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This only exists for tar2sqfs. Move the sparse file map to libtar
and add the ability to do this into the stind sqfs_file_t abstraction,
so it acts like a normal file but internally stitches the data
together from the sparse implementation.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
So far, this is mostly a direct port from the block processor. The
actual fragment checksumming is not done through the thread pool.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
If an error happpens, immediately stop everything and report only the
first error that happened.
This also fixes a memory leak in the error paths of the serial and
pthread based block processors.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
First, remove the "Don't checksum" flag, fragment blocks also need to
be checksumed and are also subject to deduplication. For sentinel
blocks, instead check if the size is zero.
Second, cleanly seperate the user settable flags from the non-user
settable flags and reject any block that has non-user settable or
unknown flags set.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of calling a callback, the block processor now takes care of
writing the data blocks to the file in the correct order, keeping
track of fragment blocks and deduplicating blocks.
Some cleanup work remains to be done and the statistics have to be
repaired (yet again).
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
