| Age | Commit message (Collapse) | Author |
|---|
|
The squashfs on-disk format uses 32 bit link counters, but the fstree
used 16 bit ones. Because the link count also includes child nodes,
this artificially limited the number of entries in a directory to ~64k
files.
This patch removes the limit by switching libfstree to 32 bit counters.
Reported-by: Marvin Renich <mrvn@renich.org>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
If fstree_mknode fails, because the parent link count would overflow,
the function fails and cleans up behind it. The problem arises because
the function does this check *after* inserting the node in the parent
node, so it is later free'd again, when destroying the rest of the
tree.
This patch moves the insertion after the check to mitigate the problem.
Reported-by: Marvin Renich <mrvn@renich.org>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Autoconf 2.69 was released in 2012. After that, 2.70 was released
in December 2020 and 2.71 in January 2021.
According to repology, only very few distros ship 2.7x, most still
being on 2.69. This includes the last Ubuntu LTS release, which is
used as goto container distro in our automated testing and analysis
setup.
Version 2.69 should cover most distros, except some outliers
like CentOS 6 or Maemo Fremantle (*cough* Nokia N900).
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Split out several repated patterns into helper functions and move the
rest of the code back into dir_reader.c
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Add a simple directory state object to the meta data reader and use
that to iterate directory entries. The code for reading the directory
listing is movde to readdir.c
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
When asking the meta data reader for its current position and
we *just* read to the end of a block, report the start of the
next block as the current location.
Otherwise, trying to *seek* to the resulting position immediately
after reporting throws an out-of-bounds error.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Implement a check that verifies that we have at least a single
SquashFS block compressor available and aborst the build if none
was found.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
On GNU/Linux, *BSD or MacOS we can simply use the system default
library. The copy was primarily only there for the Windows build.
The build script for Windows has now been adapted to download and
compile a shared library from a tarball.
This removes a huge chunk of code from the git tree as well as
the release tarballs. Additionally it gets rid of iffy things like
removing the Zlib copyright/version strings, so the libsquashfs DLL
doesn't export it.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Two flags are added to the dir reader API, one for the create function
that the dir reader should report those entries and one to the open
function to suppress that if it was enabled.
To implement the feature, a mapping of visited directory inodes is
maintained internally, that mapps inode numbers to inode references.
When opening a directory, state is maintained to generate the fake
entries for '.' and '..'. Since all the other functions are based on
the open/read/rewind API, no alterations need to be made. The tree
scan function is modified, to use the suppress flag, so it does not
accidentally catch those entries.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The idea was originally to use struct stat in the libfstree code, so
we can simply hose data read from a directory into the fstree_t. The
struct was then also used with libtar, for simpler interoperation,
but it turned out to introduce a lot of platform quirks and causes
more trouble than it's worth.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The fstree sorting code got removed recently, in favour of inserting
at the correct position. The Windows directory scanning code still
used it's own list insertion code instead of mknode (so it could
allocate and translate the directory entry name in-place), which
broke the sorting order.
This issue is fixed in this commit.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Now that there is a wrapper for main() on Windows, all executable
programs must use a common, cannonical signature for main().
Furthermore, the Windows version of the epoch test needs wrappers
for setenv/unsetenv.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
A `flags` field and `priority` are added to all file information
structs. A news fstree function is introduced for parsing a "sort-file".
Each line in the file is space separated, and has the following format:
priority [flags] filename
Priority is a 64 bit number, flags are optional and filename can be put
in quotes if it is supposed to start or end with spaces. Single line
comments can be used.
The flags can be used to set block-processor flags (e.g. don't fragment,
or don't compress), as well as instructing the parser to use file
globbing to match the filename.
After parsing the file, the list of file info structure is sorted
according to the priority (default is 0) using a stable sort algorithm.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Always insert the tree nodes in the correct oder and remove the
post-process sorting step.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Instead of having a long if-else-if chain, replace the PAX header
field parsing with a table driven approach.
Altough it is more code, it is hopefully more readable, maintainable,
extensible and it dedupliates some of the value parsing code.
The GNU.sparse parsing is left as is, because it requires
maintaining state.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Split the key/value pairs right in the header and terminate the key
name. This way, some of the magic numbers can be eliminated.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Many library destructor functions (like free()) allow a NULL
pointer as input, and do nothing in that case.
This allows easier cleanup patterns: initialize pointers to NULL
and then always pass them to the destroyer functions, no need for
verbose goto/if-else patterns.
Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Abort and retry in situations that should logically _never_
_ever_ happen.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
The block_count is a size_t, so on 32 bit platforms the multiplication
might be truncated before the comparison with filesz.
On 64 bit platforms, it could potentially also overflow the 64 bit
bounds of the data type.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Preprocessor magic is used to redirect putc/fputc/fputs/printf/fprintf
to custom implementations.
The custom implementations try to figure out if we are printing to the
console and, if so, convert the resulting strings to UTF-16 and print
them through ConsoleWriteW. If the output is redirected to a file or
a pipe, the original (presummed) UTF-8 is kept.
Simply setting the console output codepage to UTF-8 does not work,
because the standard I/O facilities of MSVCRT either does not support
unicode (in non-wchar mode), or has half-broken support through fputs,
which can still break up multi-byte sequences through its internal
buffering.
Likewise, changing the codepage and using ConsoleWriteA, or trying to
use fputws did not work in a test VM either.
This approach is the one that worked most consistently among the
ones tried, but also has problems. E.g. it breaks when setting the
codepage to UTF-8 manually (using `chcp 65001`).
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
When piping the output of another program into tar2sqfs.exe, and
the source program terminates, tar2sqfs.exe gets an ERROR_BROKEN_PIPE
when the end is reached and it trys to pre-cache more data. This
commit adds a work around, to propperly handle this as and end-of-file
condition.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Apparently, mingw implicitly included stdlib.h indirectly from either
windows.h or shellapi.h. After an upgrade, the windows build now
fails with EXIT_FAILURE being undefined.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Now that there is a wrapper for main() on Windows, all executable
programs must use a common, cannonical signature for main().
Furthermore, the Windows version of the epoch test needs wrappers
for setenv/unsetenv.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
When opening files on windows, use the widechar versions and convert
from (assumed) UTF-8 to UTF-16 as needed.
Since the broken, code-page-random API may acutall be intended in some
use cases, leave that option in through an additional flag.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
A macro and forward declaration are added to compat.h that rename
the main() function programs using compat.h into sqfs_tools_main.
An actual main() function is added to libcompat.a, that uses the
shell API to get the UTF-16 command line arguments, convert them
to UTF-8 and call sqfs_tools_main.
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
