aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2023-04-29gensquashfs: Remove flag handling from scan_dirDavid Oberhollenzer
The logic was previously there to allow recursing into directories that exist in the fstree and the scanned directory, but not create new directories. Because the dir_tree_iterator_t supports reporting contents but not directories (and does DFS), we can simply check if the parent of the directory exists. Under normal DFS, it must exist, because we created it earlier. If we are skipping the directory entries, it only exists if it was there earlier and we can skip entries that have a non-existant parent, retainint the old behavior. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29gensquashfs: Remove fstree_from_subdirDavid Oberhollenzer
It is only used in the glob function, so assemble the path there and jus tuse fstree_from_dir. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29Move type based filtering to libutil dir_tree_iterator_tDavid Oberhollenzer
Limited unit testing for the flags is added, particularly the abillity to recurse into sub-directories, but not report the parents as individual entries. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29gensquashfs: use stacked tree iterator in fstree_from_dirDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29libutil: Add a stacked, recursive directory tree iteratorDavid Oberhollenzer
The concept is simple: Use the existing, platform dependent iterator to walk a directory. If a directory entry is encountered, recurse into it using the open_subdir handler, reconstruct the full path for any entries discovered using the directory stack. An additional function is added to skip a sub-hierarchy. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29gensquashfs: simplify sort_file_listDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29Fix: gensquashfs: sort by file breaking up the directory listDavid Oberhollenzer
When inserting the selected entry, the next_by_type pointer is supposed to be cleared, but the code accidentaly broke up the next pointer, removing the element from the tree. Fixes: e1e655b02f6c54177f9070eeb221ab95c6d4e20f Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-21libutil: unix: simplify/unify directory iterator error handlingDavid Oberhollenzer
Instead of printing out error messages, return an errro ID and match the behavior with the Windows implementation. Also, don't check first if the struct stat says it is a link, the readlinkat() system call will fail if it isn't. Avoid confusing the deputy. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-21Fix: libutil: type check bug in unix directory iterator read_linkDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-21libutil: Add a method to the directory iterator to open a sub directoryDavid Oberhollenzer
This is also the reason we need to lug around the original directory path on Windows. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-21libutil: win32: clenaup dir iterator initialization, retain the pathDavid Oberhollenzer
Instead of dropping the path immediately, store it inside the structure. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-21libutil: simplify win32 directory iterator state handlingDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-20Collect and print statistics about the kind of files we are packingDavid Oberhollenzer
Using depth-first search, we collect some crude statistics about directory tree types (e.g. regular files, directories, device special files and so on) and print them out after serializing the tree. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-20libfstree: Remove special modes for hard linksDavid Oberhollenzer
Instead of adding special sentinel modes, simply treat hard links as special case of symlinks, setting a flag to indicate that it is a hard link and another flag to indicate that it has been resolved. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-19libfstree: Make hard link resolution non-recursiveDavid Oberhollenzer
Use the next_by_type pointer to create a list of all unresolved hard links and iterate over that list for link resolution. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-19libfstree: Remove external file_info_t declarationDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-19libfstree: hoist file link pointer into parent structureDavid Oberhollenzer
Instead of having a file_info_t next pointer, requiring an up-cast to tree_node_t all the time, simply add a "next_by_type" pointer to the tree node itself, which can also be used for other purposes by other node types and removes the need for up-casting. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-19libfstree: Change file_info_t already_matched into flagDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-19libfstree: get rid of dir_info_tDavid Oberhollenzer
The single boolean created_implicitly can be replaced with a general purpose flag field. The "children" pointer can then be hoisted directly into the data union of tree_node_t. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-17fstree_from_dir: Move the recursion step into scan_dirDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-17Add unit test for directory iteratorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-17Internalize fstree_insert_sortedDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-17Remove the remainder of the Win32 fstree_from_dir implementationDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-17Implement a version of the directory iterator for UnixDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-12Merge Windows and Unix implementations of fstree_from_subdirDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-12Copy the dir scan pattern from the Windows implementationDavid Oberhollenzer
If we look at a sub directory, piece totether the full path and open that instead. This allows us to move the directory open into the scan function and get rid of the dirfd trickery. Also pull the recusion step out into the parent function. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-12Cleanup: gensquashfs: split readlink handling out of directory scanDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-12gensquashfs: merge some of the Windows and Unix entry filtering codeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-12Win32: gensquashfs: Merge fstree_from_dir and subdir implementationsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-12Split out Windows directory iteration code to a dir_iterator_t typeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-12Win32: Fix fstree CLI mtime range checkDavid Oberhollenzer
On Windows, long is a 32 bit integer, so we cannot check if the long value is greater than UINT32_MAX. Instead, check if strtol sets errno. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-12Fix: missing libcompat dependency for libio tests on WindowsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-12Fix fstree defaults usage in Windows version of fstree_from_dirDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-12Add mkwinbins.sh based script to setup a mingw development enviromentDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-06libfstree: simplify hard link resolution codeDavid Oberhollenzer
We do not allow hard links to directories, so we can toss the special case handling code for that. The visited mechanism was pointless anyway, because we don't even descend down hard links in the recursive tree handling functions. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-06Cleanup: libfstree: move all the hardlink related code to hardlink.cDavid Oberhollenzer
For some reason, the recursive hardlink resolution ended up in post process, calling into the non recrusive one in hardlink.c that wasn't used elsewhere. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-06libfstree: Move left over dirscan flags to gensquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-04Move scan callback from libfstree to gensquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-02Move fstree CLI code to libcommonDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-01libsquashfs: remove the default block writer alignment featureDavid Oberhollenzer
The idea of the block align feature was to allow micro-managing that some files are forcefully aligned to 1k/4k ("device block") boundaries, hoping to improve access time at the cost of data density. The feature was not exposed in the tools for a long time and eventuall added to the sort file. Measurement and experimentation showed, that it in fact worsened the read performance on a test system with an old micro SD card as the bottle neck. The feature is removed, and if needed, can be brought back simply by wrapping/sub-classing the default block writer, if need be.. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-21libio: Add an xfrm stream wrapper testDavid Oberhollenzer
This basically re-uses the libxfrm pack/unpack tests, but runs the data through a stream wrapper. When de-compressing, we have a ridiculously tiny input buffer, to force the wrapper to snort up the data in several attempts until the it can decompress something. We read the data byte-by-byte to force the wraper to internally cache the uncompressed data and spoon feed it to us. It has to be completely transparent to us that it internaly decompresses and also reads transparently across concatenated streams. When compressing, we only require that the output is smaller than the input and not equal to it. We also require the wrapper to flush the wrapped stream when it is flushed. We then test if the compressed data can be unpacked again. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-21libxfr: Fix zstd test naming errorDavid Oberhollenzer
We were previously building pack for unpack and vice versa. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-21libtar: generate entire xattr header in a single bufferDavid Oberhollenzer
Having it all in one buffer allows us the re-use the "generat GNU record" function. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-20Remove ostream_printfDavid Oberhollenzer
By cobbling together the xattr lines manually in libtar, the need (and thus the function itself) are removed. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-20Fix compressor ID enumerator in format documentationDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-15Fix broken C++ guard in rbtree.hDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-12libtar: Add a test for the tar writing codeDavid Oberhollenzer
Generate a simple tarball and compare it with a reference. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-11libio: Add a test case for istream-to-ostream spliceDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-10libio: Add a test for istream precache/read/skipDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-08libtar: remove need for skip_padding functionDavid Oberhollenzer
In the istream implementation, automatically skip the padding when we reach end-of-file. Also skip file AND padding when we destroy the object. Replace the remaining instances with a simple istream_skip instead and remove the wrapper from libtar. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>