aboutsummaryrefslogtreecommitdiff
path: root/lib/fstree
AgeCommit message (Collapse)Author
2023-07-10Move sqfs_dir_entry_t code from libio into libsquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-07-03Refactor: rename dir_entry_t to sqfs_dir_entry_tDavid Oberhollenzer
Exact operation performed: git ls-files -z | \ xargs -0 sed -i -e 's/dir_entry_t/sqfs_dir_entry_t/g Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-01libio: split dir_entry_t from dir_iterator_t, add create helperDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-30libfstree: accept dir_entry_t instead of path and struct statDavid Oberhollenzer
Because the dir_entry_t also has a flag for had links, the regular node and hard-link node interface can be unified. This simplifies the users of libfstree (gensquashfs, tar2sqfs) since we can simply hose the entries from an iterator directly into the tree. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-05Internalize fstree_mknode, consolidate fstree functionalityDavid Oberhollenzer
The fstree_mknode function is only used internally, remove the declaration from the header and internalize it. The code using it is consolidated into fstree.c. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29gensquashfs: use prefix functionality for glob_files target nodeDavid Oberhollenzer
Piece together the prefix path and pass it to the iterator. That way, we get the full target paths back from the iterator and can use those directly in the callback for filtering. We also no longer need the root node for fstree_from_dir (always tree root) and the callback can no longer return an error state. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29gensquashfs: use stacked tree iterator in fstree_from_dirDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-20Collect and print statistics about the kind of files we are packingDavid Oberhollenzer
Using depth-first search, we collect some crude statistics about directory tree types (e.g. regular files, directories, device special files and so on) and print them out after serializing the tree. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-20libfstree: Remove special modes for hard linksDavid Oberhollenzer
Instead of adding special sentinel modes, simply treat hard links as special case of symlinks, setting a flag to indicate that it is a hard link and another flag to indicate that it has been resolved. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-19libfstree: Make hard link resolution non-recursiveDavid Oberhollenzer
Use the next_by_type pointer to create a list of all unresolved hard links and iterate over that list for link resolution. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-19libfstree: hoist file link pointer into parent structureDavid Oberhollenzer
Instead of having a file_info_t next pointer, requiring an up-cast to tree_node_t all the time, simply add a "next_by_type" pointer to the tree node itself, which can also be used for other purposes by other node types and removes the need for up-casting. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-19libfstree: get rid of dir_info_tDavid Oberhollenzer
The single boolean created_implicitly can be replaced with a general purpose flag field. The "children" pointer can then be hoisted directly into the data union of tree_node_t. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-17Internalize fstree_insert_sortedDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-06libfstree: simplify hard link resolution codeDavid Oberhollenzer
We do not allow hard links to directories, so we can toss the special case handling code for that. The visited mechanism was pointless anyway, because we don't even descend down hard links in the recursive tree handling functions. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-06Cleanup: libfstree: move all the hardlink related code to hardlink.cDavid Oberhollenzer
For some reason, the recursive hardlink resolution ended up in post process, calling into the non recrusive one in hardlink.c that wasn't used elsewhere. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-02Move fstree CLI code to libcommonDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-31Reintegrate test code with library codeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-31Move library source into src sub-directoryDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-22Move gensquashfs specific code from libfstree to gensquashfsDavid Oberhollenzer
The "from dir" and from "from file" code, as well as the "sort file" code is specific to gensquashfs, so move them there and the test cases as well. The medium term idea is to reduce libfstree to a stub, merge it into the generic writer and ultimately hoist that into libsquashfs. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-07-08Cleanup: Get rid of libfstree "internal.h" headerDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-07-08Cleanup: move source date epoch code back to libutilDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-07-08Cleanup: move filename_sane & canonicalize_path functions to libutilDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-07-08Cleanup: rename libfstream to libio, split headersDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-07-08Fix: libfstree: actually use a full 32 bit hard link counterDavid Oberhollenzer
The squashfs on-disk format uses 32 bit link counters, but the fstree used 16 bit ones. Because the link count also includes child nodes, this artificially limited the number of entries in a directory to ~64k files. This patch removes the limit by switching libfstree to 32 bit counters. Reported-by: Marvin Renich <mrvn@renich.org> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-07-08Fix: libfstree: double free in error pathDavid Oberhollenzer
If fstree_mknode fails, because the parent link count would overflow, the function fails and cleans up behind it. The problem arises because the function does this check *after* inserting the node in the parent node, so it is later free'd again, when destroying the rest of the tree. This patch moves the insertion after the check to mitigate the problem. Reported-by: Marvin Renich <mrvn@renich.org> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-06-03Fix: libfstree: free() path string in error code pathDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-30Fix: Directory insertion order on WindowsDavid Oberhollenzer
The fstree sorting code got removed recently, in favour of inserting at the correct position. The Windows directory scanning code still used it's own list insertion code instead of mknode (so it could allocate and translate the directory entry name in-place), which broke the sorting order. This issue is fixed in this commit. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-30Add a rudimentary unit test for sort filesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-30Add sort-file implementationDavid Oberhollenzer
A `flags` field and `priority` are added to all file information structs. A news fstree function is introduced for parsing a "sort-file". Each line in the file is space separated, and has the following format: priority [flags] filename Priority is a 64 bit number, flags are optional and filename can be put in quotes if it is supposed to start or end with spaces. Single line comments can be used. The flags can be used to set block-processor flags (e.g. don't fragment, or don't compress), as well as instructing the parser to use file globbing to match the filename. After parsing the file, the list of file info structure is sorted according to the priority (default is 0) using a stable sort algorithm. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-03-30Cleanup: remove node sorting from libfstreeDavid Oberhollenzer
Always insert the tree nodes in the correct oder and remove the post-process sorting step. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25libfstree: guard against possible overflow in readlink()David Oberhollenzer
*in theory*, say on a 32 bit system, we could have a 32 bit size_t and a 64 bit off_t. If the filesystem permitted this, we *could* then have a symlink with a target > 4G. Or the target is exacetely 4G, but adding a null-terminator could exceed addressable memory. This commit adds a check to guard against such an overflow and throw an error, instead of silently wrapping around. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-06-25libfstree: guard against link count and inode number overflowDavid Oberhollenzer
If the hard link counter or the inode number counter overflow the maximum representable value (for SquashFS 16 bit and 32 bit respecitively), abort with an error message. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-03-25libfstree: allow the glob path to be emptyDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-03-25libfstree: add an assertion that root is not NULLDavid Oberhollenzer
If the path argument is "", we assume that referes to root and set the *existing* target node to the root node and skip ahead across the tree search. This leaves "name" uninitialized, which makes coverity panic, because fs->root could be NULL, going down the wrong path. Obviously, this should never, *ever* happen and there is no reasonable recovery strategy if it suddenly does, so simply add an assertion. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-03-25libfstree: Allow / as argument for "glob" and "dir" commandsDavid Oberhollenzer
This allows putting globbed files & directories into the filesystem root, as well as explicitly setting attributes of the root directory from the file lisiting. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-03-24libfstree: implement directory scanning code for WindowsDavid Oberhollenzer
It's rather simplistic and doesn't account for junction/reparse points, which is the closest thing Windows has to symlinks, hard links and mount points, but it's consistent with the unpacking code that assumes Windows only has files and directories. Using the 32 bit mingw toolchain, this seems to satisfy the unit tests on wine. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-03-20Add libcompat fallback implementation for fnmatchDavid Oberhollenzer
This has basically been copied over from Musl and slightly modifed. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-02-19Fix: libfstree: add an assert the canonicalize_name return valueDavid Oberhollenzer
Since the canonicalize_name function only fails if the path contains ".." and the one we are constructing from the scanned fstree (built using canonicalized names), it should NEVER fail. However, coverity does get concerned, because we are checking the return value elesewhere. So do what we do at other, similar locations and add an assert(). Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-02-19Fix: canonicalize path names in glob pattern matchingDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-02-19libfstree: reject unknown glob options to allow future expansionsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-02-19fstree_from_file: Add fnmatch() pattern matching to file globbingDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-02-19fstree_from_file: Implement basic file globbingDavid Oberhollenzer
The base path is passed to the fstree_from_file function and in turn to the individual callbacks. The line parsing function is modified to allow '*' as mode, uid and gid for specifically marked callbacks. A glob callback is added that internally uses the fstree_from_dir scanning functions in combination with a filter callback. Directory scanning flags are parsed from the extra arguments before interpreting it as a path fragment. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-02-19libfstree: Add a filter callback to the directory scanning functionDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-02-18libfstree: add a subdirectory scanning functionDavid Oberhollenzer
So we can scan a sub-directory within a the base directory without having to do string operations first. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-02-18fstree_from_dir: add filtering flags to skip certain inode typesDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2021-02-10cleanup: fstree_from_file: split & simplify line parsing functionDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-12-29Fix normalization of slashes in filenamesDavid Oberhollenzer
All paths were canonicalized internally, which includes filtering sequences of slashes and converting backslashes to slashes. Furthermore, when unpacking files, filenames are sanity checked and rejected if they contain forward OR backward slashes. This is a problem on Unix-like systems, where files containing backslashes are a legitimate use case (*cough* SystemD *cough*). This patch removes the backslash conversion from the canonicalization and modifies the sanity check to reject backslashes only on Windows. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-12-15libfstree: make the directory scanning code a little more genericDavid Oberhollenzer
- Instead of using the fstree root, let the caller specify it. - Add a flag to prevent recursion into sub directories. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-12-10Move fstree dirscan code back to libfstreeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2020-09-29Replace file/getline usage with istreamDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-09 01:12:33 +0000