aboutsummaryrefslogtreecommitdiff
path: root/include
AgeCommit message (Collapse)Author
2023-05-30libfstree: accept dir_entry_t instead of path and struct statDavid Oberhollenzer
Because the dir_entry_t also has a flag for had links, the regular node and hard-link node interface can be unified. This simplifies the users of libfstree (gensquashfs, tar2sqfs) since we can simply hose the entries from an iterator directly into the tree. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-22libtar: add a dir_iterator_t implementation for tar filesDavid Oberhollenzer
The existing istream_t wrapper is mered into this one as well, we can open the files via the iterators open_file_ro function. Unit tests and tar2sqfs are modified accordingly. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-16libtar: replace tar_xattr_t with dir_entry_xattr_t structDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-16libio: add xattr query interface to dir_iterator_tDavid Oberhollenzer
Again, with a dummy implementation for Unix and Windows backends. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-15libio: remove device number from dir_iterator_t, add flag fieldDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-14libio: Add a function to open an istream_t from a directory iteratorDavid Oberhollenzer
For the native versions, this is currently dummied out, always returning an error number. The idea is to laster wrap the libtar interface around a directory iterator, here we need that method to support the existing use case in tar2sqfs. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-14libio: Move skip function into dir_iterator_t base interfaceDavid Oberhollenzer
For regular iterator types, it's a no-op, for the tree iterator, it skips the sub tree. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-14libio: merge directory iterator headers, document API and behaviorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-13Move directory iterator from libutil to libioDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-05Internalize fstree_mknode, consolidate fstree functionalityDavid Oberhollenzer
The fstree_mknode function is only used internally, remove the declaration from the header and internalize it. The code using it is consolidated into fstree.c. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-30Move the pattern matching from gensquashfs to dir_tree_iterator_tDavid Oberhollenzer
A simple unit test is added that mainly checks for the behavior of recursing into a sub-tree and only matching the children at the end, but not reporting the parents that don't match. The behavior is inteded to immitate the `find` command. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29gensquashfs: use prefix functionality for glob_files target nodeDavid Oberhollenzer
Piece together the prefix path and pass it to the iterator. That way, we get the full target paths back from the iterator and can use those directly in the callback for filtering. We also no longer need the root node for fstree_from_dir (always tree root) and the callback can no longer return an error state. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29Move dir entry remapping from gensquashfs to libutilDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29libutil: Add an option to the dir_tree_iterator_t to add a path prefixDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29Move type based filtering to libutil dir_tree_iterator_tDavid Oberhollenzer
Limited unit testing for the flags is added, particularly the abillity to recurse into sub-directories, but not report the parents as individual entries. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29gensquashfs: use stacked tree iterator in fstree_from_dirDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29libutil: Add a stacked, recursive directory tree iteratorDavid Oberhollenzer
The concept is simple: Use the existing, platform dependent iterator to walk a directory. If a directory entry is encountered, recurse into it using the open_subdir handler, reconstruct the full path for any entries discovered using the directory stack. An additional function is added to skip a sub-hierarchy. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-21libutil: Add a method to the directory iterator to open a sub directoryDavid Oberhollenzer
This is also the reason we need to lug around the original directory path on Windows. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-20Collect and print statistics about the kind of files we are packingDavid Oberhollenzer
Using depth-first search, we collect some crude statistics about directory tree types (e.g. regular files, directories, device special files and so on) and print them out after serializing the tree. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-20libfstree: Remove special modes for hard linksDavid Oberhollenzer
Instead of adding special sentinel modes, simply treat hard links as special case of symlinks, setting a flag to indicate that it is a hard link and another flag to indicate that it has been resolved. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-19libfstree: Make hard link resolution non-recursiveDavid Oberhollenzer
Use the next_by_type pointer to create a list of all unresolved hard links and iterate over that list for link resolution. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-19libfstree: Remove external file_info_t declarationDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-19libfstree: hoist file link pointer into parent structureDavid Oberhollenzer
Instead of having a file_info_t next pointer, requiring an up-cast to tree_node_t all the time, simply add a "next_by_type" pointer to the tree node itself, which can also be used for other purposes by other node types and removes the need for up-casting. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-19libfstree: Change file_info_t already_matched into flagDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-19libfstree: get rid of dir_info_tDavid Oberhollenzer
The single boolean created_implicitly can be replaced with a general purpose flag field. The "children" pointer can then be hoisted directly into the data union of tree_node_t. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-17Internalize fstree_insert_sortedDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-17Implement a version of the directory iterator for UnixDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-12Split out Windows directory iteration code to a dir_iterator_t typeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-06libfstree: simplify hard link resolution codeDavid Oberhollenzer
We do not allow hard links to directories, so we can toss the special case handling code for that. The visited mechanism was pointless anyway, because we don't even descend down hard links in the recursive tree handling functions. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-06Cleanup: libfstree: move all the hardlink related code to hardlink.cDavid Oberhollenzer
For some reason, the recursive hardlink resolution ended up in post process, calling into the non recrusive one in hardlink.c that wasn't used elsewhere. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-06libfstree: Move left over dirscan flags to gensquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-04Move scan callback from libfstree to gensquashfsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-02Move fstree CLI code to libcommonDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-01libsquashfs: remove the default block writer alignment featureDavid Oberhollenzer
The idea of the block align feature was to allow micro-managing that some files are forcefully aligned to 1k/4k ("device block") boundaries, hoping to improve access time at the cost of data density. The feature was not exposed in the tools for a long time and eventuall added to the sort file. Measurement and experimentation showed, that it in fact worsened the read performance on a test system with an old micro SD card as the bottle neck. The feature is removed, and if needed, can be brought back simply by wrapping/sub-classing the default block writer, if need be.. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-20Remove ostream_printfDavid Oberhollenzer
By cobbling together the xattr lines manually in libtar, the need (and thus the function itself) are removed. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-15Fix broken C++ guard in rbtree.hDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-08libtar: remove need for skip_padding functionDavid Oberhollenzer
In the istream implementation, automatically skip the padding when we reach end-of-file. Also skip file AND padding when we destroy the object. Replace the remaining instances with a simple istream_skip instead and remove the wrapper from libtar. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-08libtar: Add an istream_t implementationDavid Oberhollenzer
The tar_istream_t reads the data from a tar file, having been given the header, and synthesizes zero bytes for sparse regions. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-04libtar: internalize the declaration of read_octalDavid Oberhollenzer
Use read_number in the places that remain. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-04libtar: simplfy parsing of old GNU sparse formatDavid Oberhollenzer
There was some code duplication for extracting the sparse entry fields from the start record and the subsequent extended record. This commit introduces a data structure for both and unifies the parsing code paths. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-02-04libtar: some minor cleanupsDavid Oberhollenzer
- Use is_memory_zero from libutil - Move checksum update function to tar writer code - Move checksum verify function to tar reader code - Only export the function to compute the checksum Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-26libio: move simple wrappers back into header as inline functionsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-19Split stream compression out of libioDavid Oberhollenzer
Move it to a separate libxfrm library, where it can be independently tested as well. The bulk of the new code is also mainly test cases for the compressors. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-19Implement rudimentary reference counting for sqfs_object_tDavid Oberhollenzer
Implement grab/drop functions to increase/decrease reference count and destroy the object if the count drops to 0. Make sure that all objects that maintain internal references actually grab that reference, duplicate it in the copy function, drop it in the destroy handler. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-19Add a helper function to initialize libsquashfs objectsDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-19Drop backwards compatibillity note for sqfs_freeDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-01-19libsqfs: add a threshold for extended directory inodes with indexDavid Oberhollenzer
mksquashfs generates extended inodes if a directory contains 256 entries. libsquashfs so far only generated extended inodes if there is no other way to encode it. Mimic the behaviour of mksquashfs by adding a threshold. For this to work, the "sqfs_inode_set_xattr_index" function has to be changed to not immediately try to demote inodes to basic types. The fstree serialization is modified to do that itself if the index is 0xFFFFFFFF and the target is not a directory inode. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-22Move gensquashfs specific code from libfstree to gensquashfsDavid Oberhollenzer
The "from dir" and from "from file" code, as well as the "sort file" code is specific to gensquashfs, so move them there and the test cases as well. The medium term idea is to reduce libfstree to a stub, merge it into the generic writer and ultimately hoist that into libsquashfs. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-18Add a single, central base64 decoderDavid Oberhollenzer
Similar to the hex blob decoder, we need this once for tar and once for the filemap xattr parser. Simply add a single, central implementation to libutil, with a simple unit test, and then use it in both libtar and gensquashfs. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2022-11-18Add a single, central hex blob decoderDavid Oberhollenzer
Since we need it twice (once for tar, once for the filemap xattr parser), add a single, central implementation to libutil, add a unit test for that implementation and then use it in both libtar and gensquashfs. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-09 00:57:54 +0000