aboutsummaryrefslogtreecommitdiff
path: root/lib
AgeCommit message (Collapse)Author
2023-06-12libio: add open handle functions to istream/ostreamDavid Oberhollenzer
For the backends, this simplifies the code as both paths (open file and open stdio) use the same basic code. Even when merging them only in the backend, it would be done in a similar way. Making the functions public allows other uses as well. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-12Resolve Windows port related issuesDavid Oberhollenzer
- Reorder dependencies for tar2sqfs - Fix istream_t implementation issue Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-11libio: move istream buffer logic into interall callbacksDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-11libio: remove precache from istream_advance_bufferDavid Oberhollenzer
Since the user has to call istream_get_buffered_data afterwards anyway, we can do the precache lazily. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-10libio: add desired read size to istream_get_buffered_dataDavid Oberhollenzer
This properly maps to all of our use cases and makes istream_precache obsolete. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-09libtar: overhaul buffer management in istream_t wrapperDavid Oberhollenzer
As soon as we no longer have any data to read, unlock/drop the parent iterator_t object. Also, make sure we get the buffer count right, not all data might have been consumed yet when precache is called. Remove the precache/read loop in the non-sparse case, we have already established that there is data available. If it is insufficient, the user will simply call precache again once it's used up, which istream_get_buffered_data forwards to a precache call in the underlying stream. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-09libio: remove eof flag from istream_t interfaceDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-09libio: eliminate direct access of the interal bufferDavid Oberhollenzer
Instead, go through helper functions, which in a next step can be moved inside the implementation. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-08Move tar compressor auto wrapping code from tar2sqfs into libtarDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-07libsquashfs: Add utility functions to read xattrs into listDavid Oberhollenzer
The common pattern is used in rdsquashfs and sqfs2tar, move the code to libsquashfs. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-05libsqfs: Add an xattr writer function to add a combined key-value structDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-05Move dir_entry_xattr_t from libio to libsquashfsDavid Oberhollenzer
The structure and functions are renamed to sqfs_xattr_* instead, an additional helper is added to accept an encoded xattr. Documentation and unit test are added as well. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-05libio: istream_get_line: minor cleanupDavid Oberhollenzer
Remove redundant initializations, reduce variable scope. At exist of the function, try to shrink the buffer to the actual size if it got trimmed. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-05libio: istream_get_line: fix memory leak in error pathDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-05libio: istream_get_line: fix trailing CR sanitationDavid Oberhollenzer
The istream_get_line function is supposed to remove the line break, inclduing CR+LF if used. The previous implementation simply checked the buffer when the LF was found, missing the case where the CR was alraedy added and the LF is at the beginning of the newly loaded data. This commit modifies the check to inspect the line itself when it was found, and remove the CR after the fact. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-05libio: add a simple unit test for the memory istream_t implementationDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-05libio: remove buffer_offset from istream_tDavid Oberhollenzer
Instead, make the buffer const, let the user adjust the pointer and size. The offset can then be inferred in precache. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-04libio: move splice function from ostream to istreamDavid Oberhollenzer
It touches internals of the istream, particularly the buffer, but not of the ostream. It really belongs to istream_t. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-04libio: Move istream_t precache logic into backend implementationDavid Oberhollenzer
The end goal is to remove direct buffer access from the istream_t interfaces and make that opaque. For the tar implementation, this already safes us needless buffer copying, as we essentially allow the user to read-through from the underlying stream. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-03libio: consolidate add-hoc memory istream_t implementationsDavid Oberhollenzer
In several places, there are ad-hoc istream_t implementations that read from a memory buffer to test something else stacked on top. This commit consolidates those ad-hoc implmentations into a proper one in libio, and uses the chance to remove external files for some older tests that rely on file I/O instead. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-06-01libio: split dir_entry_t from dir_iterator_t, add create helperDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-30libfstree: accept dir_entry_t instead of path and struct statDavid Oberhollenzer
Because the dir_entry_t also has a flag for had links, the regular node and hard-link node interface can be unified. This simplifies the users of libfstree (gensquashfs, tar2sqfs) since we can simply hose the entries from an iterator directly into the tree. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-22libtar: add a dir_iterator_t implementation for tar filesDavid Oberhollenzer
The existing istream_t wrapper is mered into this one as well, we can open the files via the iterators open_file_ro function. Unit tests and tar2sqfs are modified accordingly. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-16libtar: replace tar_xattr_t with dir_entry_xattr_t structDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-16libio: add xattr query interface to dir_iterator_tDavid Oberhollenzer
Again, with a dummy implementation for Unix and Windows backends. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-15libio: remove device number from dir_iterator_t, add flag fieldDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-14libio: Add a function to open an istream_t from a directory iteratorDavid Oberhollenzer
For the native versions, this is currently dummied out, always returning an error number. The idea is to laster wrap the libtar interface around a directory iterator, here we need that method to support the existing use case in tar2sqfs. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-14libio: Move skip function into dir_iterator_t base interfaceDavid Oberhollenzer
For regular iterator types, it's a no-op, for the tree iterator, it skips the sub tree. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-14libio: merge directory iterator headers, document API and behaviorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-13Move directory iterator from libutil to libioDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-05-05Internalize fstree_mknode, consolidate fstree functionalityDavid Oberhollenzer
The fstree_mknode function is only used internally, remove the declaration from the header and internalize it. The code using it is consolidated into fstree.c. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-30Move the pattern matching from gensquashfs to dir_tree_iterator_tDavid Oberhollenzer
A simple unit test is added that mainly checks for the behavior of recursing into a sub-tree and only matching the children at the end, but not reporting the parents that don't match. The behavior is inteded to immitate the `find` command. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29gensquashfs: use prefix functionality for glob_files target nodeDavid Oberhollenzer
Piece together the prefix path and pass it to the iterator. That way, we get the full target paths back from the iterator and can use those directly in the callback for filtering. We also no longer need the root node for fstree_from_dir (always tree root) and the callback can no longer return an error state. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29Move dir entry remapping from gensquashfs to libutilDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29Cleanup: libutil: split functionality of dir tree iterator nextDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29libutil: Add an option to the dir_tree_iterator_t to add a path prefixDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29Move type based filtering to libutil dir_tree_iterator_tDavid Oberhollenzer
Limited unit testing for the flags is added, particularly the abillity to recurse into sub-directories, but not report the parents as individual entries. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29gensquashfs: use stacked tree iterator in fstree_from_dirDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-29libutil: Add a stacked, recursive directory tree iteratorDavid Oberhollenzer
The concept is simple: Use the existing, platform dependent iterator to walk a directory. If a directory entry is encountered, recurse into it using the open_subdir handler, reconstruct the full path for any entries discovered using the directory stack. An additional function is added to skip a sub-hierarchy. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-21libutil: unix: simplify/unify directory iterator error handlingDavid Oberhollenzer
Instead of printing out error messages, return an errro ID and match the behavior with the Windows implementation. Also, don't check first if the struct stat says it is a link, the readlinkat() system call will fail if it isn't. Avoid confusing the deputy. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-21Fix: libutil: type check bug in unix directory iterator read_linkDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-21libutil: Add a method to the directory iterator to open a sub directoryDavid Oberhollenzer
This is also the reason we need to lug around the original directory path on Windows. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-21libutil: win32: clenaup dir iterator initialization, retain the pathDavid Oberhollenzer
Instead of dropping the path immediately, store it inside the structure. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-21libutil: simplify win32 directory iterator state handlingDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-20Collect and print statistics about the kind of files we are packingDavid Oberhollenzer
Using depth-first search, we collect some crude statistics about directory tree types (e.g. regular files, directories, device special files and so on) and print them out after serializing the tree. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-20libfstree: Remove special modes for hard linksDavid Oberhollenzer
Instead of adding special sentinel modes, simply treat hard links as special case of symlinks, setting a flag to indicate that it is a hard link and another flag to indicate that it has been resolved. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-19libfstree: Make hard link resolution non-recursiveDavid Oberhollenzer
Use the next_by_type pointer to create a list of all unresolved hard links and iterate over that list for link resolution. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-19libfstree: hoist file link pointer into parent structureDavid Oberhollenzer
Instead of having a file_info_t next pointer, requiring an up-cast to tree_node_t all the time, simply add a "next_by_type" pointer to the tree node itself, which can also be used for other purposes by other node types and removes the need for up-casting. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-19libfstree: get rid of dir_info_tDavid Oberhollenzer
The single boolean created_implicitly can be replaced with a general purpose flag field. The "children" pointer can then be hoisted directly into the data union of tree_node_t. Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
2023-04-17Add unit test for directory iteratorDavid Oberhollenzer
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-09 00:12:26 +0000