aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
7 daystests: ubifs_tools: fsck_tests: Add powercut+fsck+mount testZhihao Cheng
Inject powercut while doing fsstress on mounted UBIFS for kinds of flashes (eg. nand, nor). This testcase mainly makes sure that fsck.ubifs can make UBIFS image be consistent on different flashes (eg. nand, nor). Because the min_io_size of nor flash is 1, the UBIFS image on nor flash will be different from nand flash after doing powercut, so we need make sure fsck.ubifs can handle these two types of flash. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daystests: ubifs_tools: fsck_tests: Add cycle mount+fsck testZhihao Cheng
Do fsstress and fsck, check whether there are any files(and their data) are lost after fsck. This testcase mainly checks whether fsck.ubifs could corrupt the filesystem content in common case. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daystests: ubifs_tools: fsck_tests: Add authentication refusing testZhihao Cheng
Authenticated UBIFS image is not supported in fsck, add testcase to check that. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daystests: Add common libs for testing fsck.ubifs/mkfs.ubifsZhihao Cheng
This is a preparation for adding testcases for fsck.ubifs and mkfs.ubifs. Add some common functions, for example: powercut, load_mtdram, mount_ubifs, encryption operations, etc. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Add README to describe fsckZhihao Cheng
Add documents to describe fsck, which includes introductions, designment, advantage and limitations. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: Zhang Yi <yi.zhang@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Do final committingZhihao Cheng
This is the 18/18 step of fsck. Do final committing, commit problem fixing modifications(which are generated since step 14) to disk, and clear %UBIFS_MST_DIRTY flag for master node. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Handle disconnected filesZhihao Cheng
This is the 17/18 step of fsck. Recover disconnected files into lost+found. If there is no free space left to recover the disconnected files, fsck may delete the files to make filesystem be consistent. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Check and create the lost+foundHuang Xiaojia
This is the 16/18 step of fsck. Check whether the lost+found is existed, create a new one if it is not found. This step makes sure that disconnected file can be recovered under the lost+found. Signed-off-by: Huang Xiaojia <huangxiaojia2@huawei.com> Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Check and create the root dirZhihao Cheng
This is the 15/18 step of fsck. Check whether the root dir is existed, create a new one if it is not found. This step makes sure that filesystem can be mounted successful. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysubifs-utils: libubifs: Support some file operationsZhihao Cheng
Add some file operations, such as ubifs_lookup, ubifs_mkdir, etc., this is a preparation for recovering disconnected files or root dir in fsck. File writing operations are based on the journal subsystem, generated dirty data depends on a new commit in subsequent steps to update disk content. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Check and correct the index sizeZhihao Cheng
This is the 14/18 step of fsck. Check and correct the index size by traversing TNC just like dbg_check_idx_size does. This step should be executed after first committing, because 'c->calc_idx_sz' can be changed in 'ubifs_tnc_start_commit' and the initial value of 'c->calc_idx_sz' read from disk is untrusted. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Commit problem fixing modifications to diskZhihao Cheng
This is the 13/18 step of fsck. Commit problem fixing modifications (which are generated from the previous steps) to disk. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: check and correct the space statisticsZhihao Cheng
This is the 12/18 step of fsck. Check and correct the space statistics. There could be following steps and possible errors: Step 1. Exit for check mode, if %FR_LPT_CORRUPTED or %FR_LPT_INCORRECT is set in lpt status, the exit code should have %FSCK_UNCORRECTED. Step 2. Check lpt status, if %FR_LPT_CORRUPTED is set in lpt status, normal mode with 'no' answer will exit, other modes will rebuild lpt. Step 3. Traverse LPT nodes, check the correctness of nnode and pnode, compare LEB scanning result with LEB properties. a. LPT node is corrupted, normal mode with 'no' answer will exit, rebuild lpt for other modes. b. Incorrect nnode/pnode, normal mode with 'no' answer will exit, other other modes will correct the nnode/pnode. c. Inconsistent comparing result, normal mode with 'no' answer will exit, other modes will correct the space statistics. Step 4. Check and correct the lprops table information. Step 5. Set gc lnum(ubifs_rcvry_gc_commit / take_gc_lnum). Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Move common functions and data structures into check_space.cZhihao Cheng
This is a preparation for adding LPT checking support. Move some data structures and functions into check_space.c, also factor out some common functions in libubifs: 1. Move 'lpts' from rebuild module, make it resuable for non-rebuild_fs modes. 2. Move function 'get_free_leb' from rebuild_fs.c, it could be reused in building LPT. 3. Move function 'build_lpt' from rebuild_fs.c, it could be reused in building LPT. 4. Factor out lpt nodes freeing into a new function ubifs_free_lpt_nodes. 5. Factor out nnode dirty marking implementations into a new function ubifs_make_nnode_dirty. 5. Export the function of nnode number calculation, calc_nnode_num is renamed as ubifs_calc_nnode_num. 6. Export the function of making pnode dirty, do_make_pnode_dirty is renamed as ubifs_make_pnode_dirty. 7. Rename next_pnode_to_dirty to ubifs_find_next_pnode and export it. 8. Export free_buds and expend its parameters. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Check whether the TNC is emptyZhihao Cheng
This is the 11/18 step of fsck. Check whether the TNC is empty, turn to rebuild_fs if it is not found. Can we recreate a new root dir to avoid empty TNC? The answer is no, lpt fixing should be done before creating new entry, but lpt fixing needs a committing before new dirty data generated to ensure that bud data won't be overwritten(bud LEB could become freeable after replaying journal, corrected lpt may treat it as a free one to hold new data, see details in space checking & correcting step). Then we have to create the new root dir after fixing lpt and a committing, znode without children(empty TNC) maybe written on disk at the moment of committing, which corrupts the UBIFS image. So we choose to rebuild the filesystem if the TNC is empty, this case is equivalent to corrupted TNC. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Check and correct files' informationXiang Yang
This is the 10/18 step of fsck. Check and handle inconsistent files, the checking rule is same as rebuild mode which has been implemented in check_and_correct_files, but the methods of handling are different: 1. Correct the file information for safe mode, danger mode and normal mode with 'yes' answer, other modes will exit. Signed-off-by: Xiang Yang <xiangyang3@huawei.com> Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Check and handle unreachable filesZhihao Cheng
This is the 9/18 step of fsck. Check and handle unreachable files, the checking rule is same as rebuild mode which has been implemented in file_is_reachable, but the methods of handling are different: 1. Move unreachable regular file into disconnected list, let subsequent steps to handle them with lost+found. 2. Delete unreachable non-regular file. 3. Delete unreachable directory entries. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Check and handle invalid filesZhihao Cheng
This is the 8/18 step of fsck. Check and handle invalid files, the checking rule is same as rebuild mode which has been implemented in file_is_valid, but the methods of handling are different: 1. Move unattached(file has no dentries) regular file into disconnected list, let subsequent steps to handle them with lost+found. 2. Make file type be consistent between inode, detries and data nodes by deleting dentries or data blocks. 3. Delete file for other invalid cases(eg. file has no inode). Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Update files' size for check modeZhihao Cheng
This is the 7/18 step of fsck. Update files' size according to size tree for check mode, now all files are updated after replaying journal. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Ensure that TNC LEB can be scanned successfulZhihao Cheng
This is the second part of 6/18 step in fsck. Add an extra checking for non-check mode while traversing TNC, make sure that all LEBs(contains TNC) can be scanned successful. There could be following steps and possible errors: Step 2. Scan all LEBs(contain TNC), remove TNC branch which points to corrupted LEB. a. corrupted node is found by scanning: If current node is index node, danger mode with rebuild_fs and normal mode with 'yes' answer will turn to rebuild filesystem, other modes will exit; If current node is non-index node, danger mode and normal mode with 'yes' answer will remove all TNC branches which point to the corrupted LEB, other modes will exit. b. LEB contains both index and non-index nodes: danger mode with rebuild_fs and normal mode with 'yes' answer will turn to rebuild filesystem, other modes will exit. This is a preparation for space checking, which means that ubifs_scan will always succeed when check properties for any TNC LEBs. We do this before checking files(step 7) & extracting dentry tree(step 8), nodes cannot be dropped(which may corrupted file and make file inconsistent again) when scanning corrupted as long as the dentry tree is extracted. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Traverse TNC and construct filesZhihao Cheng
This is the 6/18 step of fsck. Traverse TNC and construct files. There could be following steps and possible errors: Step 1. Traverse TNC, check whether the leaf node is valid, remove invalid nodes, construct file for valid node and insert file into file tree. a. corrupted node searched from TNC: remove corresponding TNC branch for danger mode and normal mode with 'yes' answer, other modes will exit. b. corrupted index node read from TNC: danger mode with rebuild_fs and normal mode with 'yes' answer will turn to rebuild filesystem, other modes will exit. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Move common functions and data structures into fsck.ubifs.cZhihao Cheng
This is a preparation for adding TNC checking support. Following data structures and functions are moved into fsck.ubifs.c: 1. Move 'scanned_files' and 'used_lebs' from rebuild module, make them resuable for non-rebuild_fs modes. 2. Move function 'handle_error' from load_fs.c, it could be reused in other steps. 3. Add new function ubifs_tnc_remove_node in libubifs, which could remove index entry for a node by given position. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Recover isizeZhihao Cheng
This is the 5/18 step of fsck. Recover isize. There could be following steps and possible errors: Step 1. Traverse size tree, lookup corresponding inode from TNC a. corrupted node searched from TNC: skip node for danger mode and normal mode with 'yes' answer, other modes will exit. b. corrupted index node read from TNC: danger mode with rebuild_fs and normal mode with 'yes' answer will turn to rebuild filesystem, other modes will exit. Step 2. update isize for inode. Keep <inum, isize> in size tree for check mode, update inode node in place for other modes. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Consolidate logZhihao Cheng
This is the 4/18 step of fsck. Consolidate log to ensure enough space in log area. There could be following possible errors: 1. corrupted scanning data in log area: danger mode with rebuild_fs and normal mode with 'yes' answer will turn to rebuild filesystem, other modes will exit. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Handle orphan nodesZhihao Cheng
This is the 3/18 step of fsck. Handle orphan nodes, update TNC & LPT. There could be following steps and possible errors: Step 1. scan orphan LEB, get all orphan nodes a. corrupted scanning data in orphan area: danger mode and normal mode with 'yes' answer will drop orphan LEB, other modes will exit. Step 2. parse orphan node, find the original inode for each inum a. corrupted node searched from TNC: skip node for danger mode and normal mode with 'yes' answer, other modes will exit. b. corrupted index node read from TNC: danger mode with rebuild_fs and normal mode with 'yes' answer will turn to rebuild filesystem, other modes will exit. Step 4. remove inode for each inum, update TNC & LPT a. corrupted index node read from TNC: danger mode with rebuild_fs and normal mode with 'yes' answer will turn to rebuild filesystem, other modes will exit. b. corrupted lpt: Set %FR_LPT_CORRUPTED for lpt status. Ignore the error. c. incorrect lpt: Set %FR_LPT_INCORRECT for lpt status. Ignore the error. d. If lpt status is not empty, skip updating lpt. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Replay journalZhihao Cheng
This is the 2/18 step of fsck. Replay journal, update TNC & LPT. There could be following steps and possible errors: Step 1. scan log LEB, get all bud LEBs a. corrupted scanning data in log area: danger mode with rebuild_fs and normal mode with 'yes' answer will turn to rebuild filesystem, other modes will exit. Step 2. scan bud LEBs, get all nodes a. corrupted scanning data in bud LEB: danger mode and normal mode with 'yes' answer will drop bud LEB and set %FR_LPT_INCORRECT for lpt status, other modes will exit. Step 3. apply nodes, record latest isize into size_tree Step 4. apply nodes, update TNC & LPT a. corrupted data searched from TNC: skip node and set %FR_LPT_INCORRECT lpt status for danger mode and normal mode with 'yes' answer, other modes will exit. b. corrupted index node read from TNC: danger mode with rebuild_fs and normal mode with 'yes' answer will turn to rebuild filesystem, other modes will exit. c. corrupted lpt: Set %FR_LPT_CORRUPTED for lpt status. Ignore the error. d. incorrect lpt: Set %FR_LPT_INCORRECT for lpt status. Ignore the error. e. If lpt status is not empty, skip updating lpt. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Read master node & init lptZhihao Cheng
This is the 1/18 step of fsck. Read and check master node, init lpt. There could be following errors: 1. corrupted scanning data in master area or invalid master node: danger mode with rebuild_fs and normal mode with 'yes' answer will turn to rebuild filesystem, other modes will exit. 2. incorrect space statistics in master node: Set %FR_LPT_INCORRECT for for lpt status. Ignore the error. 3. corrupted lpt: Set %FR_LPT_CORRUPTED for lpt status. Ignore the error. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: rebuild_fs: Write master nodeZhihao Cheng
This is the 12/12 step of rebuilding. Since all meta areas are ready, master node can be updated. After this step, a consistent UBIFS image can be mounted, and it should pass all tests from chk_fs, chk_general, chk_index, chk_lprops and chk_orphans. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: rebuild_fs: Clean up log and orphan areaZhihao Cheng
This is the 11/12 step of rebuilding. Clean up log and orphan area, all nodes have been recovered, these two areas should be cleared, otherwise old content in journal/orphan could be replayed in next mounting. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: rebuild_fs: Build LPTZhihao Cheng
This is the 10/12 step of rebuilding. All LEBs' properties can be calculated in previous steps according to all nodes' position, then construct LPT just like mkfs does, and write LPT on flash. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: rebuild_fs: Build TNCZhihao Cheng
This is the 9/12 step of repairing. Construct TNC according to scanned files, and write TNC on flash, just like mkfs does. Building TNC can effectively solve many failed mounting problems caused by bad TNC (eg. bad node pointed by TNC, bad key order in znode, etc.). Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: rebuild_fs: Create new root dir if there are no scanned filesZhihao Cheng
This is a preparation for building TNC, there must at least one file in filesystem, if not, just create new root dir. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: rebuild_fs: Re-write dataZhihao Cheng
This is the 8/12 step of rebuilding. Re-write data. Read data from LEB and write back data, make sure that all LEB is ended with empty data(0xFF). It will prevent failed gc scanning in next mounting. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: rebuild_fs: Record used LEBsZhihao Cheng
This is the 7/12 step of rebuilding. Record used LEBs which may hold useful nodes, then left unused LEBs could be taken for storing new index tree. Notice, LEB that contains effective nodes on deleted trees in step 1 is regarded as used. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: rebuild_fs: Check and correct files' informationZhihao Cheng
This is the 6/12 step of rebuilding. Correct the file information. Traverse all files and calculate information (nlink, size, xattr_cnt, etc.) for each file just like check_leaf() does, correct inode node based on the calculated information. Now, all files are consistent, and UBIFS will pass chk_fs after mounting. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: rebuild_fs: Extract reachable directory entries treeZhihao Cheng
This is the 5/12 step of rebuilding. Extract reachable directory entries tree. Make sure that all files can be searched from '/', unreachable file is deleted. So, all files can be accessible in userspace after reparing. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: rebuild_fs: Filter invalid filesZhihao Cheng
This is the 4/12 step of rebuilding. Filter out invalid files and drop them, for example: 1. File has no inode node or inode nlink is zero 2. Nonconsistent file type between inode node and dentry nodes 3. File has no dentry nodes(excepts '/') 4. Encrypted file has no xattr information 5. Non regular file has data nodes 6. Directory/xattr file has more than one dentries 7. Xattr file has no host inode, or the host inode is a xattr ... Valid xattr file will be inserted into corresponding host file's subtree. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: rebuild_fs: Add valid nodes into fileZhihao Cheng
This is the 3/12 step of rebuilding. Generate file according to left valid inode nodes and dentry nodes. Based on the results from step 2, it is easy to understand: Step 2 has done: valid_inos - del_inos = left_inos valid_dents - del_dents = left_dents Step 3 should do: Traverse left_inos and left_dents, insert inode/dentry nodes into corresponding file. After that, all files are generated by scanning, the next thing to do is dropping invalid files(eg. nonconsistent file type between inode node and dentry nodes, file has no dentry nodes(excepts '/'), encrypted file has no xattr information, etc.), which will be done in next step. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: rebuild_fs: Remove deleted nodes from valid node treeZhihao Cheng
This is the 2/12 step of rebuilding. Traverse nodes from del_inos and del_dents trees, remove inode nodes and dentry nodes with smaller sqnum from valid trees. This step handles deleting case, for example, file A is deleted, deleted inode node and deleted dentry node are written, if we ignore the deleted nodes, file A can be recovered after rebuilding because undeleted inode node and undeleted dentry node can be scanned. There's an exception, if deleted inode node and deleted dentry node are reclaimed(by gc) after deletion, file A is recovered. UBIFS rebuild_fs cannot solve it, because the real existence information of nodes depends on TNC, but TNC should not be depended for UBIFS rebuild_fs. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Add rebuilding filesystem supportZhihao Cheng
Add rebuilding filesystem support. This is the 1/12 step of rebuilding. Collect files, valid inode nodes, deleted inode nodes, valid dentry nodes and deleted dentry nodes in kinds of trees by scanning nodes from flash. Corrupted nodes(eg. incorrect crc, bad inode size, bad dentry name length, etc.) are dropped during scanning. Larger sqnum node is picked when more than 1 nodes with same index. In this step, trun node and data nodes are put into corresponding file, inode/dentry nodes are put into four trees: valid_inos(nlink != 0), del_inos(nlink is 0), valid_dents(inum != 0), del_dents(inum is 0). Next step will process above four trees to deal deletion situations. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Add file organization realizationZhihao Cheng
In order to check the consistency of each file and the reachability of the whole dentry tree, fsck orginizes all nodes into files. And the final recovered file(xattr is treated as a file) is organized as: (rbtree, inum indexed) / \ file1 file2 / \ file3 file4 file { inode node // each file has 1 inode node dentry (sub rb_tree, sqnum indexed) // '/' has no dentries, // otherwise at least 1 // dentry is required. trun node // the newest one truncation node data (sub rb_tree, block number indexed) // Each file may have 0 // or many data nodes xattrs (sub rb_tree, inum indexed) // Each file may have 0 or // many xattr files } Each file from file rb_tree is constructed by scanning nodes(eg. inode, dentry, etc.) from the TNC or the UBI volume. File's xattrs will be initialized in subsequent steps. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Add node parsing functionsZhihao Cheng
Add parsing functions for each type of nodes, which will be used for checking the validity of raw node data while reading from TNC or scanning from UBIFS logical erase block. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Load filesystem information from UBI volumeZhihao Cheng
Load filesystem information from UBI volume (Similar to UBIFS mounting process), initialize kinds of buffers and read superblock. This is the base step for both fsck and rebuild_fs. Subsequent pacthes will complete this step by adding more steps(eg. read master, replay journal, etc.) which are only used in fsck. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Distinguish reasons when certain failures happenZhihao Cheng
Read failure caused by scanning corrupted data or invalid data members should be identified, because fsck can handle it. Updating lp failure caused by bad space statistics should be identified too, because fsck can handle it. Add eight callback functions to implement it for fsck: 1. set_failure_reason_callback: Record failure reasons when reading or parsing node failed, there are four reasons: a. FR_DATA_CORRUPTED: scanning corrupted data or invalid nodes found b. FR_TNC_CORRUPTED: invalid index nodes c. FR_LPT_CORRUPTED: invalid pnode/nnode d. FR_LPT_INCORRECT: invalid space statistics or invalid LEB properties 2. get_failure_reason_callback: get failure reasons 3. clear_failure_reason_callback: Clear the error which is caused by above reasons. 4. test_and_clear_failure_reason_callback: Check and clear the error which is caused by above reasons, if so, fsck will handle it according to specific situation. For example, fsck will drop data node rather than fails to return when reading failure is caused by DATA_CORRUPTED. For another example, journal replaying will continue rather than fails to return if updating lpt failure is caused by LPT_CORRUPTED. 5. set_lpt_invalid_callback: Set the invalid lpt status 6. test_lpt_valid_callback: Check whether the lpt is corrupted/incorrect, it should be invoked before updating lp, if lpt status is invalid, returns false (which means that caller should skip updating lp, because updating lp could trigger assertion failed in ubifs_change_lp). 7. can_ignore_failure_callback: Check whether the failure can be ignored, some inconsistent errors won't affect the fsck process, for example wrong space statistics can be fixed after traversing TNC, so failures caused by incorrect space statistics can be ignored. 8. handle_failure_callback: Check whether the failure can be handled, some inconsistent errors could be fixed by fsck, we have fix_problem to do that, but UBIFS needs a callback function to invoke it in common libs. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Add inconsistent problem handling asking functionZhihao Cheng
There are four dimensions to define the type of inconsistent problems: 1. fixable: Some inconsistent problems can't be fixed, for example corrupted superblock. Un-fixable problem will abort program. 2. must fix: Some inconsistent problems can be ignored(eg. incorrect isize), but some are not(eg. corrupted TNC), which will affect the subsequent fsck steps. 3. drop data: Some fixing methods will drop user data, which is unacceptable for safe mode. If it happens, fsck will be aborted. 4. need rebuild: Some inconsistent problems depends on rebuilding filesystem to be fixed(eg. corrupted master node, corrupted TNC). Define an asking function to handle above kinds of inconsistent problems. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysfsck.ubifs: Add fsck supportZhihao Cheng
Add basic process code for fsck.ubifs. There are following modes for fsck: 1. normal mode: Check the filesystem, ask user whether or not to fix the problem as long as inconsistent data is found during fs checking. 2. safe mode: Check and safely repair the filesystem, if there are any data dropping operations needed by fixing, fsck will fail. 3. danger mode: Answer 'yes' to all questions. There two sub modes: a) Check and repair the filesystem according to TNC, data dropping will be reported. If TNC/master/log is corrupted, fsck will fail. b) Check and forcedly repair the filesystem according to TNC, turns to rebuild filesystem if TNC/master/log is corrupted. Always make fsck succeed. 4. check mode: Make no changes to the filesystem, only check the filesystem. 5. rebuild mode: Scan entire UBI volume to find all nodes, and rebuild filesystem, always make fsck success. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysubifs-utils: open_ubi: Set errno if the target is not char deviceZhihao Cheng
Set errno if the target is not char device. It will be useful for fsck to print error message if open_ubi failed. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysubifs-utils: Replace ubifs related source code with linux kernel implementationZhihao Cheng
Replace ubifs related source code with the implementation of linux kernel. It makes userspace implementation be same with linux kernel, then fsck.ubifs can resuse the code. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysubifs-utils: Add descriptions for new lib files in libubifs/READMEZhihao Cheng
There are many linux kernel source files have been adapted into ubifs-utils, add descriptions for these source files in README. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
7 daysubifs-utils: Move ubifs-media.h in libubifsZhihao Cheng
Since ubifs-media.h is only used for ubifs-utils, move it under ubifs-utils/libubifs. Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>