| Age | Commit message (Collapse) | Author |
|---|
|
This is a preparation for adding TNC checking support. Following data
structures and functions are moved into fsck.ubifs.c:
1. Move 'scanned_files' and 'used_lebs' from rebuild module, make them
resuable for non-rebuild_fs modes.
2. Move function 'handle_error' from load_fs.c, it could be reused in
other steps.
3. Add new function ubifs_tnc_remove_node in libubifs, which could
remove index entry for a node by given position.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This is the 5/18 step of fsck. Recover isize. There could be following
steps and possible errors:
Step 1. Traverse size tree, lookup corresponding inode from TNC
a. corrupted node searched from TNC: skip node for danger mode and
normal mode with 'yes' answer, other modes will exit.
b. corrupted index node read from TNC: danger mode with rebuild_fs and
normal mode with 'yes' answer will turn to rebuild filesystem, other
modes will exit.
Step 2. update isize for inode. Keep <inum, isize> in size tree for check
mode, update inode node in place for other modes.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This is the 4/18 step of fsck. Consolidate log to ensure enough space
in log area. There could be following possible errors:
1. corrupted scanning data in log area: danger mode with rebuild_fs and
normal mode with 'yes' answer will turn to rebuild filesystem, other
modes will exit.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This is the 3/18 step of fsck. Handle orphan nodes, update TNC & LPT.
There could be following steps and possible errors:
Step 1. scan orphan LEB, get all orphan nodes
a. corrupted scanning data in orphan area: danger mode and normal mode
with 'yes' answer will drop orphan LEB, other modes will exit.
Step 2. parse orphan node, find the original inode for each inum
a. corrupted node searched from TNC: skip node for danger mode and
normal mode with 'yes' answer, other modes will exit.
b. corrupted index node read from TNC: danger mode with rebuild_fs and
normal mode with 'yes' answer will turn to rebuild filesystem, other
modes will exit.
Step 4. remove inode for each inum, update TNC & LPT
a. corrupted index node read from TNC: danger mode with rebuild_fs and
normal mode with 'yes' answer will turn to rebuild filesystem, other
modes will exit.
b. corrupted lpt: Set %FR_LPT_CORRUPTED for lpt status. Ignore the
error.
c. incorrect lpt: Set %FR_LPT_INCORRECT for lpt status. Ignore the
error.
d. If lpt status is not empty, skip updating lpt.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This is the 2/18 step of fsck. Replay journal, update TNC & LPT.
There could be following steps and possible errors:
Step 1. scan log LEB, get all bud LEBs
a. corrupted scanning data in log area: danger mode with rebuild_fs and
normal mode with 'yes' answer will turn to rebuild filesystem, other
modes will exit.
Step 2. scan bud LEBs, get all nodes
a. corrupted scanning data in bud LEB: danger mode and normal mode with
'yes' answer will drop bud LEB and set %FR_LPT_INCORRECT for lpt
status, other modes will exit.
Step 3. apply nodes, record latest isize into size_tree
Step 4. apply nodes, update TNC & LPT
a. corrupted data searched from TNC: skip node and set %FR_LPT_INCORRECT
lpt status for danger mode and normal mode with 'yes' answer, other
modes will exit.
b. corrupted index node read from TNC: danger mode with rebuild_fs and
normal mode with 'yes' answer will turn to rebuild filesystem, other
modes will exit.
c. corrupted lpt: Set %FR_LPT_CORRUPTED for lpt status. Ignore the
error.
d. incorrect lpt: Set %FR_LPT_INCORRECT for lpt status. Ignore the
error.
e. If lpt status is not empty, skip updating lpt.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This is the 1/18 step of fsck. Read and check master node, init lpt.
There could be following errors:
1. corrupted scanning data in master area or invalid master node:
danger mode with rebuild_fs and normal mode with 'yes' answer will
turn to rebuild filesystem, other modes will exit.
2. incorrect space statistics in master node: Set %FR_LPT_INCORRECT for
for lpt status. Ignore the error.
3. corrupted lpt: Set %FR_LPT_CORRUPTED for lpt status. Ignore the error.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This is the 10/12 step of rebuilding. All LEBs' properties can be
calculated in previous steps according to all nodes' position, then
construct LPT just like mkfs does, and write LPT on flash.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This is the 8/12 step of rebuilding. Re-write data. Read data from
LEB and write back data, make sure that all LEB is ended with empty
data(0xFF). It will prevent failed gc scanning in next mounting.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This is the 6/12 step of rebuilding. Correct the file information.
Traverse all files and calculate information (nlink, size, xattr_cnt,
etc.) for each file just like check_leaf() does, correct inode node
based on the calculated information.
Now, all files are consistent, and UBIFS will pass chk_fs after mounting.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
This is the 4/12 step of rebuilding. Filter out invalid files and drop
them, for example:
1. File has no inode node or inode nlink is zero
2. Nonconsistent file type between inode node and dentry nodes
3. File has no dentry nodes(excepts '/')
4. Encrypted file has no xattr information
5. Non regular file has data nodes
6. Directory/xattr file has more than one dentries
7. Xattr file has no host inode, or the host inode is a xattr
...
Valid xattr file will be inserted into corresponding host file's subtree.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Add parsing functions for each type of nodes, which will be used for
checking the validity of raw node data while reading from TNC or
scanning from UBIFS logical erase block.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Read failure caused by scanning corrupted data or invalid data members
should be identified, because fsck can handle it. Updating lp failure
caused by bad space statistics should be identified too, because fsck
can handle it.
Add eight callback functions to implement it for fsck:
1. set_failure_reason_callback: Record failure reasons when reading or
parsing node failed, there are four reasons:
a. FR_DATA_CORRUPTED: scanning corrupted data or invalid nodes found
b. FR_TNC_CORRUPTED: invalid index nodes
c. FR_LPT_CORRUPTED: invalid pnode/nnode
d. FR_LPT_INCORRECT: invalid space statistics or invalid LEB properties
2. get_failure_reason_callback: get failure reasons
3. clear_failure_reason_callback: Clear the error which is caused by
above reasons.
4. test_and_clear_failure_reason_callback: Check and clear the error
which is caused by above reasons, if so, fsck will handle it
according to specific situation.
For example, fsck will drop data node rather than fails to return
when reading failure is caused by DATA_CORRUPTED.
For another example, journal replaying will continue rather than
fails to return if updating lpt failure is caused by LPT_CORRUPTED.
5. set_lpt_invalid_callback: Set the invalid lpt status
6. test_lpt_valid_callback: Check whether the lpt is corrupted/incorrect,
it should be invoked before updating lp, if lpt status is invalid,
returns false (which means that caller should skip updating lp, because
updating lp could trigger assertion failed in ubifs_change_lp).
7. can_ignore_failure_callback: Check whether the failure can be
ignored, some inconsistent errors won't affect the fsck process,
for example wrong space statistics can be fixed after traversing
TNC, so failures caused by incorrect space statistics can be ignored.
8. handle_failure_callback: Check whether the failure can be handled,
some inconsistent errors could be fixed by fsck, we have fix_problem
to do that, but UBIFS needs a callback function to invoke it in common
libs.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Add basic process code for fsck.ubifs. There are following modes for fsck:
1. normal mode: Check the filesystem, ask user whether or not to fix
the problem as long as inconsistent data is found during fs checking.
2. safe mode: Check and safely repair the filesystem, if there are any
data dropping operations needed by fixing, fsck will fail.
3. danger mode: Answer 'yes' to all questions. There two sub modes:
a) Check and repair the filesystem according to TNC, data dropping
will be reported. If TNC/master/log is corrupted, fsck will fail.
b) Check and forcedly repair the filesystem according to TNC, turns
to rebuild filesystem if TNC/master/log is corrupted. Always make
fsck succeed.
4. check mode: Make no changes to the filesystem, only check the
filesystem.
5. rebuild mode: Scan entire UBI volume to find all nodes, and rebuild
filesystem, always make fsck success.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Set errno if the target is not char device. It will be useful for
fsck to print error message if open_ubi failed.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
There are many linux kernel source files have been adapted into
ubifs-utils, add descriptions for these source files in README.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Since ubifs-media.h is only used for ubifs-utils, move it under
ubifs-utils/libubifs.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Adapt gc subsystem(find.c, gc.c, scan.c) in libubifs, compared with
linux kernel implementations:
1. Adapt print_hex_dump based on implementations in hexdump.c.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Adapt orphan.c in libubifs, compared with linux kernel implementations:
1. Keep the commit related implementations, because do_commit depends
on these functions which will be invoked in fsck.
2. Keep the orphan replaying implementations, because fsck needs them.
3. Other implementations are removed which won't be used in mkfs/fsck.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Adapt misc.h in libubifs, compared with linux kernel implementations:
1. Remove some functions(eg. ubifs_compr_name, ubifs_wake_up_bgt) which
won't be used in fsck/mkfs.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Adapt master.c in libubifs, compared with linux kernel implementations:
1. Remove authentication related implementations
(eg. mst_node_check_hash), authentication is not supported in fsck
for now.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Adapt key.h in libubifs, compared with linux kernel implementations:
1. Add '__unused' modifier for unused parameters to avoid compiling
warnings.
2. Remove some functions(eg. lowest_dent_key, dent_key_init_flash)
which won't be used in fsck/mkfs.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Adapt debug subsystem(debug.c, debug.h) in libubifs, compared with
linux kernel implementations:
1. Only keep the disk data and space statistics dumping implementations,
dbg_walk_index and add_size which are used by fsck, other debuging
related implementations and sysfs interfaces are removed, because
fsck will check fs in another way.
2. Change the behavior of ubifs_assert_failed(), make filesystem
readonly when assertion is failed.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Adapt commit.c in libubifs, compared with linux kernel implementations:
1. Remove debug related implementations(eg. dbg_check_old_index), debug
functions are not needed by fsck, because fsck will check fs in
another way.
2. Remove ubifs background committing related implementations, there
will be no background ubifs threads in fsck.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Adapt budget.c in libubifs, compared with linux kernel implementations:
1. Remove writeback related functions, there are no dirty pages/inodes
for UBIFS in userspace process.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Adapt journal.c in libubifs, compared with linux kernel implementations:
1. Remove all ubifs_jnl_XXX functions. Only keep the basic space
reservation code, fsck will add new functions for journaling
operations without using linux in-memory inode/dentry.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Adapt dir.c in libubifs, compared with linux kernel implementations:
1. Remove all functions. Only keep an empty source file, fsck will
add new functions for mkdir/link operations without using linux
in-memory inode/dentry.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Adapt auth.c in libubifs, compared with linux kernel implementations:
1. Only keep implementations used by mkfs, other implementations
are removed.
2. Adapt functions based on implementations in common/sign.c.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Adapt sb.c in libubifs, compared with linux kernel implementations:
1. Remove authentication related implementations
(eg. authenticate_sb_node), authentication is not supported in fsck
for now.
2. Remove some functions(eg. create_default_filesystem) which won't be
used in fsck/mkfs.
3. Modify ubifs_read_superblock(), remove some assignments which won't
be used in mkfs/fsck.
4. Apapt fixup_leb to ignore %-EBADMSG, subsequent steps will check
all areas carefully.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Adapt recovery subsystem(replay.c, recovery.c) in libubifs, compared with
linux kernel implementations:
1. Remove authentication related implementations
(eg. authenticate_sleb_hash), authentication is not supported in fsck
for now.
2. Add explicit type conversions(const char *) to avoid compiling
warnings.
3. Replace implementations of inode_fix_size() with ubifs_assert(0),
authentication is not supported in fsck, so this function won't
be invoked.
4. Remove unused ubifs_clean_lebs() and ubifs_write_rcvrd_mst_node().
5. Adapt fix_unclean_leb/recover_head/fix_size_in_place to ignore
%-EBADMSG, subsequent steps will check nodes in lpt/main area
carefully.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Adapt log.c in libubifs, compared with linux kernel implementations:
1. Remove debug related implementations(eg. dbg_check_bud_bytes), debug
functions are not needed by fsck, because fsck will check fs in
another way.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Adapt tnc subsystem(tnc.c,tnc_misc.c,tnc_commit.c) in libubifs, compared
with linux kernel implementations:
1. Remove debug related functions(eg. dbg_check_inode_size), debug
functions are not needed by fsck, because fsck will check fs in
another way.
2. Remove some functions(eg. ubifs_tnc_bulk_read) which won't be used
in fsck/mkfs.
3. Adapt tnc_delete and ubifs_search_zbranch to handle empty TNC case,
which could happen in fsck.
4. Don't skip checking the length of non-leaf index node's branch in
read_znode.
5. Adapt try_read_node to ignore %-EBADMSG, subsequent steps will check
nodes carefully.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Adapt lpt subsystem(lpt.c,lprops.c,lpt_commit.c) in libubifs, compared
with linux kernel implementations:
1. Remove debug related functions(eg. dbg_chk_lpt_sz, dbg_chk_pnode),
some of debug functions are not needed by fsck, because fsck will
check fs in another way.
2. Remove some functions(eg. ubifs_create_dflt_lpt) which won't be used
in fsck/mkfs.
3. Adapt do_calc_lpt_geom() to mkfs/fsck situations.
4. Adapt calc_dflt_lpt_geom to support the mkfs tool, and export it.
5. Adapt ubifs_create_lpt() according to create_lpt(mkfs), make sure that
the height of created lpt is right.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Adapt io.c in libubifs, compared with linux kernel implementations:
1. Modify io related functions(eg. ubifs_leb_read/ubifs_leb_write,
etc.), adapt them with userspace io functions lseek/read/write.
2. Remove some functions(eg. record_magic_error, ubifs_bg_wbufs_sync)
which won't be used in fsck/mkfs.
3. Replce ubifs_errc with ubifs_err, because there will be no SB_SILENT
options in mkfs/fsck.
4. Initiate wbuf->size as c->max_write_size.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Adapt super.c in libubifs, compared with linux kernel implementations:
1. It contains all definitions in common/super.c(Message printing
functions are replaced with linux kernel styles).
2. Remove some functions(eg. ubifs_iget, ubifs_dirty_inode) which won't
be used in fsck/mkfs.
3. Remove unused variables initialization in some functions(eg.
init_constants_early, init_constants_sb).
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Adapt ubifs header file in libubifs, compared with linux kernel
implementations:
1. New header file contains all definitions in common/ubifs.h
2. Remove some structures(eg. ubifs_mount_opts, bu_info) and
functions(eg. ubifs_sync_wbufs_by_inode, ubifs_jnl_XXX) which
won't be used in fsck/mkfs.
3. Modify some authentication related functions, keep functions
that mkfs may need, other functions are defined as empty.
4. Move ubifs message printing functions(ubifs_err/warn/msg) which
are originally defined in misc.c into ubifs.h.
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
|
Import UBIFS libs from linux kernel. Next patches will replace ubifs
related source code with implementation of linux kernel, which makes
userspace implementation be same with linux kernel, then fsck.ubifs
can resuse the code.
Notice: lpt.c is modified with [1] applied. ubifs.h and orphan.c are
modified with [2] applied, journal.c is modified with [3] reverted(
because fsck runs in a single thread, so waitqueue is not needed to
be implemented in userspace.).
[1] https://lore.kernel.org/linux-mtd/20231228014112.2836317-13-chengzhihao1@huawei.com/
[2] https://lore.kernel.org/linux-mtd/20240410073751.2522830-1-chengzhihao1@huawei.com/
[3] https://lore.kernel.org/linux-mtd/20240122063103.359501-1-chengzhihao1@huawei.com/
Signed-off-by: Zhihao Cheng <chengzhihao1@huawei.com>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
|
