mtd-utils.git, branch v2.1.1

Release mtd-utils-2.1.1

2019-07-21T20:40:28+00:00

Signed-off-by: David Oberhollenzer

mkfs.ubifs: fix description of favor_lzo

2019-07-18T09:45:37+00:00

"favor_lzo" uses "lzo" unless the space savings when using "zlib" are big. The current wording got this wrong. Signed-off-by: David Oberhollenzer

mkfs.ubifs: fix compilation without ZSTD

2019-07-16T05:53:33+00:00

Fix a trivial typo to make sure that zstd.h is included only if zstd is not disabled. Signed-off-by: Martin Kaiser Signed-off-by: David Oberhollenzer

mkfs.ubifs: remove ZSTD_CLEVEL_DEFAULT for backwards compatibillity

2019-07-09T10:34:04+00:00

Support for ZSTD compression has been added recently through the ZSTD library, which is famously known for its incredibly well designed and stable API. This patch removes usage of ZSTD_CLEVEL_DEFAULT, which isn't exposed in older versions of the ZSTD library, and replaces it with with the constant parameter 0. According to the documentation this should then use a reasonable default (which is defined internally). Other possible approachs include defining ZSTD_CLEVEL_DEFAULT to 3 (the value it _currently_ has) if it isn't defined. This patch chooses the approach of passing 0 since this seems to be encouraged by the existing documentation. Signed-off-by: David Oberhollenzer

ubinize: Exit with non-zero exit code on error.

2019-07-02T12:33:07+00:00

...specifically -1 in all of the new cases. Signed-off-by: Patrick Doyle Signed-off-by: David Oberhollenzer

ubi-tests: fm_param: Replace 'fm_auto' with 'fm_autoconvert'

2019-06-16T10:27:18+00:00

The value of fm_param should be 'fm_autoconvert' rather than 'fm_auto' when fastmap is supported by kernel. Currently, following verbose will appear in dmesg when fm_param is set to 'fm_auto': ubi: unknown parameter 'fm_auto' ignored This patch replace 'fm_auto' with 'fm_autoconvert' for fm_param, so ubi kernel module can receive correct parameters. ---------------------------------------- Signed-off-by: Zhihao Cheng Signed-off-by: David Oberhollenzer

ubi-tests: mkvol test: Checks return value 'ENOSPC' for 'ubi_mkvol'

2019-06-16T10:27:15+00:00

UBI tests try to create too many volumes in mkvol_bad and mkvol_basic. Currently mtd-utils allows return value 'ENFILE' from 'ubi_mkvol', that works fine in most situations. But what if the number of PEBs equals to the maximum count of volumes? For example, mkvol_basic test will fail in a 64MiB flash with 512KiB PEB size. Following is the output of mkvol_basic test: ====================================================================== ====================================================================== ====================================================================== Test on mtdram, fastmap enabled, VID header offset factor 1 ====================================================================== ====================================================================== ====================================================================== mtdram: 64MiB, PEB size 512KiB, fastmap enabled Running mkvol_basic /dev/ubi0 [mkvol_basic] mkvol_multiple():182: function ubi_mkvol() failed with error 28 (No space left on device) [mkvol_basic] mkvol_multiple():183: vol_id 122 Error: mkvol_basic failed FAILURE The reason is that there is no available PEB to support a new volume. We can see following verbose in dmesg: ubi0: attached mtd0 (name "mtdram test device", size 64 MiB) ubi0: user volume: 0, internal volumes: 1, max. volumes count: 128 ubi0: available PEBs: 122, total reserved PEBs: 6, PEBs reserved for bad PEB handling: 0 The maximum count of volumes is 128, so we can create 128 volumes theoretically. But there are 122 available PEBs becauese of existence of reserved PEBs. In addition, a volume occupies at least one PEB. Actually, we can only create 122 volumes, Therefore, 'ubi_mkvol' returns 'ENOSPC' when mkvol_basic tries to create 123rd volume. And we can see corresponding error message in dmesg: ubi0 error: ubi_create_volume [ubi]: not enough PEBs, only 0 available ubi0 error: ubi_create_volume [ubi]: cannot create volume 122, error -28 So, 'ENOSPC' can happen before 'ENFILE' in flash with a small amount of PEBs. This patch checks return value 'ENOSPC' for 'ubi_mkvol' when mkvol test is trying to create too many volumes. ---------------------------------------- Signed-off-by: Zhihao Cheng Signed-off-by: David Oberhollenzer

ubi-tests: io_read: Filter invalid offset value before 'lseek' in io_read test

2019-06-16T10:27:13+00:00

There are many different offset values passed in 'lseek' during io_read testing of ubi test. The offset value maybe a negative number or a big number that exceeds the volume data size, which can lead to ubi tests failure by passing invalid offset value to 'lseek'. For example: Example 1: The data size of volume is 39525 bytes, offset = (sz) - MAX_NAND_PAGE_SIZE - 1, where MAX_NAND_PAGE_SIZE is 65536. Here, offset is a negative value passed to 'lseek', which leads to fail in io_read. ====================================================================== ====================================================================== ====================================================================== Test on mtdram, fastmap enabled, VID header offset factor 1 ====================================================================== ====================================================================== ====================================================================== mtdram: 16MiB, PEB size 16KiB, fastmap enabled Running mkvol_basic /dev/ubi0 Running mkvol_bad /dev/ubi0 Running mkvol_paral /dev/ubi0 Running rsvol /dev/ubi0 Running io_basic /dev/ubi0 Running io_read /dev/ubi0 [io_basic] test_read3():189: function seek() failed with error 22 (Invalid argument) [io_basic] test_read3():190: len = 1 [io_basic] test_read2():237: offset = -26012 [io_basic] test_read1():303: length = 1 [io_basic] test_read():362: alignment = 7905 Error: io_read failed FAILURE Example 2: The data size of volume is 79035 bytes, offset = 2 * MAX_NAND_PAGE_SIZE, where MAX_NAND_PAGE_SIZE is 65536. Here, offset is a value exceeds volume size, which leads to fail in io_read. ====================================================================== ====================================================================== ====================================================================== Test on mtdram, fastmap enabled, VID header offset factor 1 ====================================================================== ====================================================================== ====================================================================== mtdram: 16MiB, PEB size 16KiB, fastmap enabled Running mkvol_basic /dev/ubi0 Running mkvol_bad /dev/ubi0 Running mkvol_paral /dev/ubi0 Running rsvol /dev/ubi0 Running io_basic /dev/ubi0 Running io_read /dev/ubi0 [io_basic] test_read3():185: function seek() failed with error 22 (Invalid argument) [io_basic] test_read3():186: len = 1 [io_basic] test_read2():233: offset = 131072 [io_basic] test_read1():299: length = 1 [io_basic] test_read():358: alignment = 3 Error: io_read failed FAILURE This patch checks offset value before executing 'lseek', invalid offset values are filtered. ---------------------------------------- Signed-off-by: Zhihao Cheng Signed-off-by: David Oberhollenzer

ubi-tests: ubi_mkvol_request: Fully initialize 'struct ubi_mkvol_request req'

2019-06-16T10:27:10+00:00

'struct ubi_mkvol_request req' is one parameter of the function 'ubi_mkvol' , this parameter will be passed to kernel and then be checked. It acts as a local variable in many ubi tests, such as io_basic, io_read, mkvol_bad, mkvol_basic, etc. After commit c355aa465fce ("ubi: expose the volume CRC check skip flag") in linux-stable, 'struct ubi_mkvol_request' supports a new configuration named 'flags', and req.flags will be checked in kernel function 'verify_mkvol_req'. Currently, there is no initialization for req.flags before 'ubi_mkvol' invoked. So, req.flags can be an arbitrary number passed to kernel. When we run ubi tests in qemu (x86_64, kernel image: 5.2.0-rc4), the following errors may occur: ====================================================================== ====================================================================== ====================================================================== Test on mtdram, fastmap enabled, VID header offset factor 1 ====================================================================== ====================================================================== ====================================================================== mtdram: 16MiB, PEB size 16KiB, fastmap enabled Running mkvol_basic /dev/ubi0 Running mkvol_bad /dev/ubi0 [mkvol_bad] test_mkvol():105: ubi_mkvol failed with error 22 (Invalid argument), expected 28 (No space left on device) [mkvol_bad] test_mkvol():105: bytes = 16060929 Error: mkvol_bad failed FAILURE This patch fully initializes every 'struct ubi_mkvol_request req' passed to 'ubi_mkvol', which can fix the bug that the ubi test failed caused by that req.flags was not initialized. And it is still compatible with old kernel before kernel commit c355aa465fce ("ubi: expose the volume CRC check skip flag"). ---------------------------------------- Signed-off-by: Zhihao Cheng Signed-off-by: David Oberhollenzer

mkfs.ubifs: Add ZSTD compression

2019-06-01T17:38:44+00:00

I added ZSTD support to mkfs.ubifs and compared the ZSTD results with zlib/lzo and the available ZSTD compression levels. The results are in the following table: Comp image MiB time image2 MiB time none 271 0m 0,723s 223 0m 0,589s lzo 164 0m13,705s 116 0m11,636s zlib 150 0m 7,654s 103 0m 6,347s favor-lzo 158 0m21,137s 110 0m17,764s zstd-01 154 0m 1,607s 106 0m 1,429s zstd-02 153 0m 1,704s 105 0m 1,479s zstd-03* 152 0m 1,888s 104 0m 1,668s zstd-04 151 0m 2,741s 103 0m 2,391s zstd-05 150 0m 3,257s 102 0m 2,916s zstd-06 150 0m 3,735s 102 0m 3,356s zstd-07 150 0m 4,066s 102 0m 3,705s zstd-08 152 0m 1,857s 104 0m 1,644s zstd-09 152 0m 1,855s 104 0m 1,639s zstd-10 150 0m 6,654s 102 0m 6,195s zstd-11 150 0m10,027s 102 0m 9,130s zstd-12 149 0m14,724s 101 0m13,415s zstd-13 148 0m18,232s 100 0m16,719s zstd-14 148 0m20,859s 100 0m19,554s zstd-15 148 0m25,033s 100 0m23,186s zstd-16 148 0m38,837s 100 0m36,543s zstd-17 148 0m46,051s 100 0m43,120s zstd-18 148 0m49,157s 100 0m45,807s zstd-19 148 0m49,421s 100 0m45,951s zstd-20 148 0m51,271s 100 0m48,030s zstd-21 148 0m51,015s 100 0m48,676s zstd-22 148 0m52,575s 100 0m50,013s The UBIFS image was created via mkfs.ubifs -x $Comp -m 512 -e 128KiB -c 2200 -r $image $out I used "debootstrap sid" to create a basic RFS and the results are in the `image' column. The image2 column denotes the results for the same image but with .deb files removed. The time column contains the output of the run time of the command. ZSTD's compression level three is currently default. Based on the compression results (for the default level) it outperforms LZO in run time and compression and is almost as good as ZLIB in terms of compression but quicker. The higher compression levels make almost no difference in compression but take a lot of time. The compression level used is the default offered by ZSTD. It does not make sense the higher levels. Signed-off-by: Sebastian Andrzej Siewior Signed-off-by: David Oberhollenzer