lib: libmtd: fix sign extension of ioctl request in mtd_xlock

The req parameter of mtd_xlock() was declared as int.  On 64-bit
big-endian architectures such as ppc64el, _IOW()-derived ioctl numbers
have bit 31 set (because _IOC_WRITE=4 is placed at bit 29).  Storing
such a value in a signed int and then passing it to ioctl(), whose
second argument is unsigned long, causes implicit sign extension:
0x80084d05 becomes 0xffffffff80084d05, which does not match the
expected constant and fails the cmocka check_expected() assertion in
the unit tests for mtd_lock and mtd_unlock.

Fix by declaring req as unsigned long, matching the type used by the
ioctl() syscall interface.

Fixes: test_mtd_lock / test_mtd_unlock failures on ppc64el
Signed-off-by: Bastian Germann <bage@debian.org>
Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>

0 files changed, 0 insertions, 0 deletions