namjaejeon / linux-exfat-oot Goto Github PK

exFAT for Linux (Backport for low kernel version support)

Makefile 0.37% C 99.63%

linux-exfat exfat exfat-filesystem-driver filesystem linux-kernel

linux-exfat-oot's Introduction

exFAT filesystem

This is the exfat filesystem for support from the linux 5.15 kernel to the latest kernel. If you want to use exfat from 4.1 ~ 5.15 kernel, Please use #for-kernel-version-from-4.1.0 branch.

Installing as a stand-alone module

Install prerequisite package for Fedora, RHEL:

	yum install kernel-devel-$(uname -r)

Build step:

	make
	sudo make install

To load the driver manually, run this as root:

	modprobe exfat

Installing as a part of the kernel

Let's take [linux] as the path to your kernel source dir.

	cd [linux]
	cp -ar exfat [linux]/fs/

edit [linux]/fs/Kconfig

	source "fs/fat/Kconfig"
	+source "fs/exfat/Kconfig"
	source "fs/ntfs/Kconfig"

edit [linux]/fs/Makefile

	obj-$(CONFIG_FAT_FS)          += fat/
	+obj-$(CONFIG_EXFAT_FS)       += exfat/
	obj-$(CONFIG_BFS_FS)          += bfs/

make menuconfig and set exfat

	File systems  --->
		DOS/FAT/NT Filesystems  --->
			<M> exFAT filesystem support
			(utf8) Default iocharset for exFAT

build your kernel

linux-exfat-oot's People

Contributors

Stargazers

Watchers

Forkers

ptpt52 sbwml hclee shell-script mat-c tmm1 lxiaogao juanmiguelbg christianhaitian tmcofslp sticpaper snnbyyds i928 mavigsm schneith tsingui higarfield gmh5225

linux-exfat-oot's Issues

Ftruncate failed when the original file size = 0 and the extended part is not continuous clusters.

I built the exfat module with the latest code on mainline.
I encountered a bug during the bellow experiment:

steps to reproduce:

create 4G exfat filesystem.
create a file and truncate to 3G.
delete the file.
repeat step 2 again.
receive EIO, ftruncate failed.

Here is the script I used for testing:

#include <iostream>
#include <fcntl.h>
#include <unistd.h>

int main() {
    const char* filename = "example_file";
    off_t file_size = 3LL * 1024 * 1024 * 1024;

    // Open the file or create it if it doesn't exist
    int fd = open(filename, O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
    if (fd == -1) {
        std::cerr << "Error opening/creating the file" << std::endl;
        return 1;
    }

    // Set the file size using ftruncate
    if (ftruncate(fd, file_size) == -1) {
        std::cerr << "Error setting file size using ftruncate" << std::endl;
        close(fd);
        return 1;
    }

    // Cleanup
    close(fd);

    return 0;
}

and this is a patch I wrote to fix the bug:
0001-fix-bug-ftruncate-failed.patch
Please help check if this modification is correct and necessary

exfat read speed was only 22MB/S in 4.9.112+sdcard ，but in vfat it can be 80MB/S

HI namjaejeon：
Thanks for your code, I have tested your exfat code in qcom+4.9+sdcard，write speed was ok，but read speed was only about 22MB/S. Can you provide some suggestion about how to debug this,thank you again. one more question ,which exfat kernel version you based?

exfat disk usage of large file misreported

A large file in the exfat partition shows a very small disk-usage. This may be easy bug to fix. Seems like a 64bit number is truncated to lower 32bits in disk usage reporting.

https://bugzilla.kernel.org/show_bug.cgi?id=214701

https://gitlab.gnome.org/GNOME/baobab/-/issues/58

sorry for duplicate bug filing. I just did not know where the right place to file the bug was.

If you can't reproduce it yourself, let me know how to troubleshoot the file on my system which misreports its disk-usage to get the information needed.

delete large size files will stuck when SSD enable trim

When SSD enable trim(mount with discard),delete large size files will stuck.
The reason is sb_issue_discard cost long time in exfat_clear_bitmap.

Cannot compile on 5.4.52

make[3]: Entering directory '/home/mangix/devstuff/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/linux-mvebu_cortexa9/linux-5.4.52'
  CC [M]  /home/mangix/devstuff/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/linux-mvebu_cortexa9/exfat-5.8.4/super.o
/home/mangix/devstuff/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/linux-mvebu_cortexa9/exfat-5.8.4/super.c:296:9: error: macro "__fsparam" passed 5 arguments, but takes just 4
  296 |     NULL),
      |         ^
In file included from /home/mangix/devstuff/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/linux-mvebu_cortexa9/exfat-5.8.4/super.c:9:
./include/linux/fs_parser.h:118: note: macro "__fsparam" defined here
  118 | #define __fsparam(TYPE, NAME, OPT, FLAGS) \
      | 
/home/mangix/devstuff/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/linux-mvebu_cortexa9/exfat-5.8.4/super.c:295:2: error: '__fsparam' undeclared here (not in a function)
  295 |  __fsparam(NULL, "utf8",   Opt_utf8, fs_param_deprecated,
      |  ^~~~~~~~~
/home/mangix/devstuff/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/linux-mvebu_cortexa9/exfat-5.8.4/super.c:298:9: error: macro "__fsparam" passed 5 arguments, but takes just 4
  298 |     NULL),
      |         ^
In file included from /home/mangix/devstuff/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/linux-mvebu_cortexa9/exfat-5.8.4/super.c:9:
./include/linux/fs_parser.h:118: note: macro "__fsparam" defined here
  118 | #define __fsparam(TYPE, NAME, OPT, FLAGS) \
      | 
/home/mangix/devstuff/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/linux-mvebu_cortexa9/exfat-5.8.4/super.c:300:30: error: macro "__fsparam" passed 5 arguments, but takes just 4
  300 |     fs_param_deprecated, NULL),
      |                              ^
In file included from /home/mangix/devstuff/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/linux-mvebu_cortexa9/exfat-5.8.4/super.c:9:
./include/linux/fs_parser.h:118: note: macro "__fsparam" defined here
  118 | #define __fsparam(TYPE, NAME, OPT, FLAGS) \
      | 
/home/mangix/devstuff/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/linux-mvebu_cortexa9/exfat-5.8.4/super.c:302:30: error: macro "__fsparam" passed 5 arguments, but takes just 4
  302 |     fs_param_deprecated, NULL),
      |                              ^
In file included from /home/mangix/devstuff/openwrt/build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/linux-mvebu_cortexa9/exfat-5.8.4/super.c:9:
./include/linux/fs_parser.h:118: note: macro "__fsparam" defined here
  118 | #define __fsparam(TYPE, NAME, OPT, FLAGS) \

exfat driver may fail to write files with a size > 4GB when used in a 32bit kernel

The exfat driver fails to write files with a size > 4GB when these conditions are met:

The kernel is built for a 32bit system
The kernel config switch CONFIG_LBDAF is not set (i.e. no files > 2TB are supported)

We stumbled upon this when integrating the driver into our 32bit embedded Linux ecosystem.

The reason for this is an integer overflow when converting number of blocks into a file position, as for instance here:
https://github.com/namjaejeon/linux-exfat-oot/blob/5.14.1/inode.c#L338
Line:
pos = EXFAT_BLK_TO_B((iblock + 1), sb);
which translates to:
pos = (iblock + 1) << sb->s_blocksize_bits;

iblock is of type sector_t. Normally this is a 64bit integer, but on a 32bit system without CONFIG_LBDAF being set, it is just 32bit.
pos, on the other hand is a full 64bit integer

Unfortunately, the compiler performs the calculation in 32bit range without issuing a warning. As a result, an overflow will occur as soon as the file size reaches 4GB

A simple fix would be:
pos = EXFAT_BLK_TO_B((loff_t)(iblock + 1), sb);
This makes sure that the left side is cast to 64bit before shifting it.

Uninitialized return value on kernel < 4.16

In exfat_d_revalidate the ret variable is only initialized in

	if (dentry->d_parent->d_inode->i_version != exfat_d_version(dentry))
		ret = 0;

If this is false, ret is returned with an uninitialized value.

Exfat filesystem broken will case other files lost

(1). format the disk(SSD) on PC(win7);
(2). upload some files to the disk in directory A and umount from PC;
(3). mount the disk by linux-exfat-oot on arm device;
(4). upload files to disk on arm device to the directory A and poweroff when uploading not finished.
some files uploaded in step (2) will be lost when the arm device reboot and mount the disk by linux-exfat-oot.

release 5.12.3 failed to build on kernel 4.19

Kernel version:

$ uname -r
4.19.0-9-amd64

Build log:

$ make
make -C /lib/modules/4.19.0-9-amd64/build M=/home/CN_SZTL/linux-exfat-oot modules
make[1]: Entering directory '/usr/src/linux-headers-4.19.0-9-amd64'
  CC [M]  /home/CN_SZTL/linux-exfat-oot/inode.o
  CC [M]  /home/CN_SZTL/linux-exfat-oot/namei.o
  CC [M]  /home/CN_SZTL/linux-exfat-oot/dir.o
  CC [M]  /home/CN_SZTL/linux-exfat-oot/super.o
  CC [M]  /home/CN_SZTL/linux-exfat-oot/fatent.o
  CC [M]  /home/CN_SZTL/linux-exfat-oot/cache.o
  CC [M]  /home/CN_SZTL/linux-exfat-oot/nls.o
  CC [M]  /home/CN_SZTL/linux-exfat-oot/misc.o
  CC [M]  /home/CN_SZTL/linux-exfat-oot/file.o
  CC [M]  /home/CN_SZTL/linux-exfat-oot/balloc.o
/home/CN_SZTL/linux-exfat-oot/balloc.c: In function ‘exfat_trim_fs’:
/home/CN_SZTL/linux-exfat-oot/balloc.c:325:7: error: implicit declaration of function ‘fatal_signal_pending’; did you mean ‘mod_timer_pending’? [-Werror=implicit-function-declaration]
   if (fatal_signal_pending(current)) {
       ^~~~~~~~~~~~~~~~~~~~
       mod_timer_pending
cc1: some warnings being treated as errors
make[4]: *** [/usr/src/linux-headers-4.19.0-9-common/scripts/Makefile.build:309: /home/CN_SZTL/linux-exfat-oot/balloc.o] Error 1
make[3]: *** [/usr/src/linux-headers-4.19.0-9-common/Makefile:1537: _module_/home/CN_SZTL/linux-exfat-oot] Error 2
make[2]: *** [Makefile:146: sub-make] Error 2
make[1]: *** [Makefile:8: all] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-4.19.0-9-amd64'
make: *** [Makefile:21: all] Error 2

cannot copy files( few mbs) to SSD

namjaejeon,
Thanks for your work. I test read/write on memory stick. It works properly.
For 245G SSD, it can copy small files(few k) from SSD to internal SD, and copy small files from internal SD to SSD. It does not work for file size greater than few mb.
my phone is Pixel 3XL on lineage 20 with kernel 4.9.337 version.

linux-exfat-oot vs exfat-linux?

Hi, I am trying to understand how this driver is related to https://github.com/arter97/exfat-linux ?

Write failure when using fdatasync on linux-4.4

I was debugging an issue when using exfat on a linux 4.4 based system where write calls on a exfat partition returned -EIO.
I can reproduce it with the following short C program. https://gist.github.com/kuehnelth/5f217bd96efef09d4f2841de32b3ef06

After running the program the kernel prints

exFAT-fs (ubda): error, failed to bmap (inode : 000000007041b0a8 iblock : 0, err : -5)
exFAT-fs (ubda): Filesystem has been set read-only

After remounting both test files are gone so the fdatasyc doesn't seem to cause the new directory entries to be written to disk.
Without the fdatasync the program runs fine. On linux-4.9 it also works with fdatasync.

kernel 3.18.31 support possible?

TODO

FITRIM ioctl
NFS support
fallocate
symlink ?

Apr 27 17:45:06 dvr-server kernel: rcu: INFO: rcu_sched self-detected stall on CPU
Apr 27 17:45:06 dvr-server kernel: rcu:         0-....: (779176 ticks this GP) idle=9fe/1/0x4000000000000002 softirq=2500332/2500470 fqs=388572 
Apr 27 17:45:06 dvr-server kernel: rcu:          (t=777147 jiffies g=27281 q=27)
Apr 27 17:45:06 dvr-server kernel: Task dump for CPU 0:
Apr 27 17:45:06 dvr-server kernel: channels-dvr    R  running task        0   972      1 0x0000020a
Apr 27 17:45:06 dvr-server kernel: Call trace:
Apr 27 17:45:06 dvr-server kernel:  dump_backtrace+0x0/0x170
Apr 27 17:45:06 dvr-server kernel:  show_stack+0x24/0x30
Apr 27 17:45:06 dvr-server kernel:  sched_show_task+0x144/0x170
Apr 27 17:45:06 dvr-server kernel:  dump_cpu_task+0x48/0x60
Apr 27 17:45:06 dvr-server kernel:  rcu_dump_cpu_stacks+0x98/0xd4
Apr 27 17:45:06 dvr-server kernel:  rcu_check_callbacks+0x7e4/0x980
Apr 27 17:45:06 dvr-server kernel:  update_process_times+0x34/0x80
Apr 27 17:45:06 dvr-server kernel:  tick_sched_handle.isra.0+0x44/0x70
Apr 27 17:45:06 dvr-server kernel:  tick_sched_timer+0x5c/0xb0
Apr 27 17:45:06 dvr-server kernel:  __hrtimer_run_queues+0x150/0x3a0
Apr 27 17:45:06 dvr-server kernel:  hrtimer_interrupt+0xfc/0x260
Apr 27 17:45:06 dvr-server kernel:  arch_timer_handler_phys+0x3c/0x50
Apr 27 17:45:06 dvr-server kernel:  handle_percpu_devid_irq+0xa4/0x2b0
Apr 27 17:45:06 dvr-server kernel:  generic_handle_irq+0x34/0x50
Apr 27 17:45:06 dvr-server kernel:  __handle_domain_irq+0x98/0x110
Apr 27 17:45:06 dvr-server kernel:  gic_handle_irq+0x58/0xb0
Apr 27 17:45:06 dvr-server kernel:  el1_irq+0xb4/0x130
Apr 27 17:45:06 dvr-server kernel:  exfat_iget+0x58/0xd0 [exfat]
Apr 27 17:45:06 dvr-server kernel:  exfat_build_inode+0x30/0x2f0 [exfat]
Apr 27 17:45:06 dvr-server kernel:  exfat_lookup+0xf4/0x230 [exfat]
Apr 27 17:45:06 dvr-server kernel:  __lookup_slow+0x98/0x170
Apr 27 17:45:06 dvr-server kernel:  lookup_slow+0x44/0x70
Apr 27 17:45:06 dvr-server kernel:  walk_component+0x21c/0x330
Apr 27 17:45:06 dvr-server kernel:  path_lookupat.isra.0+0xa4/0x210
Apr 27 17:45:06 dvr-server kernel:  filename_lookup+0x9c/0x170
Apr 27 17:45:06 dvr-server kernel:  user_path_at_empty+0x58/0x70
Apr 27 17:45:06 dvr-server kernel:  vfs_statx+0x90/0x110
Apr 27 17:45:06 dvr-server kernel:  __se_sys_newfstatat+0x54/0x90
Apr 27 17:45:06 dvr-server kernel:  __arm64_sys_newfstatat+0x24/0x30
Apr 27 17:45:06 dvr-server kernel:  el0_svc_common+0x88/0x1b0
Apr 27 17:45:06 dvr-server kernel:  el0_svc_handler+0x38/0x80
Apr 27 17:45:06 dvr-server kernel:  el0_svc+0x8/0xc
Apr 27 17:45:06 dvr-server kernel: Task dump for CPU 3:
Apr 27 17:45:06 dvr-server kernel: kswapd0         R  running task        0    46      2 0x0000002a
Apr 27 17:45:06 dvr-server kernel: Call trace:
Apr 27 17:45:06 dvr-server kernel:  __switch_to+0xfc/0x160
Apr 27 17:45:06 dvr-server kernel:  release_pages+0x2e4/0x380
Apr 27 17:45:06 dvr-server kernel:  0x1
Apr 27 17:46:03 dvr-server kernel: rcu: INFO: rcu_preempt self-detected stall on CPU
Apr 27 17:46:03 dvr-server kernel: rcu:         3-....: (792899 ticks this GP) idle=156/1/0x4000000000000002 softirq=1025377/1025377 fqs=396449 
Apr 27 17:46:03 dvr-server kernel: rcu:          (t=792900 jiffies g=4621797 q=16324)
Apr 27 17:46:03 dvr-server kernel: Task dump for CPU 3:
Apr 27 17:46:03 dvr-server kernel: kswapd0         R  running task        0    46      2 0x0000002a
Apr 27 17:46:03 dvr-server kernel: Call trace:
Apr 27 17:46:03 dvr-server kernel:  dump_backtrace+0x0/0x170
Apr 27 17:46:03 dvr-server kernel:  show_stack+0x24/0x30
Apr 27 17:46:03 dvr-server kernel:  sched_show_task+0x144/0x170
Apr 27 17:46:03 dvr-server kernel:  dump_cpu_task+0x48/0x60
Apr 27 17:46:03 dvr-server kernel:  rcu_dump_cpu_stacks+0x98/0xd4
Apr 27 17:46:03 dvr-server kernel:  rcu_check_callbacks+0x7e4/0x980
Apr 27 17:46:03 dvr-server kernel:  update_process_times+0x34/0x80
Apr 27 17:46:03 dvr-server kernel:  tick_sched_handle.isra.0+0x44/0x70
Apr 27 17:46:03 dvr-server kernel:  tick_sched_timer+0x5c/0xb0
Apr 27 17:46:03 dvr-server kernel:  __hrtimer_run_queues+0x150/0x3a0
Apr 27 17:46:03 dvr-server kernel:  hrtimer_interrupt+0xfc/0x260
Apr 27 17:46:03 dvr-server kernel:  arch_timer_handler_phys+0x3c/0x50
Apr 27 17:46:03 dvr-server kernel:  handle_percpu_devid_irq+0xa4/0x2b0
Apr 27 17:46:03 dvr-server kernel:  generic_handle_irq+0x34/0x50
Apr 27 17:46:03 dvr-server kernel:  __handle_domain_irq+0x98/0x110
Apr 27 17:46:03 dvr-server kernel:  gic_handle_irq+0x58/0xb0
Apr 27 17:46:03 dvr-server kernel:  el1_irq+0xb4/0x130
Apr 27 17:46:03 dvr-server kernel:  queued_spin_lock_slowpath+0x9c/0x2e0
Apr 27 17:46:03 dvr-server kernel:  _raw_spin_lock+0x54/0x60
Apr 27 17:46:03 dvr-server kernel:  exfat_inode_tree_erase+0x34/0x70 [exfat]
Apr 27 17:46:03 dvr-server kernel:  exfat_evict_inode+0x4c/0x90 [exfat]
Apr 27 17:46:03 dvr-server kernel:  evict+0xa8/0x170
Apr 27 17:46:03 dvr-server kernel:  dispose_list+0x48/0x60
Apr 27 17:46:03 dvr-server kernel:  prune_icache_sb+0x6c/0xa0
Apr 27 17:46:03 dvr-server kernel:  super_cache_scan+0xf4/0x170
Apr 27 17:46:03 dvr-server kernel:  do_shrink_slab+0x150/0x3e0
Apr 27 17:46:03 dvr-server kernel:  shrink_slab+0xbc/0x2c0
Apr 27 17:46:03 dvr-server kernel:  shrink_node+0xd0/0x470
Apr 27 17:46:03 dvr-server kernel:  kswapd+0x394/0x850
Apr 27 17:46:03 dvr-server kernel:  kthread+0x104/0x130
Apr 27 17:46:03 dvr-server kernel:  ret_from_fork+0x10/0x1c

FITRIM not working

With 42d15a8 support for FITRIM was added, but when I try to use it with v5.14.1 it is not working for me:

$ uname -a
Linux tmm1-dvrpi4 5.10.17-v8 #1 SMP PREEMPT Mon Oct 11 19:20:05 UTC 2021 aarch64

$ dmesg | grep exfat
[    6.971824] exfat: loading out-of-tree module taints kernel.

$ lsmod | grep exfat
exfat                  86016  1

$ modinfo exfat
filename:       /lib/modules/5.10.17-v8/extra/exfat.ko
version:        5.14.1

$ mount | grep /media/DVR
/dev/sda9 on /media/DVR type exfat (rw,relatime,uid=501,gid=501,fmask=0000,dmask=0000,allow_utime=0022,iocharset=utf8,errors=remount-ro)

$ fstrim --verbose /media/DVR/
fstrim: /media/DVR/: the discard operation is not supported

$ strace -ttT fstrim --verbose /media/DVR/ 2>&1 | grep ioctl
12:57:21.535633 ioctl(3, FITRIM, {start=0, len=0xffffffffffffffff, minlen=0}) = -1 EOPNOTSUPP (Operation not supported) <0.000044>

cc @hyeongseok-kim901