[Bug] Incorrect handling of large files / storages

[alexcekay]

Describe the bug

Topic 1:
When handling large files (> 2 GB) unexpected behavior can occur.
The observed behavior in this case was an error reported due to: No space left on device when trying to open this file.

Topic 2:
Executing df -h will report an incorrect size when using an SD card with more than 4GB of storage.

To Reproduce

Topic 1:
Try to open a file that is larger than 2 GB. Unexpected errors will show up.

Topic 2:
Execute df -h. The reported size will not match the size of the storage.

Expected behavior

Topic 1:
Handling files larger than 2 GB shall be possible.

Topic 2:
The command df -h shall report the correct size of the storage.

Screenshot / Media

No response

Flight Log

No response

Software Version

nsh> ver all
HW arch: PX4_FMU_V5X
HW type: V5X
HW FMUM ID: 0x001
HW BASE ID: 0x009
PX4 git-hash: 3e3151c047587c4bef69289a211a9025a48d745e
PX4 version: 1.15.0 80 (17760384)
PX4 git-branch: main
OS: NuttX
OS version: Release 11.0.0 (184549631)
OS git-hash: e61fdd019de6ee7685c071760a965961c5ef5227
Build datetime: Nov 11 2024 13:40:44
Build uri: localhost
Build variant: default
Toolchain: GNU GCC, 13.3.1 20240614
PX4GUID: 000200000000203935534b5650020029001b
MCU: STM32F76xxx, rev. Z

Flight controller

No response

Vehicle type

None

How are the different components wired up (including port information)

No response

Additional context

No response

[alexcekay]

Both of the topics are linked to the same root cause being CONFIG_FS_LARGEFILE not being enabled in the NuttX KConfig.
As an example the output of df -h is the following without CONFIG_FS_LARGEFILE enabled:

nsh> df -h
  Filesystem    Size      Used  Available Mounted on
  binfs      0B        0B         0B /bin
  cromfs     216K      216K         0B /etc
  vfat       1844M     3488K      1840M /fs/microsd
  procfs     0B        0B         0B /proc

With CONFIG_FS_LARGEFILE enabled it is correct:

nsh> df -h
  Filesystem    Size      Used  Available Mounted on
  binfs      0B        0B         0B /bin
  cromfs     216K      216K         0B /etc
  vfat       29G     3488K        29G /fs/microsd
  procfs     0B        0B         0B /proc

So if you have this problem you can fix it by enabling CONFIG_FS_LARGEFILE in the NuttX KConfig.
This should only be done when this really causes an issue, as enabling it will cause a 3K flash increase
on a FMU v5x:

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +3.2%    +740   +22%    +598    fat/fs_fat32util.c
     +22%    +598   +22%    +598    .text
    +0.6%     +61  [ = ]       0    .debug_info
    +0.9%     +52  [ = ]       0    .debug_line
    +2.6%     +29  [ = ]       0    .debug_abbrev
  +1.8%    +556   +11%    +424    fat/fs_fat32.c
     +11%    +424   +11%    +424    .text
    +1.1%     +77  [ = ]       0    .debug_line
    +0.5%     +70  [ = ]       0    .debug_info
    -1.4%     -15  [ = ]       0    .debug_abbrev
  +0.1%    +388  [ = ]       0    [section .debug_frame]
  +3.4%    +386   +35%    +334    mtd/mtd_partition.c
     +35%    +334   +35%    +334    .text
    +3.7%     +86  [ = ]       0    .debug_line
    +1.1%     +10  [ = ]       0    .debug_abbrev
    -0.7%     -44  [ = ]       0    .debug_info
  +2.8%    +286  +5.2%     +32    ioexpander/gpio.c
    +3.2%    +179  [ = ]       0    .debug_info
    +6.2%     +72  [ = ]       0    .debug_abbrev
    +5.2%     +32  +5.2%     +32    .text
    +0.1%      +3  [ = ]       0    .debug_line
  +0.9%    +231  +5.3%    +188    fat/fs_fat32dirent.c
    +5.3%    +188  +5.3%    +188    .text
    +0.3%     +46  [ = ]       0    .debug_info
    -0.0%      -3  [ = ]       0    .debug_line
  +2.0%    +195   +20%    +136    mount/fs_procfs_mount.c
     +20%    +136   +20%    +136    .text
    +1.0%     +53  [ = ]       0    .debug_info
    +0.4%      +6  [ = ]       0    .debug_line
  +1.1%    +194  +5.5%     +88    romfs/fs_romfs.c
    +5.5%     +88  +5.5%     +88    .text
    +0.5%     +52  [ = ]       0    .debug_info
    +1.1%     +41  [ = ]       0    .debug_line
    +1.3%     +13  [ = ]       0    .debug_abbrev
  +1.0%    +155  +8.1%     +92    chip/stm32_bbsram.c
    +8.8%     +92  +8.8%     +92    .text
    +0.5%     +44  [ = ]       0    .debug_info
    +0.7%     +19  [ = ]       0    .debug_line
  +1.5%    +144   +13%    +100    mtd/ftl.c
     +13%    +100   +13%    +100    .text
    +2.8%     +54  [ = ]       0    .debug_line
    +3.7%     +36  [ = ]       0    .debug_abbrev
    -0.9%     -46  [ = ]       0    .debug_info
  +0.5%    +107  +6.1%     +72    /home/alex/PX4-Autopilot/src/modules/logger/util.cpp
    +6.1%     +72  +6.1%     +72    .text
    +0.2%     +25  [ = ]       0    .debug_info
    +0.3%     +10  [ = ]       0    .debug_line
  +1.0%     +88   +10%     +58    bch/bchdev_driver.c
     +10%     +58   +10%     +58    .text
    +0.5%     +24  [ = ]       0    .debug_info
    +0.4%      +6  [ = ]       0    .debug_line
  +0.5%     +86  +4.5%     +78    cromfs/fs_cromfs.c
    +4.5%     +78  +4.5%     +78    .text
    +0.2%     +15  [ = ]       0    .debug_info
    -0.2%      -7  [ = ]       0    .debug_line
  +1.0%     +84  +9.7%     +62    vfs/fs_dir.c
     +10%     +62   +10%     +62    .text
    +1.7%     +28  [ = ]       0    .debug_line
    -0.2%      -2  [ = ]       0    .debug_abbrev
    -0.1%      -4  [ = ]       0    .debug_info
  +1.2%     +79   +39%     +30    stdio/lib_ftello.c
    +0.9%     +44  [ = ]       0    .debug_info
     +39%     +30   +39%     +30    .text
    +0.6%      +5  [ = ]       0    .debug_line
  +0.3%     +76  +1.0%     +26    procfs/fs_procfsproc.c
    +0.4%     +49  [ = ]       0    .debug_info
    +1.0%     +26  +1.0%     +26    .text
    +0.0%      +1  [ = ]       0    .debug_line
  +0.1% +34.9Ki  +0.1% +3.02Ki    TOTAL 

[dagar]

We should start enabling it on non-flash constrained systems.