Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance(main/libusb): Add integration with termux-usb -E #21620

Merged
merged 1 commit into from
Nov 10, 2024

Conversation

kyufie
Copy link
Contributor

@kyufie kyufie commented Oct 1, 2024

This commit adds the ability for libusb to access connected USB devices through file descriptor given by termux-usb -E, extending its usability for non-root users.

In the presence of TERMUX_USB_FD, libusb will try to read the fd number given and add it to the list of available devices, disregarding other discovery methods.
Otherwise it will try to use the usual /dev/bus/usb-based device discovery so that root users can still enjoy the benefits of having root access.

To test it, run termux-usb -r -E -e 'command' device_name. Make sure that command uses libusb.

@kyufie
Copy link
Contributor Author

kyufie commented Oct 1, 2024

Here's a little program that I made to help testing this library, compile with -lusb-1.0.
All that is needed is a USB drive.
Once it's running with the drive connected, the kernel driver should be detached from the drive and the drive should disappear from the operating system.

#include <libusb-1.0/libusb.h>
#include <stdlib.h>

int main(int argc, char *argv[])
{
    int status;
    size_t devnum;
    struct libusb_device **devlist;
    struct libusb_device_handle *devhdl;

    libusb_set_debug(NULL, 3);

    status = libusb_init(NULL);
    if (status < 0) {
        puts(libusb_error_name(status));
        return 1;
    }

    devnum = libusb_get_device_list(NULL, &devlist);
    printf("Number of device attached: %lu\n", devnum);
    if (devnum == 0)
        return 1;

    status = libusb_open(devlist[0], &devhdl);
    if (status < 0) {
        puts(libusb_error_name(status));
        return 1;
    }

    status = libusb_detach_kernel_driver(devhdl, 0);
    if (status < 0) {
        puts(libusb_error_name(status));
        return 1;
    }

    return EXIT_SUCCESS;
}

@hansm629
Copy link

hansm629 commented Oct 6, 2024

@kyufie
Hello sir, thank you for your hard work in development.
If I understand correctly, does this mean that now in the Termux environment, USB storage can be recognized without needing root access?

@twaik
Copy link
Member

twaik commented Oct 6, 2024

No. Termux does not have any special rights on system, it is a regular Android app and it can not access HID or Mass Storage devices using USB Host API.

@kyufie
Copy link
Contributor Author

kyufie commented Oct 7, 2024

@hansm629
As of now, it's only useful for accessing other device through ADB.

@IntinteDAO
Copy link
Contributor

@kyufie Hello sir, thank you for your hard work in development. If I understand correctly, does this mean that now in the Termux environment, USB storage can be recognized without needing root access?

But it seems to me that you can access files on a USB device without root, simply by symlinking the directory mounted to termux.

Not a good option, but a workaround if you mean it

@hansm629
Copy link

@IntinteDAO
If that's possible, I believe it would be a major turning point for computing in Termux.
The only thing I found lacking in the Termux computing environment was the inability to mount USB storage.

I really hope it becomes possible.

@twaik
Copy link
Member

twaik commented Oct 13, 2024

I really hope it becomes possible.

It will not be possible without root.

@IntinteDAO
Copy link
Contributor

IntinteDAO commented Oct 13, 2024

I really hope it becomes possible.

It will not be possible without root.

@IntinteDAO If that's possible, I believe it would be a major turning point for computing in Termux. The only thing I found lacking in the Termux computing environment was the inability to mount USB storage.

I really hope it becomes possible.

I was sure it could be done from within Termux directly, however, I actually have a problem with that, so I suspect I may had root.

On the other hand, other apps can access the USB flash drive (e.g. Ghost Commander), so it's probably a technically doable issue.

@twaik
Copy link
Member

twaik commented Oct 14, 2024

On the other hand, other apps can access the USB flash drive (e.g. Ghost Commander), so it's probably a technically doable issue.

They do not use usb api for that. They use storage API.

@IntinteDAO
Copy link
Contributor

On the other hand, other apps can access the USB flash drive (e.g. Ghost Commander), so it's probably a technically doable issue.

They do not use usb api for that. They use storage API.

I assume so. Is there an option for Termux to use Storage API to read data over USB or not? From what I understand, no (no one has implemented it).

@kyufie
Copy link
Contributor Author

kyufie commented Oct 14, 2024

On the other hand, other apps can access the USB flash drive (e.g. Ghost Commander), so it's probably a technically doable issue.

They do not use usb api for that. They use storage API.

I assume so. Is there an option for Termux to use Storage API to read data over USB or not? From what I understand, no (no one has implemented it).

I usually just read the available mount points and figure out the path to the external storage from it.
Then, I use that path to access the external storage directly.
As long as you have access to that path, this shouldn't be a problem. Works for me to say the least.
mount | awk '/\/mnt\/media_rw/ && split($3, a, /\//){print "/storage/" a[4]}'

@IntinteDAO
Copy link
Contributor

On the other hand, other apps can access the USB flash drive (e.g. Ghost Commander), so it's probably a technically doable issue.

They do not use usb api for that. They use storage API.

I assume so. Is there an option for Termux to use Storage API to read data over USB or not? From what I understand, no (no one has implemented it).

I usually just read the available mount points and figure out the path to the external storage from it. Then, I use that path to access the external storage directly. As long as you have access to that path, this shouldn't be a problem. Works for me to say the least. mount | awk '/\/mnt\/media_rw/ && split($3, a, /\//){print "/storage/" a[4]}'

For external storage there is support in Termux thanks to termux-setup-storage. Termux, on the other hand, does not have access to USB storage (without root), although it is theoretically possible for it to have (someone would have to implement some “driver” (?) fuse / gvfs.

@agnostic-apollo
Copy link
Member

@kyufie
Copy link
Contributor Author

kyufie commented Oct 14, 2024

Qemu USB passthrough is now possible thanks to this PR.
I managed to connect my USB flash drive to my Windows VM and access it from there.
Screenshot_2024-10-14-23-24-01-534_com realvnc viewer android
Screenshot_2024-10-14-23-31-56-178_com realvnc viewer android

termux-usb -r -E -e 'qemu-system-x86_64 -hda /sdcard/win7.qcow2 -m 4096 -accel tcg,thread=multi -smp 2 -device usb-ehci -device usb-host,hostbus=1,hostaddr=2 -vnc :0' /dev/bus/usb/001/002
Adjust hostbus and hostaddr according to the numbers present in /dev/bus/usb/...

In theory, this should permit unbounded access to USB devices without root access.

although it is theoretically possible for it to have (someone would have to implement some “driver” (?) fuse / gvfs.

This is probably the "driver" that you need.

I'm also thinking about other possibilities this might opens up (obtain root access without PC, arduino, all those things that people have been struggling with).

There's still a bit of a problem when the USB device accidentally disconnected from the host. When that happens, the VM must be restarted.

Someone seems to have already done it, but it was quite janky (#19635).

@hansm629
Copy link

@kyufie
It seems to be working! That's great news.

Ultimately, I hope that USB storage can be freely mounted in both the Termux XFCE4 desktop environment and PRoot Linux!

@kyufie
Copy link
Contributor Author

kyufie commented Oct 15, 2024

On the other hand, other apps can access the USB flash drive (e.g. Ghost Commander), so it's probably a technically doable issue.

They do not use usb api for that. They use storage API.

I assume so. Is there an option for Termux to use Storage API to read data over USB or not? From what I understand, no (no one has implemented it).

I usually just read the available mount points and figure out the path to the external storage from it. Then, I use that path to access the external storage directly. As long as you have access to that path, this shouldn't be a problem. Works for me to say the least. mount | awk '/\/mnt\/media_rw/ && split($3, a, /\//){print "/storage/" a[4]}'

For external storage there is support in Termux thanks to termux-setup-storage. Termux, on the other hand, does not have access to USB storage (without root), although it is theoretically possible for it to have (someone would have to implement some “driver” (?) fuse / gvfs.

@hansm629 If you're talking about access to the files inside the USB storage, as opposed to raw access, that one-liner I gave also works for USB drive.
USB drive counts as external storage too.

@hansm629
Copy link

@kyufie

  • libusb_1.0.27-1_aarch64.deb
  • libusb-static_1.0.27-1_aarch64.deb

After installing these two packages, which are currently in a pull request, do I just need to add the following environment variables and start XFCE4?

export mount | awk '/\/mnt\/media_rw/ && split($3, a, /\//){print "/storage/" a[4]}'

@kyufie
Copy link
Contributor Author

kyufie commented Oct 15, 2024

@hansm629
No that's not for envvar.
It's a command that gives you a list of path to the directory where your files in the external storage are located. You can cd into it, and do something just like you would on a regular directory.

$ mount | awk '/\/mnt\/media_rw/ && split($3, a, /\//){print "/storage/" a[4]}'
/storage/9016-4EF8 <-- External storage 1
/storage/ABCD-EFG <-- External storage 2
$ ls /storage/9016-4EF8
<Shows the contents of the external storage>

In the desktop environment, you should be able to type in the path in the file manager to go straight to that directory.

libusb is not required in this case.

@hansm629
Copy link

@kyufie
It is displaying as shown below. Is there something I am missing?

$ mount | awk '/\/mnt\/media_rw/ && split($3, a, /\//){print "/storage/" a[4]}'
/storage/7EEB-CB06
$ ls /storage/7EEB-CB06
"/storage/7EEB-CB06": No such file or directory (os error 2)

@kyufie
Copy link
Contributor Author

kyufie commented Oct 15, 2024

@hansm629
Perhaps the drive got disconnected by the time you use ls.
Also, if there's a prompt raised by an app asking to use the drive, reject it, that app may claim and detach the drive from the operating system.

@hansm629
Copy link

@kyufie
I still cannot access the storage. Are there any additional steps I should take?

The steps I followed are as follows:

  • Run Termux
  • Installed libusb_1.0.27-1/ibusb-static_1.0.27-1
  • Inserted the USB storage into the USB OTG
  • Ran the following command
~ $ mount | awk '/\/mnt\/media_rw/ && split($3, a, /\//){print "/storage/" a[4]}'
/storage/7EEB-CB06
~ $ ls /storage/7EEB-CB06
ls: cannot access '/storage/7EEB-CB06': No such file or directory

Is there anything wrong or missing from the steps I took?

Could the USB storage format also have an effect?

@kyufie
Copy link
Contributor Author

kyufie commented Oct 16, 2024

@hansm629
Not sure about that.
Can you do mount | grep 7EEB-CB06 with the drive connected?
And what Android version are you using?

@hansm629
Copy link

@kyufie

It is displayed as below.

~ $ mount | awk '/\/mnt\/media_rw/ && split($3, a, /\//){print "/storage/" a[4]}'
/storage/7EEB-CB06
~ $ mount | grep 7EEB-CB06
/dev/block/vold/public:8,65 on /mnt/media_rw/7EEB-CB06 type vfat (rw,nosuid,nodev,noexec,noatime,dirsync,gid=1077,fmask=0007,dmask=0007,allow_utime=0020,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)

This is my termux information.
The same symptom occurs on the Galaxy S20 Ultra, Android 13, and the Galaxy S24+, which is Android 14.

Termux Variables:
TERMUX_API_VERSION=0.50.1
TERMUX_APK_RELEASE=F_DROID
TERMUX_APP_PACKAGE_MANAGER=apt
TERMUX_APP_PID=8419
TERMUX_IS_DEBUGGABLE_BUILD=0
TERMUX_MAIN_PACKAGE_FORMAT=debian
TERMUX_VERSION=0.118.0
TERMUX__USER_ID=0
Packages CPU architecture:
aarch64
Subscribed repositories:
# sources.list
deb https://mirrors.nguyenhoang.cloud/termux/termux-main stable main
# tur-repo (sources.list.d/tur.list)
deb https://tur.kcubeterm.com tur-packages tur tur-on-device tur-continuous
# x11-repo (sources.list.d/x11.list)
deb https://packages-cf.termux.dev/apt/termux-x11/ x11 main
Updatable packages:
c-ares/stable 1.34.2 aarch64 [upgradable from: 1.34.1]
clang/stable 19.1.2 aarch64 [upgradable from: 19.1.1-1]
libcompiler-rt/stable 19.1.2 aarch64 [upgradable from: 19.1.1-1]
libllvm/stable 19.1.2 aarch64 [upgradable from: 19.1.1-1]
libwnck/x11 43.1 aarch64 [upgradable from: 43.0-3]
libxmlb/stable 0.3.21 aarch64 [upgradable from: 0.3.20]
lld/stable 19.1.2 aarch64 [upgradable from: 19.1.1-1]
llvm-tools/stable 19.1.2 aarch64 [upgradable from: 19.1.1-1]
llvm/stable 19.1.2 aarch64 [upgradable from: 19.1.1-1]
mlir/stable 19.1.2 aarch64 [upgradable from: 19.1.1-1]
termux-tools version:
1.43.6
Android version:
13
Kernel build information:
Linux localhost 4.19.113-27166950 #1 SMP PREEMPT Wed Aug 7 11:41:11 KST 2024 aarch64 Android
Device manufacturer:
samsung
Device model:
SM-G988N
LD Variables:
LD_LIBRARY_PATH=
LD_PRELOAD=/data/data/com.termux/files/usr/lib/libtermux-exec.so
Installed termux plugins:
com.termux.widget versionCode:13
com.termux.x11 versionCode:14
com.termux.api versionCode:51

@kyufie
Copy link
Contributor Author

kyufie commented Oct 16, 2024

@hansm629
Here's mine for comparison:

:: ~ % mount | awk '/\/mnt\/media_rw/ && split($3, a, /\//){print "/storage/" a[4]}'
/storage/9016-4EF8
/storage/4374-D373
 :: ~ % mount | grep 4374-D373
/dev/block/vold/public:8,97 on /mnt/media_rw/4374-D373 type exfat (rw,nosuid,nodev,noexec,noatime,dirsync,gid=1023,fmask=0007,dmask=0007,allow_utime=0020,iocharset=utf8,errors=remount-ro)
/dev/fuse on /mnt/installer/0/4374-D373 type fuse (rw,nosuid,nodev,noexec,noatime,lazytime,user_id=0,group_id=0,allow_other)
/dev/fuse on /mnt/androidwritable/0/4374-D373 type fuse (rw,nosuid,nodev,noexec,noatime,lazytime,user_id=0,group_id=0,allow_other)
/dev/fuse on /storage/4374-D373 type fuse (rw,nosuid,nodev,noexec,noatime,lazytime,user_id=0,group_id=0,allow_other)
/dev/fuse on /mnt/user/0/4374-D373 type fuse (rw,nosuid,nodev,noexec,noatime,lazytime,user_id=0,group_id=0,allow_other)
/dev/block/vold/public:8,97 on /mnt/pass_through/0/4374-D373 type exfat (rw,nosuid,nodev,noexec,noatime,dirsync,gid=1023,fmask=0007,dmask=0007,allow_utime=0020,iocharset=utf8,errors=remount-ro)
 :: ~ % mount | grep 9016-4EF8
/dev/fuse on /mnt/installer/0/9016-4EF8 type fuse (rw,nosuid,nodev,noexec,noatime,lazytime,user_id=0,group_id=0,allow_other)
/dev/fuse on /mnt/androidwritable/0/9016-4EF8 type fuse (rw,nosuid,nodev,noexec,noatime,lazytime,user_id=0,group_id=0,allow_other)
/dev/block/vold/public:179,1 on /mnt/media_rw/9016-4EF8 type vfat (rw,nosuid,nodev,noexec,noatime,dirsync,gid=1023,fmask=0007,dmask=0007,allow_utime=0020,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)
/dev/fuse on /mnt/user/0/9016-4EF8 type fuse (rw,nosuid,nodev,noexec,noatime,lazytime,user_id=0,group_id=0,allow_other)
/dev/block/vold/public:179,1 on /mnt/pass_through/0/9016-4EF8 type vfat (rw,nosuid,nodev,noexec,noatime,dirsync,gid=1023,fmask=0007,dmask=0007,allow_utime=0020,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)
/dev/fuse on /storage/9016-4EF8 type fuse (rw,nosuid,nodev,noexec,noatime,lazytime,user_id=0,group_id=0,allow_other)

I'm using Github debug build.

Termux Variables:
TERMUX_API_VERSION=0.50.1
TERMUX_APK_RELEASE=GITHUB
TERMUX_APP_PACKAGE_MANAGER=apt
TERMUX_APP_PID=27857
TERMUX_IS_DEBUGGABLE_BUILD=1
TERMUX_MAIN_PACKAGE_FORMAT=debian
TERMUX_VERSION=0.118.1
TERMUX__USER_ID=0
Packages CPU architecture:
aarch64
Subscribed repositories:
# sources.list
deb https://packages-cf.termux.dev/apt/termux-main stable main
# root-repo (sources.list.d/root.list)
deb https://packages-cf.termux.dev/apt/termux-root root stable
# x11-repo (sources.list.d/x11.list)
deb https://packages-cf.termux.dev/apt/termux-x11 x11 main
# glibc-repo (sources.list.d/glibc.list)
deb https://packages-cf.termux.dev/apt/termux-glibc/ glibc stable
Updatable packages:
binutils-bin/stable 2.43.1 aarch64 [upgradable from: 2.43]
binutils-gold/stable 2.43.1 aarch64 [upgradable from: 2.43]
binutils-libs/stable 2.43.1 aarch64 [upgradable from: 2.43]
boost/stable 1.83.0-3 aarch64 [upgradable from: 1.83.0-2]
busybox/stable 1.36.1-2 aarch64 [upgradable from: 1.36.1-1]
ca-certificates-glibc/glibc 2024.09.24 all [upgradable from: 2024.07.02]
ccls/stable 0.20240505 aarch64 [upgradable from: 0.20240202]
clang/stable 19.1.2 aarch64 [upgradable from: 18.1.8]
cmake-curses-gui/stable 3.30.5 aarch64 [upgradable from: 3.30.4]
cmake/stable 3.30.5 aarch64 [upgradable from: 3.30.4]
command-not-found/stable 2.4.0-46 aarch64 [upgradable from: 2.4.0-44]
curl/stable 8.10.1-1 aarch64 [upgradable from: 8.10.1]
gcc-libs-glibc/glibc 14.2.0 aarch64 [upgradable from: 14.1.0]
gdb/stable 15.2 aarch64 [upgradable from: 15.1]
gdbserver/stable 15.2 aarch64 [upgradable from: 15.1]
git/stable 2.47.0 aarch64 [upgradable from: 2.46.2]
glib/stable 2.80.5-2 aarch64 [upgradable from: 2.80.5]
imagemagick/stable 7.1.1.39 aarch64 [upgradable from: 7.1.1.38-1]
libarchive/stable 3.7.7 aarch64 [upgradable from: 3.7.6]
libcompiler-rt/stable 19.1.2 aarch64 [upgradable from: 18.1.8]
libcurl/stable 8.10.1-1 aarch64 [upgradable from: 8.10.1]
libllvm-static/stable 19.1.2 aarch64 [upgradable from: 18.1.8]
libllvm/stable 19.1.2 aarch64 [upgradable from: 18.1.8]
libltdl/stable 2.5.3 aarch64 [upgradable from: 2.4.7-4]
libluajit/stable 1:2.1.1727870382 aarch64 [upgradable from: 1:2.1.1727621189]
liblzma-glibc/glibc 5.6.3 aarch64 [upgradable from: 5.6.2]
liblzma/stable 5.6.3 aarch64 [upgradable from: 5.6.2]
libnghttp3/stable 1.6.0 aarch64 [upgradable from: 1.5.0]
libpolly/stable 19.1.2 aarch64 [upgradable from: 18.1.8]
libspice-server/stable 0.15.2 aarch64 [upgradable from: 0.15.1-1]
libtool/stable 2.5.3 aarch64 [upgradable from: 2.4.7-4]
libunbound/stable 1.21.1-1 aarch64 [upgradable from: 1.21.0]
libuv/stable 1.49.1 aarch64 [upgradable from: 1.49.0]
libxi/stable 1.8.2 aarch64 [upgradable from: 1.8.1]
libxml2/stable 2.13.4-1 aarch64 [upgradable from: 2.13.4]
lld/stable 19.1.2 aarch64 [upgradable from: 18.1.8]
llvm-tools/stable 19.1.2 aarch64 [upgradable from: 18.1.8]
llvm/stable 19.1.2 aarch64 [upgradable from: 18.1.8]
llvmgold/stable 19.1.2 aarch64 [upgradable from: 18.1.8]
lua-language-server/stable 3.11.1 aarch64 [upgradable from: 3.11.0]
matplotlib/stable 3.9.2-1 aarch64 [upgradable from: 3.9.1]
mesa/stable 24.0.8-1 aarch64 [upgradable from: 24.0.8]
mlir/stable 19.1.2 aarch64 [upgradable from: 18.1.8]
ncdu/stable 1.20-1 aarch64 [upgradable from: 1.20]
ncurses-ui-libs/stable 6.5.20240831-1 aarch64 [upgradable from: 6.5.20240831]
ncurses-utils/stable 6.5.20240831-1 aarch64 [upgradable from: 6.5.20240831]
ncurses/stable 6.5.20240831-1 aarch64 [upgradable from: 6.5.20240831]
neovim/stable 0.10.2-2 aarch64 [upgradable from: 0.10.1-1]
openjdk-17-x/stable 17.0-36 aarch64 [upgradable from: 17.0-35]
openjdk-17/stable 17.0-36 aarch64 [upgradable from: 17.0-35]
python-contourpy/stable 1.3.0-1 aarch64 [upgradable from: 1.2.1]
python-ensurepip-wheels/stable 3.12.7 all [upgradable from: 3.11.10]
python-numpy/stable 2.1.2-1 aarch64 [upgradable from: 1.26.5]
python-pillow/stable 11.0.0 aarch64 [upgradable from: 10.4.0]
python-pip/stable 24.2-1 all [upgradable from: 24.2]
python/stable 3.12.7 aarch64 [upgradable from: 3.11.10]
rhash/stable 1.4.5 aarch64 [upgradable from: 1.4.4]
rust-std-aarch64-linux-android/stable 1.81.0-1 all [upgradable from: 1.81.0]
rust/stable 1.81.0-1 aarch64 [upgradable from: 1.81.0]
termux-tools/stable 1.43.6 aarch64 [upgradable from: 1.43.5]
termux-x11-nightly/x11 1.03.01 all [upgradable from: 1.03.00-7]
tmux/stable 3.5a aarch64 [upgradable from: 3.5]
tree-sitter/stable 0.24.3 aarch64 [upgradable from: 0.23.1]
unbound/stable 1.21.1-1 aarch64 [upgradable from: 1.21.0]
vulkan-tools/stable 1.3.298 aarch64 [upgradable from: 1.3.296]
xkeyboard-config/x11 2.43-1 all [upgradable from: 2.42]
xz-utils-glibc/glibc 5.6.3 aarch64 [upgradable from: 5.6.2]
xz-utils/stable 5.6.3 aarch64 [upgradable from: 5.6.2]
termux-tools version:
1.43.5
Android version:
14
Kernel build information:
Linux localhost 5.15.94-android13-8-00001-g171753a15b59-ab11001737 #1 SMP PREEMPT Wed Oct 25 05:41:09 UTC 2023 aarch64 Android
Device manufacturer:
Xiaomi
Device model:
23021RAA2Y
LD Variables:
LD_LIBRARY_PATH=
LD_PRELOAD=/data/data/com.termux/files/usr/lib/libtermux-exec.so
Installed termux plugins:
com.termux.x11 versionCode:14
com.termux.api versionCode:51
com.termux.styling versionCode:31

Taking from Apollo's link.

MANAGE_EXTERNAL_STORAGE permission is declared in AndroidManifest.xml of the Termux app, but currently only available for GitHub action builds, but not F-Droid or GitHub release builds and will be available for them in next release. Currently, termux uses targetSdkVersion 28. F-Droid and GitHub release builds will not show All files access and GitHub action builds will show both Files and All fIles access.

You might need to update the app.

@kyufie
Copy link
Contributor Author

kyufie commented Oct 16, 2024

Also:

WRITE_EXTERNAL_STORAGE vs MANAGE_EXTERNAL_STORAGE
Enabling Android Settings -> Apps -> Termux -> Permissions -> Files will only grant legacy WRITE_EXTERNAL_STROAGE permission and enabling Android Settings -> Apps -> Special app access -> All files access -> Termux will only grant MANAGE_EXTERNAL_STORAGE permission. They are two different permissions, but access to primary external storage permission should still be available with either under the /storage/emulated/<user_id> or /sdcard paths.

However, granting MANAGE_EXTERNAL_STORAGE should additionally grant access to unreliable/removable volumes like USB OTG devices under the /mnt/media_rw/XXXX-XXXX paths on Android >= 12, check below. The termux-setup-storage command only requests the WRITE_EXTERNAL_STROAGE if the Termux app is using targetSdkVersion < 30, which is 28 currently for F-Droid and GitHub releases.

termux/termux-app#71 (comment)

@hansm629
Copy link

hansm629 commented Oct 16, 2024

@kyufie
Thank you so much for providing such a detailed response despite being busy. Should I try installing version 0.118.1 from the Termux-app GitHub release and test it again?

I’ll update you after I’ve done the update and tested it! :)

@agnostic-apollo
Copy link
Member

You need https://github.com/termux/termux-app/releases/tag/v0.119.0-beta.1 from github/fdroid for the MANAGE_EXTERNAL_STORAGE permission.

@hansm629
Copy link

@agnostic-apollo
Hello Sir,
I tried again after installing Termux v0.119.0-beta.1, but the same issue persists.

I activated the MANAGE_EXTERNAL_STORAGE permission and manually disabled the WRITE_EXTERNAL_STORAGE permission, but is there something else I might be missing?

Could it be that it's not working because it's without root?

Here is the termux-info:

Screenshot_20241018_002900_Settings
Screenshot_20241018_002929_Permission controller

$ mount | awk '/\/mnt\/media_rw/ && split($3, a, /\//){print "/storage/" a[4]}'
/storage/7EEB-CB06
$ ls /storage/7EEB-CB06
"/storage/7EEB-CB06": No such file or directory (os error 2)
Termux Variables:
TERMUX_APP_PACKAGE_MANAGER=apt
TERMUX_APP__APK_FILE=/data/app/~~xN6e9m3kSw38MFPf1yepwg==/com.termux-x6aA1bFU2_ANGtnHxSWsqg==/base.apk
TERMUX_APP__APK_RELEASE=GITHUB
TERMUX_APP__APP_VERSION_CODE=1020
TERMUX_APP__APP_VERSION_NAME=0.119.0-beta.1
TERMUX_APP__DATA_DIR=/data/user/0/com.termux
TERMUX_APP__IS_DEBUGGABLE_BUILD=true
TERMUX_APP__IS_INSTALLED_ON_EXTERNAL_STORAGE=false
TERMUX_APP__PACKAGE_NAME=com.termux
TERMUX_APP__PID=27696
TERMUX_APP__TARGET_SDK=28
TERMUX_VERSION=0.119.0-beta.1
TERMUX__SE_FILE_CONTEXT=u:object_r:app_data_file:s0:c127,c257,c512,c768
TERMUX__SE_INFO=default:targetSdkVersion=28:complete
TERMUX__SE_PROCESS_CONTEXT=u:r:untrusted_app_27:s0:c127,c257,c512,c768
TERMUX__UID=10383
TERMUX__USER_ID=0
Packages CPU architecture:
aarch64
Subscribed repositories:
# sources.list
deb https://mirrors.hust.edu.cn/termux/apt/termux-main stable main
# tur-repo (sources.list.d/tur.list)
deb https://tur.kcubeterm.com tur-packages tur tur-on-device tur-continuous
# x11-repo (sources.list.d/x11.list)
deb https://mirrors.hust.edu.cn/termux/apt/termux-x11 x11 main
Updatable packages:
All packages up to date
termux-tools version:
1.44.1
Android version:
14
Kernel build information:
Linux localhost 6.1.75-android14-11 #1 SMP PREEMPT Mon Aug 19 04:01:50 UTC 2024 aarch64 Android
Device manufacturer:
samsung
Device model:
SM-S926N
LD Variables:
LD_LIBRARY_PATH=
LD_PRELOAD=/data/data/com.termux/files/usr/lib/libtermux-exec.so
Installed termux plugins:
com.termux.widget versionCode:13
com.termux.x11 versionCode:14
com.termux.api versionCode:51

@kyufie
Copy link
Contributor Author

kyufie commented Oct 17, 2024

Try granting both permissions.

@hansm629
Copy link

@kyufie
Unfortunately, I've enabled both permissions and it still doesn't work...

I'm not sure what I'm missing... 😭

@agnostic-apollo
Copy link
Member

mount | awk '/\/mnt\/media_rw/ && split($3, a, /\//){print "/storage/" a[4]}'
/storage/7EEB-CB06
$ ls /storage/7EEB-CB06
"/storage/7EEB-CB06": No such file or directory

From the linked comment above.

Unreliable/Removable volumes like USB OTG devices that are only available on the /mnt/media_rw paths with their own filesystem (vfat/exfat) are assigned the root (0) owner and external_storage (1077) group.

You are getting an /mnt path and replacing it with a /storage path, which wouldn't exist as android never fuse mounts on it for usb drives like it does for sd cards.

@agnostic-apollo
Copy link
Member

There is also a script given to find the usb drive path properly.

termux/termux-app#71 (comment)

@hansm629
Copy link

@agnostic-apollo
I added the following script to ~/.bashrc, but I still don't have permission to access it.

Are there any parts of the script that need to be added or modified?

<~/.bashrc>

export LANG=ko_KR.UTF-8
export LC_MONETARY=ko_KR.UTF-8
export LC_PAPER=ko_KR.UTF-8
export LC_NAME=ko_KR.UTF-8
export LC_ADDRESS=ko_KR.UTF-8
export LC_TELEPHONE=ko_KR.UTF-8
export LC_MEASUREMENT=ko_KR.UTF-8
export LC_IDENTIFICATION=ko_KR.UTF-8
export LC_ALL=
export XDG_CONFIG_HOME=/data/data/com.termux/files/home/.config
export XMODIFIERS=@im=fcitx5
export GTK_IM_MODULE=fcitx5
export QT_IM_MODULE=fcitx5

find_most_recent_unreliable_storage_mount() {

    local return_value

    local i
    local storage_mounts_string
    local storage_mount
    local storage_mount_basename
    local unreliable_storage_mount
    local android_build_version_sdk
    local valid_number_regex='^[0-9]+$'

    local -a storage_mounts_array=()

    # Find android sdk/os version
    # Termux app exports ANDROID__BUILD_VERSION_SDK for `>= v0.119.0`
    if [[ ! "$ANDROID__BUILD_VERSION_SDK" =~ $valid_number_regex ]]; then
        android_build_version_sdk="$(unset LD_LIBRARY_PATH; unset LD_PRELOAD; getprop "ro.build.version.sdk")"
        return_value=$?
        if [ $return_value -ne 0 ] || [[ ! "$android_build_version_sdk" =~ $valid_number_regex ]]; then
            echo "Failure while finding \"ro.build.version.sdk\" property" 1>&2
            echo "ANDROID__BUILD_VERSION_SDK = \"$android_build_version_sdk\"" 1>&2
            if [ $return_value -eq 0 ]; then
                return_value=1
            fi
            return $return_value
        fi
    else
        android_build_version_sdk="$ANDROID__BUILD_VERSION_SDK"
    fi

    # If on Android `< 11`
    if [ "$android_build_version_sdk" -lt 30 ]; then
        echo "Cannot find unreliable storage mounts on Android sdk version '< 30'. Current sdk version is '$android_build_version_sdk'." 1>&2
        return 1
    fi

    storage_mounts_string="$(grep -E "^[^ ]+ /mnt/media_rw/[A-Z0-9]{4}-[A-Z0-9]{4} " /proc/mounts | cut -d' ' -f2)"
    return_value=$?
    if [ $return_value -ne 0 ]; then
        echo "Failure while finding most recent unreliable storage mount" 1>&2
        return $return_value
    fi

    if [ -z "$storage_mounts_string" ]; then
        echo "Failed to find any reliable or unreliable storage mounts" 1>&2
        return 1
    fi

    IFS=$'\n' read -r -d '' -a storage_mounts_array <<< "$storage_mounts_string"

    for (( i=${#storage_mounts_array[@]} - 1; i >= 0; i-- )) ; do
        storage_mount="${storage_mounts_array[i]}"
        storage_mount_basename="${storage_mount##*/}"

        grep -qE "^[^ ]+ /storage/$storage_mount_basename " /proc/mounts
        return_value=$?
        if [ $return_value -eq 1 ]; then
            unreliable_storage_mount="$storage_mount"
            break
        elif [ $return_value -ne 0 ]; then
            echo "Failure while finding if storage mount is unreliable" 1>&2
            return $return_value
        fi
    done

    if [ -z "$unreliable_storage_mount" ]; then
        echo "Failed to find any unreliable storage mounts" 1>&2
        return 1
    fi

    echo "$unreliable_storage_mount"

}

$ df | grep vold
/dev/block/vold/public:179,1 192219136 29112320 163106816  16% /mnt/media_rw/3232-6230

$ ls /mnt/media_rw/3232-6230
"/mnt/media_rw/3232-6230": Permission denied (os error 13)

Termux Variables:
TERMUX_APP_PACKAGE_MANAGER=apt
TERMUX_APP__APK_FILE=/data/app/~~uWpBhZ-vqdWJ__WFV9jslw==/com.termux-nm8zBMA-Cv4kSqJI8_UJbA==/base.apk
TERMUX_APP__APK_RELEASE=GITHUB
TERMUX_APP__APP_VERSION_CODE=1020
TERMUX_APP__APP_VERSION_NAME=0.119.0-beta.1
TERMUX_APP__DATA_DIR=/data/user/0/com.termux
TERMUX_APP__IS_DEBUGGABLE_BUILD=true
TERMUX_APP__IS_INSTALLED_ON_EXTERNAL_STORAGE=false
TERMUX_APP__PACKAGE_NAME=com.termux
TERMUX_APP__PID=20003
TERMUX_APP__TARGET_SDK=28
TERMUX_VERSION=0.119.0-beta.1
TERMUX__SE_FILE_CONTEXT=u:object_r:app_data_file:s0:c222,c257,c512,c768
TERMUX__SE_INFO=default:targetSdkVersion=28:complete
TERMUX__SE_PROCESS_CONTEXT=u:r:untrusted_app_27:s0:c222,c257,c512,c768
TERMUX__UID=10478
TERMUX__USER_ID=0
Packages CPU architecture:
aarch64
Subscribed repositories:
# sources.list
deb https://mirror.sunred.org/termux/termux-main stable main
# tur-repo (sources.list.d/tur.list)
deb https://tur.kcubeterm.com tur-packages tur tur-on-device tur-continuous
# x11-repo (sources.list.d/x11.list)
deb https://packages-cf.termux.dev/apt/termux-x11/ x11 main
Updatable packages:
termux-am/stable 0.8.0-1 all [upgradable from: 0.8.0]
termux-tools version:
1.44.1
Android version:
14
Kernel build information:
Linux localhost 5.15.123-android13-8-29539737-abX916NKOU4BXHB #1 SMP PREEMPT Fri Aug 30 12:32:27 UTC 2024 aarch64 Android
Device manufacturer:
samsung
Device model:
SM-X916N
LD Variables:
LD_LIBRARY_PATH=
LD_PRELOAD=/data/data/com.termux/files/usr/lib/libtermux-exec.so
Installed termux plugins:
com.termux.widget versionCode:13
com.termux.x11 versionCode:14
com.termux.api versionCode:51

@twaik
Copy link
Member

twaik commented Oct 20, 2024

This will require updating docs, @agnostic-apollo can you help with this after it gets merged please?

@twaik
Copy link
Member

twaik commented Oct 20, 2024

@kyufie I do not think you need TERMUX guard for this. This patch will be applied only in termux.

@hansm629
Copy link

@twaik
I succeeded! It works!

The external USB storage through USB OTG has been mounted on a without root device!

Both read and write operations are possible!

Although the volume doesn’t automatically appear in the Thunar file manager, I can access it in real-time by manually entering the path!

@kyufie @agnostic-apollo
Thank you so much for your help!

Screenshot_2024-10-20_19-41-39
7345
123
345

@kyufie
Copy link
Contributor Author

kyufie commented Oct 20, 2024

@twaik
That was a residue when the changes are still in my local libusb repo.
I'll remove that and do some more cleanup sometime later.

@hansm629
Copy link

@agnostic-apollo
Hello sir,
Thanks to your help, I was able to successfully mount USB storage via OTG on my non-root device!
I really appreciate it.

However, I have one issue I need to ask about.

Mount/unmount USB storage via OTG works perfectly, but I can't access the MicroSDXC card on the device due to a permission issue.

/mnt/media_rw/3232-6230 = MicroSDXC on the device
/mnt/media_rw/3EFF-C796 = USB storage mount via USB OTG adapter

Screenshot_2024-10-20_23-17-59

/dev/block/vold/public:179,1 192219136 29112576 163106560  16% /mnt/media_rw/3232-6230
/dev/block/vold/public:8,97   30013440  6582544  23430896  22% /mnt/media_rw/3EFF-C796

$ ls /mnt/media_rw/3232-6230
"/mnt/media_rw/3232-6230": Permission denied (os error 13)

$ ls /mnt/media_rw/3EFF-C796
.rwxrwx---  128 root  7 Oct  2021autorun.inf*
drwxrwx---    - root 21 Jan  2022boot/
.rwxrwx--- 414k root  7 Oct  2021bootmgr*
.rwxrwx--- 1.5M root  7 Oct  2021bootmgr.efi*
.rwxrwx--- 1.1G root 29 Jun 01:19  'COSTA RICA IN 4K@60fps VP9 HDR.webm'*
drwxrwx---    - root 21 Jan  2022efi/
drwxrwx---    - root 18 Oct  2023LOST.DIR/
.rwxrwx---  74k root  7 Oct  2021setup.exe*
drwxrwx---    - root 21 Jan  2022sources/
drwxrwx---    - root 21 Jan  2022support/
drwxrwx---    - root 21 Jan  2022  'System Volume Information'/
drwxrwx---    - root  7 Dec  2023포트폴리오/

Is there something I might be missing?
the method for accessing the MicroSDXC on the device different by any?

Below is the script entered in my .bashrc:

find_most_recent_unreliable_storage_mount() {

    local return_value

    local i
    local storage_mounts_string
    local storage_mount
    local storage_mount_basename
    local unreliable_storage_mount
    local android_build_version_sdk
    local valid_number_regex='^[0-9]+$'

    local -a storage_mounts_array=()

    # Find android sdk/os version
    # Termux app exports ANDROID__BUILD_VERSION_SDK for `>= v0.119.0`
    if [[ ! "$ANDROID__BUILD_VERSION_SDK" =~ $valid_number_regex ]]; then
        android_build_version_sdk="$(unset LD_LIBRARY_PATH; unset LD_PRELOAD; getprop "ro.build.version.sdk")"
        return_value=$?
        if [ $return_value -ne 0 ] || [[ ! "$android_build_version_sdk" =~ $valid_number_regex ]]; then
            echo "Failure while finding \"ro.build.version.sdk\" property" 1>&2
            echo "ANDROID__BUILD_VERSION_SDK = \"$android_build_version_sdk\"" 1>&2
            if [ $return_value -eq 0 ]; then
                return_value=1
            fi
            return $return_value
        fi
    else
        android_build_version_sdk="$ANDROID__BUILD_VERSION_SDK"
    fi

    # If on Android `< 11`
    if [ "$android_build_version_sdk" -lt 30 ]; then
        echo "Cannot find unreliable storage mounts on Android sdk version '< 30'. Current sdk version is '$android_build_version_sdk'." 1>&2
        return 1
    fi

    storage_mounts_string="$(grep -E "^[^ ]+ /mnt/media_rw/[A-Z0-9]{4}-[A-Z0-9]{4} " /proc/mounts | cut -d' ' -f2)"
    return_value=$?
    if [ $return_value -ne 0 ]; then
        echo "Failure while finding most recent unreliable storage mount" 1>&2
        return $return_value
    fi

    if [ -z "$storage_mounts_string" ]; then
        echo "Failed to find any reliable or unreliable storage mounts" 1>&2
        return 1
    fi

    IFS=$'\n' read -r -d '' -a storage_mounts_array <<< "$storage_mounts_string"

    for (( i=${#storage_mounts_array[@]} - 1; i >= 0; i-- )) ; do
        storage_mount="${storage_mounts_array[i]}"
        storage_mount_basename="${storage_mount##*/}"

        grep -qE "^[^ ]+ /storage/$storage_mount_basename " /proc/mounts
        return_value=$?
        if [ $return_value -eq 1 ]; then
            unreliable_storage_mount="$storage_mount"
            break
        elif [ $return_value -ne 0 ]; then
            echo "Failure while finding if storage mount is unreliable" 1>&2
            return $return_value
        fi
    done

    if [ -z "$unreliable_storage_mount" ]; then
        echo "Failed to find any unreliable storage mounts" 1>&2
        return 1
    fi

    echo "$unreliable_storage_mount"

}
df | grep vold

@agnostic-apollo
Copy link
Member

This will require updating docs, @agnostic-apollo can you help with this after it gets merged please?

Send a MediaWiki formatted text and I'll upload.

@agnostic-apollo
Copy link
Member

Welcome.

Read my post in the linked issue, external sd cards (reliable storages) are not accessible even with the permission, although some devices may allow access, like samsung. Only external hd (unreliable storages) will be accessible.

The find_most_recent_unreliable_storage_mount() is a shell function to get the most recent external hd mnt path, it does not give you special permissions, and adding it to .bashrc won't do anything alone, you need to call it by running find_most_recent_unreliable_storage_mount in a bash shell as a command and it will print the path so that you need to do df/vold shenanigans.

@hansm629
Copy link

@agnostic-apollo
Hello sir,
Thank you for your kind and detailed comment.

After checking with the command below, it seems that the USB storage connected via USB OTG is recognized, but the MicroSDXC card installed on the device is not recognized.

If it's displayed as shown below, should I assume that even with permissions, it's impossible to access the MicroSDXC card directly installed on the device (as opposed to via OTG)?

/dev/block/vold/public:179,1 192219136 29112576 163106560  16% /mnt/media_rw/3232-6230

$ find_most_recent_unreliable_storage_mount
Failed to find any unreliable storage mounts

@kyufie
Copy link
Contributor Author

kyufie commented Oct 21, 2024

@hansm629
I'm curious, can you replace /mnt/media_rw for your microSD card with /storage and see if it works?

@hansm629
Copy link

@kyufie
Thank you, sir!
Thanks to your help, I was able to resolve the issue!

By converting /mnt/media_rw/ to /storage/, I can now access it, and both reading and writing on the MicroSDXC card work perfectly!
I also pinned it as a bookmark in the Thunar file manager, so now I can easily access it anytime!
Screenshot_2024-10-21_11-50-14

@kyufie
Copy link
Contributor Author

kyufie commented Oct 22, 2024

That should be it.
Is it ready for merging?

Copy link
Member

@TomJo2000 TomJo2000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be good to merge.
I'd like to give it another day for final comments/reviews.

@TomJo2000
Copy link
Member

TomJo2000 commented Nov 10, 2024

Ah crap it looks like we have a rebase conflict.
I just tried to rebase the branch on top of the latest state from master and it's throwing up an error.
Let me see if I can find and resolve the issue.

Edit: manually rebasing did the job.
I'm just waiting for the CI to finish before merging.

This commit adds the ability for `libusb` to access connected USB device
through file descriptor given by `termux-usb -E`
@TomJo2000 TomJo2000 merged commit ce5b7b1 into termux:master Nov 10, 2024
7 checks passed
@TomJo2000
Copy link
Member

Thank you for your contribution @kyufie.

To avoid last minute rebasing errors in the future I would suggest periodically resyncing your development fork with the upstream master branch of termux/termux-packages.

This is how I do that for my fork:

# One time setup to add the upstream remote for fetching new commits.
git remote add upstream https://github.com/termux/termux-packages.git

# Then to resync the fork:
# On your fork's master branch
git fetch --all -v && git rebase upstream/master
# and to push that back to your development fork on GitHub:
git push

# You can then switch back to your dev branch and rebase it on top of local master,
# which is now up to date, and force push it to update your PR
git switch branch-name
git rebase master
git push --force-with-lease=branch-name 
# --force would also work, but force with lease is preferred to not accidentally clobber anything

You can probably set up git aliases or hooks to make this more convenient.

@kyufie
Copy link
Contributor Author

kyufie commented Nov 10, 2024

Thanks, I'll keep that in mind.

@landfillbaby
Copy link
Member

landfillbaby commented Nov 10, 2024

to make that happen by default, here's an excerpt from my ~/.gitconfig:

[branch]
	autosetupmerge = always
	autosetuprebase = always
[pull]
	rebase = true
[rebase]
	autostash = true

then git pull just rebases automatically

@TomJo2000
Copy link
Member

Did not know about branch autosetupmerge and autosetuprebase.
Thanks for the tip.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants