Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

addpkg(x11/luanti{-common,-server}): moved from TUR #22387

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

robertkirkman
Copy link
Contributor

@robertkirkman robertkirkman commented Nov 26, 2024

Copied and pasted from https://github.com/termux-user-repository/tur/tree/master/tur/luanti

Previous documentation here:

Summary of key points:

  • for better performance than normal virglrenderer-android, test using termux-wsi-layer or new virglrenderer

  • to use touchscreen input, set "Touchscreen input mode: Direct touch" in Termux:X11 settings

image

  • to use keyboard input, set "Prefer scancodes" in Termux:X11 settings

image

  • to use mouse input, set "Capture external pointer devices" in Termux:X11 settings and then click once with the mouse

image

New recommended Termux:X11 settings for Luanti
$ termux-x11-preference list
"hideCutout"="false"
"displayStretch"="false"
"notificationButton1Action"="exit"
"forceOrientation"="reverse landscape"
"adjustResolution"="false"
"tapToMove"="false"
"enableAccessibilityServiceAutomatically"="false"
"backButtonAction"="toggle soft keyboard"
"xrMode"="false"
"adjustHeightForEK"="false"
"notificationTapAction"="open preferences"
"showAdditionalKbd"="false"
"stylusButtonContactModifierMode"="false"
"additionalKbdVisible"="true"
"opacityEKBar"="100"
"displayResolutionCustom"="1280x1024"
"stylusIsMouse"="false"
"clipboardEnable"="true"
"extra_keys_config"="[['ESC','/',{key: '-', popup: '|'},'HOME','UP','END','PGUP','PREFERENCES'], ['TAB','CTRL','ALT','LEFT','DOWN','RIGHT','PGDN','KEYBOARD']]"
"pauseKeyInterceptingWithEsc"="false"
"preferScancodes"="true"
"storeSecondaryDisplayPreferencesSeparately"="false"
"showIMEWhileExternalConnected"="true"
"useTermuxEKBarBehaviour"="false"
"showMouseHelper"="false"
"transformCapturedPointer"="No"
"displayResolutionExact"="1280x1024"
"keepScreenOn"="true"
"pointerCapture"="true"
"swipeDownAction"="toggle additional key bar"
"touchMode"="Direct touch"
"dexMetaKeyCapture"="false"
"scaleTouchpad"="true"
"hardwareKbdScancodesWorkaround"="true"
"notificationButton0Action"="open preferences"
"volumeUpAction"="no action"
"fullscreen"="true"
"enableGboardCJK"="false"
"Reseed"="false"
"PIP"="false"
"capturedPointerSpeedFactor"="100"
"displayResolutionMode"="native"
"swipeUpAction"="no action"
"filterOutWinkey"="false"
"displayScale"="100"
"volumeDownAction"="no action"
"showStylusClickOverride"="false"
  • 6 patches are written entirely or mostly by me, but 0001-enable-egl-without-sdl2.patch is not, it is this commit cherrypicked. minetest/minetest@4838eb2

Note

In the future, Luanti might "transition back to SDL2" I think that a blocker for that in this build of luanti might be that building it with SDL2 prevented accessing EGL mode, at least for me. I have been building versions of SDL2 that might work with EGL when I test them, so that might come next eventually.

@Biswa96
Copy link
Member

Biswa96 commented Nov 26, 2024

  1. Some dependencies are used from bundled code. Should those be changed to system libraries?
  2. Should the common files/assets be moved to a separate platform independent package? It would be shared with all architectures and reduce some package size. Those similar steps were done with supertuxkart in TUR.

@robertkirkman
Copy link
Contributor Author

robertkirkman commented Nov 26, 2024

  1. not for Lua because in my experience, Lua bundled inside games must identically match the same version of Lua used by other clients of the same game, down to minor revision numbers. For other dependencies it is probably ok to use a Termux package. Which others did you notice?
  2. Yes AUR does that and has four packages, "client" "server" "common" and "documentation", but the whole package combined including all dependencies is only 70 MB extracted, compared to the 655+ MB of the supertuxkart packages combined which is very large. In luanti, the gamemodes and heavy assets are mostly downloaded dynamically through manual installation or the GUI. If you think it would still be best to separate the package into at least "client" "server" and "common", let me know and I will

@Biswa96
Copy link
Member

Biswa96 commented Nov 26, 2024

Which others did you notice?

According to previous build log https://github.com/termux/termux-packages/actions/runs/12031431167/job/33540963447#step:6:1054 those bundled dependencies were gmp, jsoncpp and luajit.

@robertkirkman
Copy link
Contributor Author

robertkirkman commented Nov 26, 2024

@Biswa96 Thank you for explaining about the dependencies, I did not notice those and I had a hard time getting the other dependencies marked correctly, so it is very helpful and now I have added those dependencies in the build.sh. I changed my mind about the Lua, it looks like Luanti is designed for Luajit and can use distro packages of it.

On the other hand, I checked the total size of the installation, and it does not seem to me to be way too large, but "too large" is very relative for each person, so it would be interesting to know what others think about it. pkg install luanti in a clean Termux installation shows

The following additional packages will be installed:
  brotli freetype libandroid-spawn libglvnd-dev libice libjpeg-turbo libogg libpng libsm libsqlite libuuid libvorbis
  libx11 libxau libxcb libxdmcp libxext libxi libxmu libxt openal-soft xorg-xauth
The following NEW packages will be installed:
  brotli freetype libandroid-spawn libglvnd-dev libice libjpeg-turbo libogg libpng libsm libsqlite libuuid libvorbis
  libx11 libxau libxcb libxdmcp libxext libxi libxmu libxt luanti openal-soft xorg-xauth
0 upgraded, 23 newly installed, 0 to remove and 0 not upgraded.
Need to get 13.0 MB of archives.
After this operation, 69.3 MB of additional disk space will be used.

I do see now that AUR's luanti-modern-client package depends on libxi and that luanti-modern-server does not, so I believe that could mean that a luanti-server package here could possibly be installed and run without running pkg install x11-repo first? Is it possible to make a subpackage accessible from a different repo than the main package?

I see that how it's done in openjdk-17 and opnejdk-17-x is that they are just both in the main repo, not the x11 repo,and openjdk-17-x has similar dependencies to what a luanti-client subpackage would require if I separated it here, so I am not completely sure how it should be organized.

TERMUX_SUBPKG_DESCRIPTION="Portion of openjdk-17 requiring X11 functionality"
TERMUX_SUBPKG_DEPENDS="freetype, giflib, libandroid-shmem, libjpeg-turbo, libpng, libx11, libxext, libxi, libxrender, libxtst"

@robertkirkman robertkirkman marked this pull request as draft November 26, 2024 23:48
@robertkirkman
Copy link
Contributor Author

robertkirkman commented Nov 26, 2024

I want the benchmarks, but they are not enabled by default, and neither is the server, I did not realize these things at first, so I will try creating subpackages like this:

  • luanti-client luanti
  • luanti-server
  • luanti-common
  • luanti-benchmarks

and I'll update this with how that goes. I am not sure if the package is suited to splitting in termux-packages, but it makes sense to try it and check what happens. EDIT: it turns out the benchmarks are built into the main binary, so they can't be a subpackage. three packages probably makes the most sense.

@robertkirkman robertkirkman changed the title addpkg(x11/luanti): moved from TUR ddpkg(x11/luanti{-common,-server}): moved from TUR Nov 27, 2024
@robertkirkman robertkirkman changed the title ddpkg(x11/luanti{-common,-server}): moved from TUR addpkg(x11/luanti{-common,-server}): moved from TUR Nov 27, 2024
@robertkirkman robertkirkman marked this pull request as ready for review November 27, 2024 03:25
# When cross-compiling assume the user doesn't want to run the executable anyway,
# otherwise place it in <source dir>/bin/ since Luanti can only run from there.
-if(NOT CMAKE_CROSSCOMPILING)
+if(NOT THIS_IS_FOR_SURFACEFLINGER_IN_OWOKITTYS_OPINION)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That one patch line raises several questions.
All of them some variation of "what?"

  • What are we actually patching here?
  • What compelled you to choose that as the variable name?
  • What does this have to do with SurfaceFlinger?

Copy link
Contributor Author

@robertkirkman robertkirkman Nov 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A better name for this change would be "DISABLING_THIS_BECAUSE_IT_IS_FOR_BUILDING_AN_APK" , and the idea of "SurfaceFlinger" originates from the fact that .apk files that launch ANativeWindow activities cannot be executed on devices where the program named /system/bin/surfaceflinger is not running in the background.

On the other hand, unlike the preexisting build of Minetest that can be found on the F-Droid store, which is a .apk file, this build targets X11 and therefore it can run connected to the X11 server from the package tigervnc inside termux-docker which is an environment where /system/bin/surfaceflinger is not present (as long as in that situation, a package like libopensles-standalone is also installed).

I will work on renaming this variable now.

Copy link
Contributor Author

@robertkirkman robertkirkman Nov 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TomJo2000 to explain an answer to your question "What are we actually patching here?" in a way that is hopefully complete, precise and does not leave out any details:

Normally, Luanti does not set the EXECUTABLE_OUTPUT_PATH variable when cross-compiling is detected, because the person who wrote that section assumed/knew that they do not officially cross-compile for any platforms that require EXECUTABLE_OUTPUT_PATH.

Later on elsewhere in their CMakeLists.txt, they have this logic: "if not ANDROID... then if UNIX... then create a symlink in the EXECUTABLE_OUTPUT_PATH". A different patch, 0003-bionic-libc-x11-wsi-opengl-es-2.patch, replaces the ANDROID variable there to force this codepath, but the reason this patch is not combined into that patch is because this patch is technically only required for cross-compiling, while building the package in non-cross-compiling mode does not actually require this change in order to work, so it seemed appropriate to organize them in separate patches.

https://github.com/minetest/minetest/blob/6a1d22b2c516abbb9ce9670dedff47451317706f/src/CMakeLists.txt#L1045-L1054

The error that I saved in the comment of the patch, is the error that happens if EXECUTABLE_OUTPUT_PATH does not exist when that linked point in the CMakeLists.txt is reached, which normally happens only when cross-compiling Luanti.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what's still in the main luanti package if the server and common files are split out as subpackages?

Copy link
Contributor Author

@robertkirkman robertkirkman Nov 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question, I will explain it like this, and if the packages need to be reorganized in a different way, this could be changed:

  • the luanti package contains the .desktop shortcut, the icon, and a program $PREFIX/bin/luanti that actually has both a client and a server inside it. It can be made to behave as a server by launching it with the command luanti --server. The problem is just that this binary depends on a lot of packages for the graphical client as well, libjpeg-turbo, libxi and others, so installing it requires installing those packages as runtime dependencies.

  • The luanti-server package contains only the program $PREFIX/bin/luantiserver, which when launched, only implements the headless server functionality and does not have the graphical client built inside it. This means it does not depend on libjpeg-turbo, libxi or the other packages that are in the dependency list of the luanti package, but not the luanti-server package. This would be the package for use by someone who just wants to open Termux and run the Luanti server to connect to it remotely from Luanti clients on other devices, and doesn't have a desktop environment installed inside Termux.

  • The luanti-common package contains platform-independent files that are necessary for both the client and the server.

@robertkirkman robertkirkman marked this pull request as draft November 27, 2024 16:29
@robertkirkman robertkirkman marked this pull request as ready for review November 27, 2024 18:40
@robertkirkman
Copy link
Contributor Author

robertkirkman commented Nov 28, 2024

Brief suggestions for layer packages to use for the time being, on each of these Android 8+ devices, and the FPS you might expect to get in fullscreen with default settings on them; I listed the best-functioning layer package I have seen so far for Luanti + Termux:X11 on each of these devices, so if someone is trying to play and happens to have the same one as me and is using the newest possible Android ROM available for each of them, they should try the one listed first unless there have been more updates to them since I posted this.

  • Galaxy S7 SM-G930U: termux-wsi-layer, ~20 fps
  • Galaxy S9 SM-G960U: termux-wsi-layer, ~35 fps
  • Galaxy S8+ SM-G955F: virglrenderer-android, ~15 fps
  • Galaxy A70 SM-A705FN: termux-wsi-layer, ~35 fps
  • Steam Deck OLED: virglrenderer-android, ~59 fps

@truboxl
Copy link
Contributor

truboxl commented Dec 2, 2024

I think its better to put SUBPKG_BREAKS and SUBPKG_REPLACES fields with old version main package in the subpackages to handle split

@robertkirkman
Copy link
Contributor Author

robertkirkman commented Dec 2, 2024

@truboxl I thought it might be a good idea as well, but I am not sure how to do it properly, maybe you have an idea of what is wrong here. In my test I tried this in luanti-common.subpackage.sh:

TERMUX_SUBPKG_INCLUDE="share/luanti/ share/man/ share/doc/"
TERMUX_SUBPKG_DESCRIPTION="Common files, including docs, shared between luanti and luanti-server"
TERMUX_SUBPKG_PLATFORM_INDEPENDENT=true
TERMUX_SUBPKG_BREAKS="luanti (<< 1:5.10.0-4)"
TERMUX_SUBPKG_REPLACES="luanti (<< 1:5.10.0-4)"

and also, separately and together, I tried this as well in luanti-server.subpackage.sh

TERMUX_SUBPKG_INCLUDE="bin/luantiserver bin/minetestserver"
TERMUX_SUBPKG_DESCRIPTION="Headless server for the Luanti voxel game engine."
TERMUX_SUBPKG_DEPENDS="jsoncpp, libandroid-spawn, libc++, libcurl, libgmp, libiconv, libpng, libsqlite, luajit, luanti-common, zstd"
TERMUX_SUBPKG_BREAKS="luanti (<< 1:5.10.0-4)"
TERMUX_SUBPKG_REPLACES="luanti (<< 1:5.10.0-4)"

However, unfortunately, if I build the packages with those changes, then I try to install them, this happens, which seems to prevent luanti-server from being installed without also installing the luanti package and pulling in all the same dependencies as the main package.

luanti-server depends luanti but it is not installable
~ $ pkg i ./luanti-common.deb 
No mirror or mirror group selected. You might want to select one by running 'termux-change-repo'
Checking availability of current mirror:
[*] https://mirrors.cbrx.io/apt/termux/termux-main: ok
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'luanti-common' instead of './luanti-common.deb'
The following package was automatically installed and is no longer required:
  xorg-xrandr
Use 'apt autoremove' to remove it.
The following packages will be REMOVED:
  luanti
The following NEW packages will be installed:
  luanti-common
0 upgraded, 1 newly installed, 1 to remove and 17 not upgraded.
Need to get 0 B/3344 kB of archives.
After this operation, 10.6 MB disk space will be freed.
Do you want to continue? [Y/n] ^C
~ $ pkg i ./luanti-server.deb 
No mirror or mirror group selected. You might want to select one by running 'termux-change-repo'
Checking availability of current mirror:
[*] https://mirrors.cbrx.io/apt/termux/termux-main: ok
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'luanti-server' instead of './luanti-server.deb'
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 luanti-server : Depends: luanti-common but it is not installable
                 Depends: luanti (= 1:5.10.0-4) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
~ $ pkg i ./luanti-server.deb ./luanti-common.deb 
No mirror or mirror group selected. You might want to select one by running 'termux-change-repo'
Checking availability of current mirror:
[*] https://mirrors.cbrx.io/apt/termux/termux-main: ok
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'luanti-server' instead of './luanti-server.deb'
Note, selecting 'luanti-common' instead of './luanti-common.deb'
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 luanti-server : Depends: luanti (= 1:5.10.0-4) but it is not installable
E: Unable to correct problems, you have held broken packages.
~ $ pkg i ./luanti-server.deb ./luanti-common.deb ./luanti.deb 
No mirror or mirror group selected. You might want to select one by running 'termux-change-repo'
Checking availability of current mirror:
[*] https://mirrors.cbrx.io/apt/termux/termux-main: ok
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'luanti-server' instead of './luanti-server.deb'
Note, selecting 'luanti-common' instead of './luanti-common.deb'
Note, selecting 'luanti' instead of './luanti.deb'
The following package was automatically installed and is no longer required:
  xorg-xrandr
Use 'apt autoremove' to remove it.
The following additional packages will be installed:
  luajit xdg-utils xorg-xprop
The following NEW packages will be installed:
  luajit luanti-common luanti-server xdg-utils xorg-xprop
The following packages will be upgraded:
  luanti
1 upgraded, 5 newly installed, 0 to remove and 17 not upgraded.
Need to get 324 kB/7374 kB of archives.
After this operation, 4370 kB of additional disk space will be used.
Do you want to continue? [Y/n] 

Is there a way to do it without the main package becoming a dependency of the subpackages?

@truboxl
Copy link
Contributor

truboxl commented Dec 2, 2024

Use TERMUX_SUBPKG_DEPEND_ON_PARENT=false if you think subpackage don't need the main package

@robertkirkman
Copy link
Contributor Author

robertkirkman commented Dec 3, 2024

Thank you very much that works, I have added that now.

I would like to explain that the reason why I have not added a TERMUX_PKG_SERVICE_SCRIPT variable to the luanti-server package to make a fast launcher with a name like luantid, is because Luanti requires browsing, choosing, downloading and installing a gamemode folder into ~/.minetest/games to launch the server, and prints a message with instructions for what to do if one is not present.

Arch Linux (AUR) does make the command luantiserver --gameid devtest work out of the box anyway by preinstalling the devtest gamemode with its luanti-modern-common package, but the reason I did not include that here is because if I did that, it would change the default title screen of the client to a checkerboard pattern with text "development test" and it's not immediately obvious that the white "+" at the bottom is for adding more gamemodes.

image

Instead of that (Arch Linux's default behavior), I thought the most natural and user-friendly default title screen for Termux users would be this one, so this is how I have it set currently - the button in the center leads to a menu where every gamemode, including devtest if needed, can be found and installed.

image

Copied and pasted from https://github.com/termux-user-repository/tur/tree/master/tur/luanti

Previous documentation here:
- termux-user-repository/tur#1295
- termux-user-repository/tur#1296
- termux-user-repository/tur#1331
- termux-user-repository/tur#1327

Summary of key points:

- for better performance than normal virglrenderer-android, test using [termux-wsi-layer](termux#22353) or [new virglrenderer](termux#22385)

- to use touchscreen input, set "Touchscreen input mode: Direct touch" in Termux:X11 settings

- to use keyboard input, set "Prefer scancodes" in Termux:X11 settings

- to use mouse input, set "Capture external pointer devices" in Termux:X11 settings

- 6 patches are unique to TUR/termux, but the patch 0001-enable-egl-without-sdl2.patch is
  a cherry pick of minetest/minetest@4838eb2
TERMUX_PKG_SRCURL=https://github.com/minetest/minetest/archive/refs/tags/${TERMUX_PKG_VERSION:2}.zip
TERMUX_PKG_SHA256=e74e994c0f1b188d60969477f553ad83b8ce20ee1e0e2dcd068120189cb0f56c
TERMUX_PKG_AUTO_UPDATE=true
TERMUX_PKG_DEPENDS="freetype, jsoncpp, libandroid-spawn, libc++, libcurl, libglvnd-dev, libgmp, libjpeg-turbo, libiconv, libpng, libsqlite, libvorbis, libxi, luajit, luanti-common, openal-soft, xdg-utils, zstd"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
TERMUX_PKG_DEPENDS="freetype, jsoncpp, libandroid-spawn, libc++, libcurl, libglvnd-dev, libgmp, libjpeg-turbo, libiconv, libpng, libsqlite, libvorbis, libxi, luajit, luanti-common, openal-soft, xdg-utils, zstd"
TERMUX_PKG_DEPENDS="freetype, jsoncpp, libandroid-spawn, libc++, libcurl, libgmp, libjpeg-turbo, libiconv, libluajit, libpng, libsqlite, libvorbis, libx11, libxi, luanti-common, openal-soft, opengl, xdg-utils, zlib, zstd"

@@ -0,0 +1,6 @@
TERMUX_SUBPKG_INCLUDE="bin/luantiserver bin/minetestserver"
TERMUX_SUBPKG_DESCRIPTION="Headless server for the Luanti voxel game engine."
TERMUX_SUBPKG_DEPENDS="jsoncpp, libandroid-spawn, libc++, libcurl, libgmp, libiconv, libpng, libsqlite, luajit, luanti-common, zstd"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
TERMUX_SUBPKG_DEPENDS="jsoncpp, libandroid-spawn, libc++, libcurl, libgmp, libiconv, libpng, libsqlite, luajit, luanti-common, zstd"
TERMUX_SUBPKG_DEPENDS="jsoncpp, libandroid-spawn, libc++, libcurl, libgmp, libiconv, libluajit, libsqlite, luanti-common, zlib, zstd"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants