Unable to load libsteamworks4j.dylib on Apple Silicon M1 (new MAC hardware)

[cacophany53]

The following error occurs when attempting to run SteamAPI.loadLibraries() on MACs with Apple Silicon M1 chips (newer MAC hardware). I've attempted to rebuild the native libraries but still receive the same error while using them:

Exception in thread "main" java.lang.UnsatisfiedLinkError: Can't load library: /private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/steamworks4j/1.9.0/libsteamworks4j.dylib
at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2393)
at java.base/java.lang.Runtime.load0(Runtime.java:755)
at java.base/java.lang.System.load(System.java:1953)
at com.codedisaster.steamworks.SteamSharedLibraryLoader.loadLibrary(SteamSharedLibraryLoader.java:124)
at com.codedisaster.steamworks.SteamAPI.loadLibraries(SteamAPI.java:27)

Operating System: Mac OS X
Architecture: aarch64

[code-disaster]

The latest version on master contains two changes:

• Building fat libraries with support for M1 CPUs, see Arm64 support mac #107
• I cobbled a GitHub workflow this weekend which builds & deploys native libraries for all platforms

Would be cool if you give this a try, esp you with access to a Silicon M1. I don't have one in reach for testing myself.

[00-Evan]

I also have an M1 mac and would be happy to test this, but it doesn't look like 1.10.0-SNAPSHOT has been uploaded to sonatype. Should I build it from source or copy the newly built natives directly?

[code-disaster]

You should be fine to just build the Maven packages from source. The prebuilt libraries are up to date.

[00-Evan]

I built 1.10.0-SNAPSHOT and used it in an internal release on Steam (jpackage with an arm64 JDK). My game doesn't make extensive use of the steamworks APIs, but the game does launch and change the location of the notification popup correctly.

[nhouser9]

This seems to still be an issue for the latest builds available on gradle. Is there any way to include a version of this that works on the M1 in my build script?

[code-disaster]

You can run mvn package or mvn install on the latest version, then grab 1.10.0-SNAPSHOT.jar from java-wrapper\target or your local Maven repository.

[sebaber]

I try building locally with mvn and add manually the .jar and i have the next issue:

Execution failed for task ':desktop:compileJava'.
> Could not resolve all files for configuration ':desktop:compileClasspath'.
   > Could not find lwjgl-3.3.1-natives-linux.jar (org.lwjgl:lwjgl:3.3.1).
     Searched in the following locations:
         file:/Users/sebastianlemon/.m2/repository/org/lwjgl/lwjgl/3.3.1/lwjgl-3.3.1-natives-linux.jar
   > Could not find lwjgl-3.3.1-natives-linux-arm32.jar (org.lwjgl:lwjgl:3.3.1).
     Searched in the following locations:
         file:/Users/sebastianlemon/.m2/repository/org/lwjgl/lwjgl/3.3.1/lwjgl-3.3.1-natives-linux-arm32.jar
   > Could not find lwjgl-3.3.1-natives-linux-arm64.jar (org.lwjgl:lwjgl:3.3.1).
     Searched in the following locations:
         file:/Users/sebastianlemon/.m2/repository/org/lwjgl/lwjgl/3.3.1/lwjgl-3.3.1-natives-linux-arm64.jar
   > Could not find lwjgl-3.3.1-natives-macos.jar (org.lwjgl:lwjgl:3.3.1).
     Searched in the following locations:
         file:/Users/sebastianlemon/.m2/repository/org/lwjgl/lwjgl/3.3.1/lwjgl-3.3.1-natives-macos.jar
   > Could not find lwjgl-3.3.1-natives-windows.jar (org.lwjgl:lwjgl:3.3.1).
     Searched in the following locations:
         file:/Users/sebastianlemon/.m2/repository/org/lwjgl/lwjgl/3.3.1/lwjgl-3.3.1-natives-windows.jar
   > Could not find lwjgl-3.3.1-natives-windows-x86.jar (org.lwjgl:lwjgl:3.3.1).
     Searched in the following locations:
         file:/Users/sebastianlemon/.m2/repository/org/lwjgl/lwjgl/3.3.1/lwjgl-3.3.1-natives-windows-x86.jar

Possible solution:
 - Declare repository providing the artifact, see the documentation at https://docs.gradle.org/current/userguide/declaring_repositories.html

Anyone knows if i have to add another dependency?

This is my repositories declaration in build.gradle:

repositories {
      mavenLocal()
      mavenCentral()
      gradlePluginPortal()
      maven { url "https://oss.sonatype.org/content/repositories/snapshots/" }
      google()
  }

[sebaber]

Nevermind, it was a problem with my repositories declarations.

I follow the steps of run mvn install and copy .jar and works perfect! TYSM ❤️

[LobbyDivinus]

@code-disaster Do we need to do anything else? I installed arm64 versions of both Java and Maven and built the current state of the repo using mvn install

In Version 1.10.0 there seems to be a major change in that the loaders were put into separate libraries. Since I'm using libGDX I used the GDX loader like this:
SteamAPI.loadLibraries(new SteamLibraryLoaderGdx());

For it to work I had to add these artifacts into the gradle file (even though I'm not using the lwjgl3 loader, but not having it leads to the error message posted by sebaber):

compile files("libs/steamworks4j-1.10.0-SNAPSHOT.jar")
compile files("libs/steamworks4j-gdx-1.10.0-SNAPSHOT.jar")
compile files("libs/steamworks4j-lwjgl3-1.10.0-SNAPSHOT.jar")

However, the arm64 machine code seems to still not be included in the jar files:

java.lang.UnsatisfiedLinkError: /private/var/folders/8s/qythkrr54hx8dy6j51_z9yr00000gn/T/jna-103144406/jna12887211283955696318.tmp: dlopen(/private/var/folders/8s/qythkrr54hx8dy6j51_z9yr00000gn/T/jna-103144406/jna12887211283955696318.tmp, 0x0001): tried: '/private/var/folders/8s/qythkrr54hx8dy6j51_z9yr00000gn/T/jna-103144406/jna12887211283955696318.tmp' (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/private/var/folders/8s/qythkrr54hx8dy6j51_z9yr00000gn/T/jna-103144406/jna12887211283955696318.tmp' (no such file), '/private/var/folders/8s/qythkrr54hx8dy6j51_z9yr00000gn/T/jna-103144406/jna12887211283955696318.tmp' (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64'))
	at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)

[code-disaster]

fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64') sounds very wrong. Can you check this temp path and try to figure out what file that is?

Also, what libGDX version are you on? Might be worth a look to step into SharedLibraryLoader.mapLibraryName() and see if it picks the right version.

[LobbyDivinus]

Hi, thanks for the fast reply. I am using libGDX version 1.11.0.

Since I cannot do much about them here are the jna files:
jna.zip
From my understanding it creates a new one for every attempt to start the game.

In terms of stepping it fails in SharedLibraryLoader because libsteam_api.dylib doesn't exist in file:<..>/game/desktop/build/classes/java/main/

I guess it does execute some more code after that, but AndroidStudio doesn't find the context.

[code-disaster]

Oh this got me an idea. Please try building with mvn install -Psnapshot,libs -pl java-wrapper, then pick the new jar from your target folder. I believe it doesn't include the redistributable libraries with just mvn install.

[LobbyDivinus]

Ah, I apparently forgot to put the Steamworks SDK's files into the project folder. Loading of "steam_api" seems to work now.

However, after steam_api was loaded from /var/folders/8s/qythkrr54hx8dy6j51_z9yr00000gn/T/libgdxlobby/cad5c6cd/libsteam_api.dylib it fails to load libsteamworks4j.dylib from /var/folders/8s/qythkrr54hx8dy6j51_z9yr00000gn/T/libgdxlobby/ba1935a9/libsteamworks4j.dylib as it cannot resolve libsteam_api.dylib

java.lang.UnsatisfiedLinkError: /private/var/folders/8s/qythkrr54hx8dy6j51_z9yr00000gn/T/libgdxlobby/ba1935a9/libsteamworks4j.dylib: dlopen(/private/var/folders/8s/qythkrr54hx8dy6j51_z9yr00000gn/T/libgdxlobby/ba1935a9/libsteamworks4j.dylib, 0x0001): Library not loaded: @loader_path/libsteam_api.dylib
  Referenced from: <328AE6AD-085B-3FA2-9D45-ED471210EF84> /private/var/folders/8s/qythkrr54hx8dy6j51_z9yr00000gn/T/libgdxlobby/ba1935a9/libsteamworks4j.dylib
  Reason: tried: '/private/var/folders/8s/qythkrr54hx8dy6j51_z9yr00000gn/T/libgdxlobby/ba1935a9/libsteam_api.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS@loader_path/libsteam_api.dylib' (no such file), '/private/var/folders/8s/qythkrr54hx8dy6j51_z9yr00000gn/T/libgdxlobby/ba1935a9/libsteam_api.dylib' (no such file), '/usr/lib/libsteam_api.dylib' (no such file, not in dyld cache)

I wonder if it is supposed to search for libsteam_api.dylib after it was loaded successfully (if yes it has to fail due to the changed path).

Edit: Actually, it runs just fine after copying libsteam_api.dylib manually into the folder where libsteamworks4j.dylib gets copied to.

[LobbyDivinus]

Well, it really seems to be related to the Gdx library loader as using SteamLibraryLoaderLwjgl3 solves the issue completely.

In case someone needs the built jar files: steamworks4j-1.10.0-SNAPSHOT_fixed.zip
(I updated it to fix a bytecode issue described in #122)

I continued to encounter the Could not find lwjgl-3.3.1-natives-linux.jar (org.lwjgl:lwjgl:). compile error described by sebaber after building with maven. Deleting the .m2 folder in the user folder is a temporary fix for that.

[Rizato]

Want to add that this affects intel Macs as well.

I tried building my own loaders, based on the gdx-jnigen-loader, like in master, but I could not get it to work.

The only way I was able to get it working with the 1.57 SDK was to branch at 33ae54e87657afe8464eb01845e62b79403f5bfa (version 1.9.0 which has the old library loader) then cherry-pick 0756cbf2fc1948d5f2974a212c3f582969e3ac56, and f69b809ba03d808c40521ad3073c455b9525c462.