Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let workspace build on arm64 / M1 macs #313

Closed
afranken opened this issue Nov 23, 2021 · 13 comments
Closed

Let workspace build on arm64 / M1 macs #313

afranken opened this issue Nov 23, 2021 · 13 comments
Assignees

Comments

@afranken
Copy link
Member

Finally got my M1.
We knew that the Docker container we publish is x86_64 only, which we intend to fix.

Now I see that this workspace does not even build on arm64...
Seems like fabric8 maven plugin bundles some native x86_64 dependencies.

[ERROR] 
[ERROR] -----------------------------------------------------
[ERROR] : could not get native definition for type `POINTER`, original error message follows: java.lang.UnsatisfiedLinkError: Unable to execute or load jffi binary stub from `/var/folders/c8/wddjl72s5vsbz4jzf_3m_w000000gn/T/`. Set `TMPDIR` or Java property `java.io.tmpdir` to a read/write path that is not mounted "noexec".
[ERROR] /Users/arnefranken/github/adobe/S3Mock/jffi833879770895826780.dylib: dlopen(/Users/arnefranken/github/adobe/S3Mock/jffi833879770895826780.dylib, 0x0001): tried: '/Users/arnefranken/github/adobe/S3Mock/jffi833879770895826780.dylib' (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64e')), '/usr/lib/jffi833879770895826780.dylib' (no such file)
[ERROR]         at com.kenai.jffi.internal.StubLoader.tempLoadError(StubLoader.java:424)
[ERROR]         at com.kenai.jffi.internal.StubLoader.loadFromJar(StubLoader.java:409)
[ERROR]         at com.kenai.jffi.internal.StubLoader.load(StubLoader.java:278)
[ERROR]         at com.kenai.jffi.internal.StubLoader.<clinit>(StubLoader.java:487)
[ERROR]         at java.base/java.lang.Class.forName0(Native Method)
[ERROR]         at java.base/java.lang.Class.forName(Class.java:398)
[ERROR]         at com.kenai.jffi.Init.load(Init.java:68)
[ERROR]         at com.kenai.jffi.Foreign$InstanceHolder.getInstanceHolder(Foreign.java:49)
[ERROR]         at com.kenai.jffi.Foreign$InstanceHolder.<clinit>(Foreign.java:45)
[ERROR]         at com.kenai.jffi.Foreign.getInstance(Foreign.java:103)
[ERROR]         at com.kenai.jffi.Type$Builtin.lookupTypeInfo(Type.java:242)
[ERROR]         at com.kenai.jffi.Type$Builtin.getTypeInfo(Type.java:237)
[ERROR]         at com.kenai.jffi.Type.resolveSize(Type.java:155)
[ERROR]         at com.kenai.jffi.Type.size(Type.java:138)
[ERROR]         at jnr.ffi.provider.jffi.NativeRuntime$TypeDelegate.size(NativeRuntime.java:178)
[ERROR]         at jnr.ffi.provider.AbstractRuntime.<init>(AbstractRuntime.java:48)
[ERROR]         at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:57)
[ERROR]         at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:41)
[ERROR]         at jnr.ffi.provider.jffi.NativeRuntime$SingletonHolder.<clinit>(NativeRuntime.java:53)
[ERROR]         at jnr.ffi.provider.jffi.NativeRuntime.getInstance(NativeRuntime.java:49)
[ERROR]         at jnr.ffi.provider.jffi.Provider.<init>(Provider.java:29)
[ERROR]         at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
[ERROR]         at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
[ERROR]         at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
[ERROR]         at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
[ERROR]         at java.base/java.lang.Class.newInstance(Class.java:584)
[ERROR]         at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.getInstance(FFIProvider.java:68)
[ERROR]         at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.<clinit>(FFIProvider.java:57)
[ERROR]         at jnr.ffi.provider.FFIProvider.getSystemProvider(FFIProvider.java:35)
[ERROR]         at jnr.ffi.LibraryLoader.create(LibraryLoader.java:73)
[ERROR]         at jnr.unixsocket.Native.<clinit>(Native.java:76)
[ERROR]         at jnr.unixsocket.UnixSocketChannel.<init>(UnixSocketChannel.java:101)
[ERROR]         at jnr.unixsocket.UnixSocketChannel.open(UnixSocketChannel.java:60)
[ERROR]         at io.fabric8.maven.docker.access.util.LocalSocketUtil.canConnectUnixSocket(LocalSocketUtil.java:38)
[ERROR]         at io.fabric8.maven.docker.access.DockerConnectionDetector$UnixSocketDockerHostProvider.getConnectionParameter(DockerConnectionDetector.java:111)
[ERROR]         at io.fabric8.maven.docker.access.DockerConnectionDetector.detectConnectionParameter(DockerConnectionDetector.java:80)
[ERROR]         at io.fabric8.maven.docker.service.DockerAccessFactory.createDockerAccess(DockerAccessFactory.java:34)
[ERROR]         at io.fabric8.maven.docker.AbstractDockerMojo.execute(AbstractDockerMojo.java:268)
[ERROR]         at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
[ERROR]         at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
[ERROR]         at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
[ERROR]         at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
[ERROR]         at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
[ERROR]         at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
[ERROR]         at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
[ERROR]         at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
[ERROR]         at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
[ERROR]         at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
[ERROR]         at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
[ERROR]         at org.apache.maven.cli.MavenCli.execute(MavenCli.java:972)
[ERROR]         at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:293)
[ERROR]         at org.apache.maven.cli.MavenCli.main(MavenCli.java:196)
[ERROR]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR]         at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR]         at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[ERROR]         at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282)
[ERROR]         at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225)
[ERROR]         at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406)
[ERROR]         at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347)
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginContainerException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :s3mock-docker
@afranken afranken self-assigned this Nov 23, 2021
@afranken
Copy link
Member Author

afranken commented Nov 23, 2021

What does work, however:

Download an x86_64 JDK, for example:
https://www.azul.com/downloads/?package=jdk

Open terminal (e.g. in Idea)
Set JDK as JAVA_HOME
Run maven in x86_64 compatibility mode using arch -x86_64 mvn clean install

afranken added a commit that referenced this issue Nov 23, 2021
Currently still fails without rosetta compatibility mode, see ticket.

Fixes #313
@afranken
Copy link
Member Author

Ideally, we'd find a Docker image including Java that is available for both x86_64 and amd64, so we can build on both architectures, and also publish for both architectures.

Technically, that's possible.
Unfortunately I haven't found a Docker image fulfilling all three criteria yet.

afranken added a commit that referenced this issue Nov 24, 2021
Currently still fails without rosetta compatibility mode, see ticket.

Fixes #313
@afranken
Copy link
Member Author

Just pushed the nonsense JDK17 JavaDoc fixes to main.
Now I can build the branch using a x86 JDK17 without explicitly calling arch -x86_64.

@afranken
Copy link
Member Author

Seems like this is the underlying issue:
jnr/jnr-ffi#257
jnr/jffi#116

afranken added a commit that referenced this issue Nov 24, 2021
This image is available in multiple architectures, i.e. arm64 and x86_64

Fixs #313
afranken added a commit that referenced this issue Nov 24, 2021
This image is available in multiple architectures, i.e. arm64 and x86_64

Fixs #313
@afranken
Copy link
Member Author

Must have been blind when I looked yesterday - the official openjdk is indeed available in a lot of architectures:
https://hub.docker.com/_/openjdk
Updated my PR, this works now locally in Github actions (x86_64) and M1 macs (arm64) to produce a Docker image that can be executed locally.

This does not solve the "docker buildx" problem where we want to build both x86_64 and arm64 compatible Docker images on one architecture.

afranken added a commit that referenced this issue Nov 25, 2021
This image is available in multiple architectures, i.e. arm64 and x86_64
Also create minimal JRE to use in final image.

Fixs #313
@afranken
Copy link
Member Author

afranken commented Nov 25, 2021

Yet another iteration on the Dockerfile - the official alpine image is now extremely small, ~2.5MB.
And we can just install JDK11, create a minimal JRE and then use that in the final image.

Current S3Mock: ~91MB
New alpine image: ~74MB

Confirmed on both M1 and Intel MacBook that the proposed Docker image built is working on the platform it's being built on.
M1 build is still failing without a x86 JDK, since the fabric8 maven plugin depends on jffi which is currently only working on x86:
fabric8io/docker-maven-plugin#1213
See also issues linked above.

Since Github Actions is using x86_64 as an architecture, the published Docker image will still be "amd64" only.

@ruudk
Copy link

ruudk commented Dec 13, 2021

Since Github Actions is using x86_64 as an architecture, the published Docker image will still be "amd64" only.

It's possible to build ARM64 images on GitHub Actions, example:

            -   name: Set up QEMU
                uses: docker/[email protected]
            -   name: Set up Docker Buildx
                uses: docker/[email protected]
                with:
                    version: v0.6.0
                    buildkitd-flags: --debug
            -   name: Login to DockerHub
                uses: docker/[email protected]
                with:
                    username: ${{ secrets.DOCKER_USERNAME }}
                    password: ${{ secrets.DOCKER_PASSWORD }}
            -   name: Build and push
                uses: docker/[email protected]
                with:
                    file: etc/Dockerfile
                    context: .
                    platforms: linux/amd64,linux/arm64
                    push: true
                    tags: s3mock/tag

@afranken
Copy link
Member Author

@ruudk thanks, the main problem right now is that we use the docker-maven-plugin to both build and publish the Docker container.
There are two problems:

  1. the plugin itself is not executable on M1 (ARM64) processors.
    That's what this ticket is about.

  2. the plugin does not allow publishing to multiple platforms at once. (it does not use docker buildx for example)
    see Facing issue while running docker image on Apple M1 platform. #287 and uncaught target signal 11 (Segmentation fault) - core dumped, running a container on MacOS with M1 chip #253

I may end up removing the docker-maven-plugin altogether to fix both of these issues since there currently seems to be very little movement towards fixing the plugin.

@afranken
Copy link
Member Author

merged #333 today, and now at least I can build main on my M1 without errors.
This still does not allow us to release a multi-arch image.

@afranken
Copy link
Member Author

see last comment, we're good here for now. At least I can build the S3Mock locally and run all IntegrationTests.

@dlsrb6342
Copy link

@afranken Is there any plan to release multi-arch docker image?

@afranken
Copy link
Member Author

afranken commented Jan 5, 2022

@dlsrb6342 yes, the plan is still to release a multi-arch image, but as detailed here: #313 (comment) we're currently relying on the docker-maven-plugin which still does not support multi-arch builds or releases.
We're also relying on that plugin to test the container while it's built and before we push, so refactoring the release process alone does not work.

You could right now check out this GIT repo and run the build locally.
If you run it on an M1, it will locally create an S3Mock Docker container that runs on an M1.

@afranken afranken mentioned this issue Jan 26, 2022
@afranken
Copy link
Member Author

afranken commented Feb 1, 2022

just released version 2.4.6 which is multi arch for arm64 and amd64.
https://hub.docker.com/r/adobe/s3mock/tags?page=1&ordering=last_updated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants