Add multiplatform linux build to CI #1372

abihf · 2022-11-03T18:30:49Z

Add linux-aarch64 build and assets
Update volta-install.sh scripts to support aarch64 platform on linux

avaly · 2022-11-04T11:53:56Z

Shouldn't this install script also be updated with the support for this architecture?

charlespierce · 2022-11-04T18:00:10Z

Hi @abihf, thanks for the contribution! I'm a little worried this is conflating two related but not identical issues, making it a little more risky. Creating an arm build in the existing Linux pipeline is something we've been looking at for a while (which should be a lot easier now that we dropped the OpenSSL dependency). On the other hand, switching to a fully statically-linked libc is a bigger change that we haven't fully investigated. For that, we should have an RFC so that we can be reasonably confident that we've thought through the impact of that change.

Given that, would you be open to breaking this into two parts? The first being creating a pre-built binary for ARM Linux (using the existing pipeline) and the second being an RFC for switching to statically-linked libc. Totally reasonable to do one or the other (or neither if you don't have the bandwidth!), but I don't think we're in a state right now where we can take on both of these changes at once.

abihf · 2022-11-05T11:42:42Z

I've disabled static link & updated volta-install.sh

avaly · 2022-11-10T12:59:03Z

How can we help moving this PR along?

My team is blocked on using the new pnpm support from #1273, because we also need a linux-aarch64 binary.

nevir · 2023-02-02T01:25:40Z

Also interested in this

ditachev · 2023-02-27T11:10:22Z

Our team is also in need of a pre-built ARM64 linux binary. Is there any progress on this or is there a plan of action on this going forward?

avarun42 · 2023-03-08T21:50:57Z

Hi @abihf, thanks for the contribution! I'm a little worried this is conflating two related but not identical issues, making it a little more risky. Creating an arm build in the existing Linux pipeline is something we've been looking at for a while (which should be a lot easier now that we dropped the OpenSSL dependency). On the other hand, switching to a fully statically-linked libc is a bigger change that we haven't fully investigated. For that, we should have an RFC so that we can be reasonably confident that we've thought through the impact of that change.

Given that, would you be open to breaking this into two parts? The first being creating a pre-built binary for ARM Linux (using the existing pipeline) and the second being an RFC for switching to statically-linked libc. Totally reasonable to do one or the other (or neither if you don't have the bandwidth!), but I don't think we're in a state right now where we can take on both of these changes at once.

Hi @charlespierce, it seems that this PR no longer uses a fully statically-linked libc. Would it be possible to get this merged now?

nevir · 2023-03-31T17:40:41Z

Also very interested in this

this-is-tobi · 2023-04-21T17:42:45Z

Hi, any news about this one ?

charlespierce · 2023-04-21T20:04:50Z

.github/workflows/release.yml

+      - name: Install cross
+        run: cargo install cross
+      - name: Build
+        run: cross build --release --target ${{ matrix.target }}
+      - name: Create artifact
+        run: tar -czvf volta-${{ matrix.name }}.tar.gz volta volta-shim volta-migrate


I'm not on a machine right now where I can cleanly check, but does this change the glibc dependency for the standard linux build? The Docker system setup right now is built intentionally to support running on an old version of glibc.

For the ARM build, since we haven't supported it in the past, it's not a big deal to use a more recent version, but for the existing linux build we've been committed to backwards compatibility.

I have never done any development in Rust so it would take me a long time to do a good analysis but I can try to make a proposal.

Could QEMU be used in Github actions with another step to build a second linux builder image (linux/arm64 platform) and then use this image with the docker run --platform flag to build volta for both architectures?

EDIT : I didn't mention this but it would result in a change to the base image in the Dockerfile (cern/slc6-base) as it's an amd64 image, but I don't know why you're using that image so I can't say if that's something possible.

charlespierce

Overall looks good! Just one open question about the glibc compatibility for the existing Linux build.

abihf · 2023-04-23T11:38:30Z

Overall looks good! Just one open question about the glibc compatibility for the existing Linux build.

the cross build uses ubuntu:20.04 as base image link

I have tried running it on ubuntu 18.04 and it still running

❯ docker run -i -t --rm -v $(pwd):/volta -w /volta ubuntu:18.04 ./volta -v
Updating your Volta directory. This may take a few moments...
1.1.1

But unfortunately it can't run on centos:7

❯ docker run -i -t --rm -v $(pwd):/volta -w /volta centos:7 ./volta -v
./volta: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by ./volta)

will try to investigate it later

Fydon · 2023-10-11T20:09:17Z

dev/unix/volta-install.sh

@@ -129,6 +129,8 @@ parse_os_info() {
    Linux)
      if [ "$arch" == "x86_64" ]; then
        echo "linux"
+      elif [ "$arch" == "aarch64" ]; then
+        echo "linux-aarch64"
      else
        error "Releases for non x64 architectures are not currently supported."


Thank you for this. Might be worth updating the message as well

Hi @Fydon would you point me which message I need to update?

Sure. For macOS there is no error text, so perhaps the same is true for Linux when ARM64 is supported. Otherwise the error message should be something like "Only x64 and ARM64 architectures are currently supported". I believe that x64 is x86_64 and not all architectures ending in 64.

Fydon · 2023-11-07T16:43:45Z

Will using cargo clean resolve the GLIBC issue, as recommended here?

Fydon · 2023-11-07T17:19:58Z

I assume that a similar change to this for Linux would be ideal, in another pull request?

charlespierce · 2024-02-17T00:26:01Z

I believe this is handled by #1696. Thanks again for the inspiration!

abihf force-pushed the linux-multi-platform branch from 2d8d832 to e1a2c28 Compare November 3, 2022 18:46

abihf added 2 commits November 5, 2022 18:28

Add multiplatform linux build to CI

7a36f37

support linux-aarch on install script

58ed28b

abihf force-pushed the linux-multi-platform branch from e1a2c28 to 58ed28b Compare November 5, 2022 11:33

Merge branch 'main' into linux-multi-platform

3ecc0e0

chawyehsu mentioned this pull request Jan 15, 2023

Gate pnpm support behind a feature flag to avoid compatibility issues #1394

Merged

Merge branch 'main' into linux-multi-platform

e785b84

charlespierce reviewed Apr 21, 2023

View reviewed changes

Fydon reviewed Oct 11, 2023

View reviewed changes

charlespierce closed this Feb 17, 2024

TheJaredWilcurt mentioned this pull request Feb 22, 2024

Provide prebuilt binaries for alpine arm64 environments #1049

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add multiplatform linux build to CI #1372

Add multiplatform linux build to CI #1372

abihf commented Nov 3, 2022 •

edited

Loading

avaly commented Nov 4, 2022

charlespierce commented Nov 4, 2022

abihf commented Nov 5, 2022

avaly commented Nov 10, 2022

nevir commented Feb 2, 2023

ditachev commented Feb 27, 2023

avarun42 commented Mar 8, 2023

nevir commented Mar 31, 2023

this-is-tobi commented Apr 21, 2023

charlespierce Apr 21, 2023

this-is-tobi Apr 22, 2023 •

edited

Loading

charlespierce left a comment

abihf commented Apr 23, 2023

Fydon Oct 11, 2023

abihf Jan 7, 2024

Fydon Jan 8, 2024 •

edited

Loading

Fydon commented Nov 7, 2023 •

edited

Loading

Fydon commented Nov 7, 2023

charlespierce commented Feb 17, 2024

Add multiplatform linux build to CI #1372

Add multiplatform linux build to CI #1372

Conversation

abihf commented Nov 3, 2022 • edited Loading

avaly commented Nov 4, 2022

charlespierce commented Nov 4, 2022

abihf commented Nov 5, 2022

avaly commented Nov 10, 2022

nevir commented Feb 2, 2023

ditachev commented Feb 27, 2023

avarun42 commented Mar 8, 2023

nevir commented Mar 31, 2023

this-is-tobi commented Apr 21, 2023

charlespierce Apr 21, 2023

Choose a reason for hiding this comment

this-is-tobi Apr 22, 2023 • edited Loading

Choose a reason for hiding this comment

charlespierce left a comment

Choose a reason for hiding this comment

abihf commented Apr 23, 2023

Fydon Oct 11, 2023

Choose a reason for hiding this comment

abihf Jan 7, 2024

Choose a reason for hiding this comment

Fydon Jan 8, 2024 • edited Loading

Choose a reason for hiding this comment

Fydon commented Nov 7, 2023 • edited Loading

Fydon commented Nov 7, 2023

charlespierce commented Feb 17, 2024

abihf commented Nov 3, 2022 •

edited

Loading

this-is-tobi Apr 22, 2023 •

edited

Loading

Fydon Jan 8, 2024 •

edited

Loading

Fydon commented Nov 7, 2023 •

edited

Loading