howto cross-compile to get to work for powerpc e500v2 #117361

th0ma7 · 2023-10-29T15:51:59Z

This relates to SynoCommunity spksrc project to build and package various open source software to run on Synology NAS.

I've been running into an issue where rust code cross-compiled for powerpc arch segfault (more specifically qoriq). The exact same code builds perfectly fine for all other archs (armv5, v7, aarch64, x86_64, i686). Ref: SynoCommunity/spksrc#5847 SynoCommunity/spksrc#5684

I've took two different approaches:

Using RUSTFLAGS to reproduce our CFLAGS used -mcpu=8548 -mhard-float -mfloat-gprs=double SynoCommunity/spksrc@90ad41d
Building a tier 3 powerpc-unknown-linux-gnuspe <<-- testing code only in my local branch for now

option 1

Using default powerpc-unknown-linux-gnu along with RUSTFLAGS = -Ctarget-cpu=e500 lead to the exact same result, seftault at startup. I may not be using the right RUSTFLAGS?

option 2

As for option 2 I feel I'm digging my own hole as I'm unable to build a powerpc-unknown-linux-gnuspe target using Synology provided toolchain and toolkit. I'm able to build up to stage1 & stage2 but unable to create a fully working target using cargo along with either my stage1 or stage2 builds... (clearly there is something I'm not fully understanding). Here's what I have so far:

git clone --depth 1 https://github.com/rust-lang/rust.git
./x setup compiler
PATH="$(WORK_DIR)/$(TC_TARGET)/bin:$${PATH}" ./x build --target $(RUST_TARGET)
rustup toolchain link powerpc-stage1 $(WORK_DIR)/rust/build/host/stage1
PATH="$(WORK_DIR)/$(TC_TARGET)/bin:$${PATH}" ./x build --stage 2 --target $(RUST_TARGET))
rustup toolchain link pwoerpc-stage2 $(WORK_DIR)/rust/build/host/stage2
--->> Up to this point all working OK <<---

Where it then fails:

rustup override set nightly
echo "[llvm]" >> $(WORK_DIR)/rust/config.toml
echo "allow-old-toolchain = true" >> $(WORK_DIR)/rust/config.toml   <<-- Presuming as `powerpc-e500v2-linux-gnuspe-gcc` is v4.9.3
PATH="$(WORK_DIR)/$(TC_TARGET)/bin:$${PATH}" POWERPC_UNKNOWN_LINUX_GNUSPE_OPENSSL_DIR=$(WORK_DIR)/../../../toolkit/syno-$(ARCH)-$(TCVERSION)/work/usr/ RUST_BACKTRACE=full cargo +$(firstword $(subst -, ,$(RUST_TARGET)))-stage1 build -Zbuild-std=core,alloc --target powerpc-unknown-linux-gnuspe

The error relates to llvm (enven though it did built it succesfully during either stage1 or stage2):

error: failed to run custom build command for `rustc_llvm v0.0.0 (/home/spksrc/qoriq-debug/spksrc/toolchain/syno-qoriq-6.2.4/work/rust/compiler/rustc_llvm)`
Caused by:
  process didn't exit successfully: `/home/spksrc/qoriq-debug/spksrc/toolchain/syno-qoriq-6.2.4/work/rust/target/debug/build/rustc_llvm-dc7ac9c0f6cecf54/build-script-build` (exit status: 101)
  --- stdout
  cargo:rustc-check-cfg=values(llvm_component,"ipo")
  cargo:rustc-check-cfg=values(llvm_component,"bitreader")
  cargo:rustc-check-cfg=values(llvm_component,"bitwriter")
  cargo:rustc-check-cfg=values(llvm_component,"linker")
  cargo:rustc-check-cfg=values(llvm_component,"asmparser")
  cargo:rustc-check-cfg=values(llvm_component,"lto")
  cargo:rustc-check-cfg=values(llvm_component,"coverage")
  cargo:rustc-check-cfg=values(llvm_component,"instrumentation")
  cargo:rustc-check-cfg=values(llvm_component,"x86")
  cargo:rustc-check-cfg=values(llvm_component,"arm")
  cargo:rustc-check-cfg=values(llvm_component,"aarch64")
  cargo:rustc-check-cfg=values(llvm_component,"amdgpu")
  cargo:rustc-check-cfg=values(llvm_component,"avr")
  cargo:rustc-check-cfg=values(llvm_component,"loongarch")
  cargo:rustc-check-cfg=values(llvm_component,"m68k")
  cargo:rustc-check-cfg=values(llvm_component,"csky")
  cargo:rustc-check-cfg=values(llvm_component,"mips")
  cargo:rustc-check-cfg=values(llvm_component,"powerpc")
  cargo:rustc-check-cfg=values(llvm_component,"systemz")
  cargo:rustc-check-cfg=values(llvm_component,"jsbackend")
  cargo:rustc-check-cfg=values(llvm_component,"webassembly")
  cargo:rustc-check-cfg=values(llvm_component,"msp430")
  cargo:rustc-check-cfg=values(llvm_component,"sparc")
  cargo:rustc-check-cfg=values(llvm_component,"nvptx")
  cargo:rustc-check-cfg=values(llvm_component,"hexagon")
  cargo:rustc-check-cfg=values(llvm_component,"riscv")
  cargo:rustc-check-cfg=values(llvm_component,"bpf")
  cargo:rerun-if-env-changed=RUST_CHECK
  cargo:rerun-if-env-changed=REAL_LIBRARY_PATH_VAR

  --- stderr
  thread 'main' panicked at compiler/rustc_llvm/build.rs:51:59:
  REAL_LIBRARY_PATH_VAR
  stack backtrace:
     0: rust_begin_unwind
     1: core::panicking::panic_fmt
     2: core::option::expect_failed
     3: core::option::Option<T>::expect
               at /home/spksrc/qoriq-debug/spksrc/toolchain/syno-qoriq-6.2.4/work/rust/library/core/src/option.rs:888:21
     4: build_script_build::restore_library_path
               at ./build.rs:51:15
     5: build_script_build::main
               at ./build.rs:113:5
     6: core::ops::function::FnOnce::call_once
               at /home/spksrc/qoriq-debug/spksrc/toolchain/syno-qoriq-6.2.4/work/rust/library/core/src/ops/function.rs:250:5
  note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
warning: build failed, waiting for other jobs to finish...
make[2]: *** [../../mk/spksrc.tc-rust.mk:64: rustc_target] Error 101
/home/spksrc/qoriq-debug/spksrc/cross/bat/work-qoriq-6.2.4/tc_vars.mk:1: *** An error occured while setting up the toolchain, please check the messages above.  Stop.
make[1]: Leaving directory '/home/spksrc/qoriq-debug/spksrc/cross/bat'

option 3

Now, further reading and as option 3 I may be able to "rebuild" tier 1 powerpc-unknown-linux-gnu by adding the proper target features such as #117347 . But guessing I'd have to run through the same procession as option 2 ?

The text was updated successfully, but these errors were encountered:

saethlin · 2023-10-29T16:13:19Z

Using default powerpc-unknown-linux-gnu along with RUSTFLAGS = -Ctarget-cpu=e500 lead to the exact same result, seftault at startup. I may not be using the right RUSTFLAGS?

You are using a tier 2 target. Can you compile any programs successfully? It's not clear to me from your description that this is not a miscompile.

th0ma7 · 2023-10-29T16:58:06Z

You are using a tier 2 target. Can you compile any programs successfully? It's not clear to me from your description that this is not a miscompile.

@saethlin nope, no programs seems to runs on the qoriq platform using powerpc-unknown-linux-gnu with or without RUSTFLAGS = -Ctarget-cpu=e500. Which is why I'm wondering if I'm using the right target? Or missing build flags... ?

workingjubilee · 2023-10-29T17:43:18Z

Try -Ctarget-cpu=e500mc

workingjubilee · 2023-10-29T17:56:49Z

I'm confused, you said QoriQ, but then you said you're targeting MPC8548? Which is it? No, they're not necessarily guaranteed to be compatible: ignore the vendor's claims otherwise and approach this empirically.

workingjubilee · 2023-10-29T18:05:18Z

You use a vendor-provided (and thus possibly patched?) gcc, so it seems plausible to me that LLVM might have a different model of the CPU capabilities. Can you use clang to build, instead of gcc, for this target, and get working binaries? Or even a stock gcc?

You also made a typo:
rustup toolchain link pwoerpc-stage2 $(WORK_DIR)/rust/build/host/stage2

th0ma7 · 2023-10-29T18:22:26Z

You use a vendor-provided (and thus possibly patched?) gcc, so it seems plausible to me that LLVM might have a different model of the CPU capabilities.

Indeed, Synology provide a GCC toolchain for each arch. The qoriq (e.g. powerpc) is using a version 4.9.3 of gcc.

Can you use clang to build, instead of gcc, for this target, and get working binaries? Or even a stock gcc?

That would require me to build a clang (or gcc) cross-compiler for this arch? Probably feasible but, I did try in the past using gcc in order to have a newer (and identical) version for all targets but I did not managed to assemble all the pieces together properly...

workingjubilee · 2023-10-29T18:24:04Z

LLVM-based compilers are always cross-compilers, that I've seen? You can use an x86_64-linux clang, or likewise for rustc, to emit code for PowerPC. The usual clang builds include codegen components for all their supported targets, I think.

workingjubilee · 2023-10-29T19:24:33Z

Indeed, Synology provide a GCC toolchain for each arch. The qoriq (e.g. powerpc) is using a version 4.9.3 of gcc.

Regarding this: do they patch it?

th0ma7 · 2023-10-29T19:27:03Z

Indeed, Synology provide a GCC toolchain for each arch. The qoriq (e.g. powerpc) is using a version 4.9.3 of gcc.

Regarding this: do they patch it?

No clue. And I was reviewing our own native llvm build that we provide part of synocli-devel package, we're dropping down to version 9 of llvm for archs that uses gcc <= 5.1... Which includes qoriq whereas it won't be able to build it anyhow without another version of GCC (or clang for that matter).

workingjubilee · 2023-10-29T19:30:06Z

Rust ships LLVM 17.

workingjubilee · 2023-10-29T20:36:04Z

GCC is a GPL project, so if you have been given the software, you also have the right to demand the source code, including its patches. ( also if you saw any messages between this and "Rust ships LLVM 17" from me: I'm blaming them all on the cat finding a few macro-commands and the enter key. )

th0ma7 · 2023-12-04T00:24:25Z

@workingjubilee so while my cycles where rather limited I did manage to make a bit of mileage over the last month...

In my current PR SynoCommunity/spksrc#5879 I was able to automate building a rustc compiler supporting powerpc-unknown-linux-gnuspe.

Why you might ask? because it hapens that resulting binaries contains lwsync instead of sync (ref: #96394 (comment)). It was confirmed below in the thread that using -Ctarget-cpu=e500 in conjunction with tier 3 powerpc-unknown-linux-gnuspe target support solves the issue for exactly the the Synology NAS model and CPU type I'm having issues with (ref: #96394 (comment)).

So now in one hand I do have a rustc compiler with tier 3 powerpc-unknown-linux-gnuspe target support... And in the other I'm passing -Ctarget-cpu=e500 using RUSTFLAGS variable... and I can see it being used in the build output (and compiler doesn't complain).

But sadly the end-result binaries still contains lwsync (ref: SynoCommunity/spksrc#5879 (comment)) and ends-up segfaulting (as to be expected, ref: SynoCommunity/spksrc#5847 (comment))

Question I have, what am I missing?

If someone else stumbles on this and wonder howto:

	@(cd $(WORK_DIR) && [ ! -d rust ] && git clone --depth 1 https://github.com/rust-lang/rust.git || true)
	@(cd $(WORK_DIR)/rust && ./x setup compiler)
	@(cd $(WORK_DIR)/rust && \
	    CFLAGS_$(subst -,_,$(RUST_TARGET))="$(TC_EXTRA_CFLAGS)" \
	    CXXFLAGS_$(subst -,_,$(RUST_TARGET))="$(TC_EXTRA_CFLAGS)" \
	    LDFLAGS_$(subst -,_,$(RUST_TARGET))="--sysroot=$(WORK_DIR)/$(TC_TARGET)/$(TC_SYSROOT)" \
	    RUST_BACKTRACE=full \
	    ./x build --config $(TC_LOCAL_VARS_RUST))
	@rustup toolchain link $(TC_RUSTUP_TOOLCHAIN) $(WORK_DIR)/rust/build/host/stage$(RUSTUP_DEFAULT_TOOLCHAIN_STAGE)

Where my config file refers to Synology toolchain tools:

profile = "compiler"

[build]
target = ["powerpc-unknown-linux-gnuspe"]
build-stage = 2
docs = false
docs-minification = false
compiler-docs = false

[target.powerpc-unknown-linux-gnuspe]
cc = "/home/spksrc/qoriq-debug/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500v2-l
inux-gnuspe-gcc"
cxx = "/home/spksrc/qoriq-debug/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500v2-
linux-gnuspe-g++"
ar = "/home/spksrc/qoriq-debug/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500v2-l
inux-gnuspe-ar"
ranlib = "/home/spksrc/qoriq-debug/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500
v2-linux-gnuspe-ranlib"
linker = "/home/spksrc/qoriq-debug/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500
v2-linux-gnuspe-gcc"

th0ma7 · 2024-01-11T12:44:53Z

@workingjubilee and @saethlin and @wycats I believe I have found where the issue is ... When building my powerpc-unknown-linux-gnuspe rust toolchain support I am using Synology provided toolchain and thus pointing to its relevant cc and all as per the config.toml above.

The rust toolchain do build just fine, but I noticed that lwsync do appear in the resulting libraries rust/build/host/stage1/lib/rustlib/powerpc-unknown-linux-gnuspe/lib/libstd-4b4243768d8e8241.so whereas objdump -d gives a few of those which are incompatible with this platform:

   ac98c:	7c 20 04 ac 	lwsync
   ac9a8:	7c 20 04 ac 	lwsync

My theory is that when building any software afterward and linking to these libraries, it will necessarely fail as result holds incompatible lwsync ...

Now why? I am providing the necessary CFLAGS and CXXFLAGS at rust toolchain build time... so issue must be how the flags are being passed in here somewhere where flags are not being honored:

(cd /home/spksrc/qoriq-debug-update/spksrc/toolchain/syno-qoriq-6.2.4/work/rust \
   CFLAGS_powerpc_unknown_linux_gnuspe="-mcpu=8548 -mhard-float -mfloat-gprs=double" \
   CXXFLAGS_powerpc_unknown_linux_gnuspe="-mcpu=8548 -mhard-float -mfloat-gprs=double" \
   LDFLAGS_powerpc_unknown_linux_gnuspe="--sysroot=/home/spksrc/qoriq-debug-update/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/powerpc-e500v2-linux-gnuspe/sysroot" \
   RUST_BACKTRACE=full \
   ./x build --config /home/spksrc/qoriq-debug-update/spksrc/toolchain/syno-qoriq-6.2.4/work/qoriq.toml)

Help would be much appreciated, thnx in advance!

glaubitz · 2024-01-19T11:25:22Z

Support for powerpc-unknown-linux-gnuspe was added to Rust by me and always worked fine until the port was removed from Debian due to gcc upstream dropping support for this target.

th0ma7 · 2024-01-21T22:06:42Z

@glaubitz perhaps you can assist in providing me hints as to why things are not working out?

quick background, synocommunity is spksrc framework github project to build package for Synology NAS on multiple archs (armv5, v7, v8, x86_64, ppc, etc). All other archs are well supported from a rust point of view with the exception of qoriq being a powerpc gnuspe tier 3 target. My current PR in order to support rust for qoriq is SynoCommunity/spksrc#5879

To build rust support I'm using Synology DSM 6.2.4 toolchain for that specific arch. Toolchain bundle file is located here https://sourceforge.net/projects/dsgpl/files/Tool%20Chain/DSM%206.2.4%20Tool%20Chains/PowerPC%20QorIQ%20Linux%202.6.32/qoriq-gcc493_glibc220_hard_qoriq-GPL.txz/download

gcc provided part of the toolchain is gcc version 4.9.3 20150311 (prerelease) (crosstool-NG 1.20.0)

Configuration file I'm using is the following:

profile = "compiler"

[build]
target = ["x86_64-unknown-linux-gnu", "powerpc-unknown-linux-gnuspe"]
build-stage = 1
doc-stage = 2
docs = false
docs-minification = false
compiler-docs = false

[rust]
channel = "stable"
lto = "off"

[llvm]
download-ci-llvm = "if-unchanged"

[install]

[dist]

[target.x86_64-unknown-linux-gnu]

[target.powerpc-unknown-linux-gnuspe]
cc = "/home/spksrc/qoriq-debug-update/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500v2-linux-gnuspe-gcc"
cxx = "/home/spksrc/qoriq-debug-update/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500v2-linux-gnuspe-g++"
ar = "/home/spksrc/qoriq-debug-update/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500v2-linux-gnuspe-ar"
ranlib = "/home/spksrc/qoriq-debug-update/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500v2-linux-gnuspe-ranlib"
linker = "/home/spksrc/qoriq-debug-update/spksrc/toolchain/syno-qoriq-6.2.4/work/powerpc-e500v2-linux-gnuspe/bin/powerpc-e500v2-linux-gnuspe-gcc"

Arch gcc specific flags we've been using since ever are -mcpu=8548 -mhard-float -mfloat-gprs=double. Working equivalent to be used with RUSTFLAGS for the llvm backend is -Ctarget-cpu=e500 based on #96394 (comment)

Hopefully what I'm missing is obvious... but your help would be more than welcomed ...

rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Oct 29, 2023

rustbot added the I-prioritize Issue: Indicates that prioritization has been requested for this issue. label Oct 29, 2023

saethlin added the T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) label Oct 29, 2023

saethlin removed I-unsound Issue: A soundness hole (worst kind of bug), see: https://en.wikipedia.org/wiki/Soundness I-prioritize Issue: Indicates that prioritization has been requested for this issue. labels Oct 29, 2023

th0ma7 mentioned this issue Nov 12, 2023

unable to build powerpc gnuspe tier 3 arch support rust-lang/rustup#3531

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

howto cross-compile to get to work for powerpc e500v2 #117361

howto cross-compile to get to work for powerpc e500v2 #117361

th0ma7 commented Oct 29, 2023

saethlin commented Oct 29, 2023

th0ma7 commented Oct 29, 2023

workingjubilee commented Oct 29, 2023

workingjubilee commented Oct 29, 2023

workingjubilee commented Oct 29, 2023 •

edited

Loading

th0ma7 commented Oct 29, 2023

workingjubilee commented Oct 29, 2023 •

edited

Loading

workingjubilee commented Oct 29, 2023

th0ma7 commented Oct 29, 2023

workingjubilee commented Oct 29, 2023

workingjubilee commented Oct 29, 2023 •

edited

Loading

th0ma7 commented Dec 4, 2023 •

edited

Loading

th0ma7 commented Jan 11, 2024 •

edited

Loading

glaubitz commented Jan 19, 2024

th0ma7 commented Jan 21, 2024 •

edited

Loading

howto cross-compile to get to work for powerpc e500v2 #117361

howto cross-compile to get to work for powerpc e500v2 #117361

Comments

th0ma7 commented Oct 29, 2023

option 1

option 2

option 3

saethlin commented Oct 29, 2023

th0ma7 commented Oct 29, 2023

workingjubilee commented Oct 29, 2023

workingjubilee commented Oct 29, 2023

workingjubilee commented Oct 29, 2023 • edited Loading

th0ma7 commented Oct 29, 2023

workingjubilee commented Oct 29, 2023 • edited Loading

workingjubilee commented Oct 29, 2023

th0ma7 commented Oct 29, 2023

workingjubilee commented Oct 29, 2023

workingjubilee commented Oct 29, 2023 • edited Loading

th0ma7 commented Dec 4, 2023 • edited Loading

th0ma7 commented Jan 11, 2024 • edited Loading

glaubitz commented Jan 19, 2024

th0ma7 commented Jan 21, 2024 • edited Loading

workingjubilee commented Oct 29, 2023 •

edited

Loading

workingjubilee commented Oct 29, 2023 •

edited

Loading

workingjubilee commented Oct 29, 2023 •

edited

Loading

th0ma7 commented Dec 4, 2023 •

edited

Loading

th0ma7 commented Jan 11, 2024 •

edited

Loading

th0ma7 commented Jan 21, 2024 •

edited

Loading