Modifications to ensure compatibility with OpenXL and xlC on AIX #20690

midronij · 2024-11-27T16:13:16Z

Modify compiler flags, macros, and linked libraries to ensure that OpenJ9 can be built with both OpenXL and xlC on AIX.

Updated version of #20045

pshipton · 2024-11-27T16:20:54Z

midronij · 2024-11-27T16:29:43Z

@zl-wang here is the new OpenXL PR for OpenJ9. Marked as a WIP since I'm still addressing the remaining review comments from Ishita's PR

midronij · 2024-11-28T17:54:00Z

I've just finished addressing the review comments from the previous version of this PR. @pshipton @zl-wang when you have a moment could you take a look to ensure that I didn't miss anything/provide any further feedback?

pshipton · 2024-11-28T18:37:26Z

It's @keithc-ca who should take a look, probably most of the comments were from him.

pshipton · 2024-11-28T18:42:48Z

I didn't see anything obvious. We'll need to check it doesn't break xlc, both on jenkins and vmfarm, if that hasn't been done yet.

To ensure that OpenJ9 can be built on AIX with both OpenXL and xlC, make modifications to compiler flags, macros, and linked libraries. Signed-off-by: midronij <[email protected]>

midronij · 2024-12-04T18:11:54Z

I've confirmed that these changes don't break the xlC build on either jenkins or vmfarm. However, I did run into some issues when trying to do an OpenXL build on jenkins. Namely, when building DDR this error message popped up:

12:17:56  Could not load program /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/vm/runtime/omr_ddrgen:
12:17:56  	Dependent module /usr/lib/libc++.a(shr2_64.o) could not be loaded.
12:17:56  	Member shr2_64.o is not found in archive 
12:17:56  gmake[10]: *** [CMakeFiles/j9ddr.dir/build.make:94: /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/vm/runtime/j9ddr.dat] Error 255

(source: https://hyc-runtimes-jenkins.swg-devops.com/job/Build_JDK23_ppc64_aix_Personal/61/console)

I'm able to bypass this temporarily by adding --disable-ddr to the bash configure command, but the build still fails with this error:

13:37:14  ld: 0711-317 ERROR: Undefined symbol: .inflateInit2_
13:37:14  ld: 0711-317 ERROR: Undefined symbol: .inflate
13:37:14  ld: 0711-317 ERROR: Undefined symbol: .inflateEnd
13:37:14  ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information.
13:37:14  .ibm-clang: error: linker command failed with exit code 8 (use -v to see invocation)

(source: https://hyc-runtimes-jenkins.swg-devops.com/job/Build_JDK23_ppc64_aix_Personal/62/console)

I've run into errors like this when building locally, and was able to deal with them by adding additional libraries to the linker commands, but as far as I can tell, the makefile that I would need to modify to deal with this one in particular isn't part of OpenJ9 or OMR:

13:37:14  gmake[3]: *** [test/JtregNativeJdk.gmk:143: /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/support/test/jdk/jtreg/native/bin/JliLaunchTest] Error 8

zl-wang · 2024-12-04T18:17:59Z

@pshipton for shr2_64.o issue, a new level of runtime needs to be installed.
@midronij for libz.so problem, there is a build option to bypass it;

pshipton · 2024-12-04T18:31:36Z

for shr2_64.o issue, a new level of runtime needs to be installed.

I believe we have 17.1.1.2 as per https://github.ibm.com/runtimes/infrastructure/issues/7704, but current issue is https://github.ibm.com/runtimes/infrastructure/issues/9368
The latest is 17.1.1.8 as per https://www.ibm.com/support/pages/fix-list-xl-cc-runtime-aix#171X
Whatever we install will be the minimum level required for customers. What version would you like?

zl-wang · 2024-12-04T18:52:27Z

i have these on my machine:

  libc++.adt.include        17.1.1.4  COMMITTED  C Set ++ Application
  libc++.rte                17.1.1.4  COMMITTED  IBM XL C++ Runtime for AIX 7.1
  libc++abi.rte             17.1.1.4  COMMITTED  IBM XL C++ Runtime for AIX 7.2

and, shr2_64.o is indeed in libc++.a:

-bash-5.0$ ar -Xany -t libc++.a
libc++.so.1
libc++.so.1
version.cpp.o
cxxabi.o
cxxabi_64.o
shr.o
shr_64.o
shr2.o
shr2_64.o
cxxabi_compat.imp

it might be due to your OpenXL ND (non default) installation, such that libraries were not installed under the usual directory (/usr/lpp/xlC/lib). So that, a separate build-time/link-time -L argument is needed to point to where they were installed (i would assume somewhere under OpenXL compiler installation itself).

pshipton · 2024-12-04T20:22:47Z

On the build machine

ar -Xany -t /opt/IBM/openxlC/usr/lib/libc++.a
libc++.so.1
libc++.so.1
version.cpp.o
cxxabi.o
cxxabi_64.o
shr.o
shr_64.o
shr2.o
shr2_64.o
cxxabi_compat.imp

midronij · 2024-12-05T15:22:22Z

@midronij for libz.so problem, there is a build option to bypass it;

I tried building again with --with-zlib=bundled added to the bash configure command, and it seems like that got me past the zlib issue, but the build is still failing with a bunch of errors similar to what I was getting before with DDR enabled:

14:58:03  JVMJ9VM011W Unable to load j9jit29: Could not load module /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/jdk/lib/default/libj9jit29.so.
14:58:03  	Dependent module /usr/lib/libc++.a(shr2_64.o) could not be loaded.
14:58:03  	Member shr2_64.o is not found in archive 
14:58:03  Could not load module /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/jdk/lib/default/libj9jit29.so.
14:58:03  	Dependent module /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/jdk/lib/default/li
14:58:41  Creating java.compiler.jmod
14:58:41  JVMJ9VM011W Unable to load j9jit29: Could not load module /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/jdk/lib/default/libj9jit29.so.
14:58:41  	Dependent module /usr/lib/libc++.a(shr2_64.o) could not be loaded.
14:58:41  	Member shr2_64.o is not found in archive 
14:58:41  Could not load module /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/jdk/lib/default/libj9jit29.so.
14:58:41  	Dependent module /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/jdk/lib/default/li
.
.
.

(source: https://hyc-runtimes-jenkins.swg-devops.com/job/Build_JDK23_ppc64_aix_Personal/63/console)

zl-wang · 2024-12-05T15:41:36Z

you need to add one config option to indicate to the linker where to look up the runtime libraries (instead of the default location on AIX machine ... i.e. /usr/lib or /usr/lib64). peter already got you that location: /opt/IBM/openxlC/usr/lib
Certain option which eventually turned into -L option of the compiler/linker, that is.

dsouzai

I have a few questions since some of these changes impact platforms that are not AIX+OPENXL.

runtime/cmake/platform/toolcfg/gnu.cmake

runtime/gcchk/CMakeLists.txt

dsouzai · 2024-12-09T20:11:37Z

runtime/jilgen/CMakeLists.txt

@@ -35,6 +35,7 @@ target_link_libraries(constgen
 		j9vm_interface
 		omrport
 		j9thr
+		j9util


What are the implications of this for non AIX+OPENXL?

Same q as below.

runtime/makelib/targets.mk.aix.inc.ftl

dsouzai · 2024-12-09T20:14:10Z

runtime/verbose/CMakeLists.txt

+		j9gc
+		j9gctrc
+		j9gcapi
+		j9gctrcstandard
+		j9gctrcvlhgc


What are the implications of this for non AIX+OPENXL?

Doesn't this affect configs that are not AIX+OPENXL, so shouldn't this also be moved to #20799?

dsouzai · 2024-12-09T20:14:17Z

runtime/verbose/CMakeLists.txt

+			j9gc
+			j9gctrc
+			j9gcapi
+			j9gctrcstandard
+			j9gctrcvlhgc


What are the implications of this for non AIX+OPENXL?

Same q as above.

dsouzai · 2024-12-09T20:14:22Z

runtime/verbose/module.xml

@@ -137,6 +137,7 @@ SPDX-License-Identifier: EPL-2.0 OR Apache-2.0 OR GPL-2.0-only WITH Classpath-ex
 			<library name="j9modronstartup"/>
 			<library name="omrgcstartup" type="external"/>
 			<library name="omrgcverbose" type="external"/>
+			<library name="j9gc"/>


What are the implications of this for non AIX+OPENXL?

Same q as above.

Signed-off-by: midronij <[email protected]>

midronij requested a review from dsouzai as a code owner November 27, 2024 16:13

midronij changed the title ~~Modifications to ensure compatibility with OpenXL and xlC on AIX~~ WIP: Modifications to ensure compatibility with OpenXL and xlC on AIX Nov 27, 2024

pshipton requested a review from keithc-ca November 27, 2024 16:17

midronij mentioned this pull request Nov 27, 2024

Adding libs for OpenXL #20045

Closed

midronij force-pushed the openxl-final branch from 8661a7b to 6b6cfe5 Compare November 28, 2024 17:52

midronij force-pushed the openxl-final branch from 6b6cfe5 to eac524c Compare December 3, 2024 15:59

midronij changed the title ~~WIP: Modifications to ensure compatibility with OpenXL and xlC on AIX~~ Modifications to ensure compatibility with OpenXL and xlC on AIX Dec 3, 2024

Modifications for OpenXL Compatibility on AIX

2701a7f

To ensure that OpenJ9 can be built on AIX with both OpenXL and xlC, make modifications to compiler flags, macros, and linked libraries. Signed-off-by: midronij <[email protected]>

dsouzai requested changes Dec 9, 2024

View reviewed changes

dsouzai added arch:power os:aix depends:omr:breaking labels Dec 10, 2024

Suggested changes

541b187

Signed-off-by: midronij <[email protected]>

midronij force-pushed the openxl-final branch from eac524c to 541b187 Compare December 10, 2024 16:38

midronij mentioned this pull request Dec 10, 2024

Remove conditional linking of libraries #20799

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Modifications to ensure compatibility with OpenXL and xlC on AIX #20690

Modifications to ensure compatibility with OpenXL and xlC on AIX #20690

midronij commented Nov 27, 2024 •

edited

Loading

pshipton commented Nov 27, 2024

midronij commented Nov 27, 2024

midronij commented Nov 28, 2024

pshipton commented Nov 28, 2024

pshipton commented Nov 28, 2024

midronij commented Dec 4, 2024

zl-wang commented Dec 4, 2024

pshipton commented Dec 4, 2024 •

edited

Loading

zl-wang commented Dec 4, 2024

pshipton commented Dec 4, 2024

midronij commented Dec 5, 2024

zl-wang commented Dec 5, 2024

dsouzai left a comment

dsouzai Dec 9, 2024

dsouzai Dec 10, 2024

dsouzai Dec 9, 2024

dsouzai Dec 10, 2024 •

edited

Loading

dsouzai Dec 9, 2024

dsouzai Dec 10, 2024

dsouzai Dec 9, 2024

dsouzai Dec 10, 2024

Modifications to ensure compatibility with OpenXL and xlC on AIX #20690

Are you sure you want to change the base?

Modifications to ensure compatibility with OpenXL and xlC on AIX #20690

Conversation

midronij commented Nov 27, 2024 • edited Loading

pshipton commented Nov 27, 2024

midronij commented Nov 27, 2024

midronij commented Nov 28, 2024

pshipton commented Nov 28, 2024

pshipton commented Nov 28, 2024

midronij commented Dec 4, 2024

zl-wang commented Dec 4, 2024

pshipton commented Dec 4, 2024 • edited Loading

zl-wang commented Dec 4, 2024

pshipton commented Dec 4, 2024

midronij commented Dec 5, 2024

zl-wang commented Dec 5, 2024

dsouzai left a comment

Choose a reason for hiding this comment

dsouzai Dec 9, 2024

Choose a reason for hiding this comment

dsouzai Dec 10, 2024

Choose a reason for hiding this comment

dsouzai Dec 9, 2024

Choose a reason for hiding this comment

dsouzai Dec 10, 2024 • edited Loading

Choose a reason for hiding this comment

dsouzai Dec 9, 2024

Choose a reason for hiding this comment

dsouzai Dec 10, 2024

Choose a reason for hiding this comment

dsouzai Dec 9, 2024

Choose a reason for hiding this comment

dsouzai Dec 10, 2024

Choose a reason for hiding this comment

midronij commented Nov 27, 2024 •

edited

Loading

pshipton commented Dec 4, 2024 •

edited

Loading

dsouzai Dec 10, 2024 •

edited

Loading