Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modifications to ensure compatibility with OpenXL and xlC on AIX #20690

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

midronij
Copy link
Contributor

@midronij midronij commented Nov 27, 2024

Modify compiler flags, macros, and linked libraries to ensure that OpenJ9 can be built with both OpenXL and xlC on AIX.

Updated version of #20045

@midronij midronij requested a review from dsouzai as a code owner November 27, 2024 16:13
@midronij midronij changed the title Modifications to ensure compatibility with OpenXL and xlC on AIX WIP: Modifications to ensure compatibility with OpenXL and xlC on AIX Nov 27, 2024
@pshipton pshipton requested a review from keithc-ca November 27, 2024 16:17
@pshipton
Copy link
Member

The OMR PR eclipse-omr/omr#7561

@midronij
Copy link
Contributor Author

@zl-wang here is the new OpenXL PR for OpenJ9. Marked as a WIP since I'm still addressing the remaining review comments from Ishita's PR

@midronij
Copy link
Contributor Author

I've just finished addressing the review comments from the previous version of this PR. @pshipton @zl-wang when you have a moment could you take a look to ensure that I didn't miss anything/provide any further feedback?

@pshipton
Copy link
Member

It's @keithc-ca who should take a look, probably most of the comments were from him.

@pshipton
Copy link
Member

I didn't see anything obvious. We'll need to check it doesn't break xlc, both on jenkins and vmfarm, if that hasn't been done yet.

To ensure that OpenJ9 can be built on AIX with both OpenXL and xlC, make
modifications to compiler flags, macros, and linked libraries.

Signed-off-by: midronij <[email protected]>
@midronij midronij changed the title WIP: Modifications to ensure compatibility with OpenXL and xlC on AIX Modifications to ensure compatibility with OpenXL and xlC on AIX Dec 3, 2024
@midronij
Copy link
Contributor Author

midronij commented Dec 4, 2024

I've confirmed that these changes don't break the xlC build on either jenkins or vmfarm. However, I did run into some issues when trying to do an OpenXL build on jenkins. Namely, when building DDR this error message popped up:

12:17:56  Could not load program /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/vm/runtime/omr_ddrgen:
12:17:56  	Dependent module /usr/lib/libc++.a(shr2_64.o) could not be loaded.
12:17:56  	Member shr2_64.o is not found in archive 
12:17:56  gmake[10]: *** [CMakeFiles/j9ddr.dir/build.make:94: /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/vm/runtime/j9ddr.dat] Error 255

(source: https://hyc-runtimes-jenkins.swg-devops.com/job/Build_JDK23_ppc64_aix_Personal/61/console)

I'm able to bypass this temporarily by adding --disable-ddr to the bash configure command, but the build still fails with this error:

13:37:14  ld: 0711-317 ERROR: Undefined symbol: .inflateInit2_
13:37:14  ld: 0711-317 ERROR: Undefined symbol: .inflate
13:37:14  ld: 0711-317 ERROR: Undefined symbol: .inflateEnd
13:37:14  ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information.
13:37:14  .ibm-clang: error: linker command failed with exit code 8 (use -v to see invocation)

(source: https://hyc-runtimes-jenkins.swg-devops.com/job/Build_JDK23_ppc64_aix_Personal/62/console)

I've run into errors like this when building locally, and was able to deal with them by adding additional libraries to the linker commands, but as far as I can tell, the makefile that I would need to modify to deal with this one in particular isn't part of OpenJ9 or OMR:

13:37:14  gmake[3]: *** [test/JtregNativeJdk.gmk:143: /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/support/test/jdk/jtreg/native/bin/JliLaunchTest] Error 8

@zl-wang
Copy link
Contributor

zl-wang commented Dec 4, 2024

@pshipton for shr2_64.o issue, a new level of runtime needs to be installed.
@midronij for libz.so problem, there is a build option to bypass it;

@pshipton
Copy link
Member

pshipton commented Dec 4, 2024

for shr2_64.o issue, a new level of runtime needs to be installed.

I believe we have 17.1.1.2 as per https://github.ibm.com/runtimes/infrastructure/issues/7704, but current issue is https://github.ibm.com/runtimes/infrastructure/issues/9368
The latest is 17.1.1.8 as per https://www.ibm.com/support/pages/fix-list-xl-cc-runtime-aix#171X
Whatever we install will be the minimum level required for customers. What version would you like?

@zl-wang
Copy link
Contributor

zl-wang commented Dec 4, 2024

i have these on my machine:

  libc++.adt.include        17.1.1.4  COMMITTED  C Set ++ Application
  libc++.rte                17.1.1.4  COMMITTED  IBM XL C++ Runtime for AIX 7.1
  libc++abi.rte             17.1.1.4  COMMITTED  IBM XL C++ Runtime for AIX 7.2

and, shr2_64.o is indeed in libc++.a:

-bash-5.0$ ar -Xany -t libc++.a
libc++.so.1
libc++.so.1
version.cpp.o
cxxabi.o
cxxabi_64.o
shr.o
shr_64.o
shr2.o
shr2_64.o
cxxabi_compat.imp

it might be due to your OpenXL ND (non default) installation, such that libraries were not installed under the usual directory (/usr/lpp/xlC/lib). So that, a separate build-time/link-time -L argument is needed to point to where they were installed (i would assume somewhere under OpenXL compiler installation itself).

@pshipton
Copy link
Member

pshipton commented Dec 4, 2024

On the build machine

ar -Xany -t /opt/IBM/openxlC/usr/lib/libc++.a
libc++.so.1
libc++.so.1
version.cpp.o
cxxabi.o
cxxabi_64.o
shr.o
shr_64.o
shr2.o
shr2_64.o
cxxabi_compat.imp

@midronij
Copy link
Contributor Author

midronij commented Dec 5, 2024

@midronij for libz.so problem, there is a build option to bypass it;

I tried building again with --with-zlib=bundled added to the bash configure command, and it seems like that got me past the zlib issue, but the build is still failing with a bunch of errors similar to what I was getting before with DDR enabled:

14:58:03  JVMJ9VM011W Unable to load j9jit29: Could not load module /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/jdk/lib/default/libj9jit29.so.
14:58:03  	Dependent module /usr/lib/libc++.a(shr2_64.o) could not be loaded.
14:58:03  	Member shr2_64.o is not found in archive 
14:58:03  Could not load module /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/jdk/lib/default/libj9jit29.so.
14:58:03  	Dependent module /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/jdk/lib/default/li
14:58:41  Creating java.compiler.jmod
14:58:41  JVMJ9VM011W Unable to load j9jit29: Could not load module /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/jdk/lib/default/libj9jit29.so.
14:58:41  	Dependent module /usr/lib/libc++.a(shr2_64.o) could not be loaded.
14:58:41  	Member shr2_64.o is not found in archive 
14:58:41  Could not load module /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/jdk/lib/default/libj9jit29.so.
14:58:41  	Dependent module /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/jdk/lib/default/li
.
.
.

(source: https://hyc-runtimes-jenkins.swg-devops.com/job/Build_JDK23_ppc64_aix_Personal/63/console)

@zl-wang
Copy link
Contributor

zl-wang commented Dec 5, 2024

you need to add one config option to indicate to the linker where to look up the runtime libraries (instead of the default location on AIX machine ... i.e. /usr/lib or /usr/lib64). peter already got you that location: /opt/IBM/openxlC/usr/lib
Certain option which eventually turned into -L option of the compiler/linker, that is.

Copy link
Contributor

@dsouzai dsouzai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a few questions since some of these changes impact platforms that are not AIX+OPENXL.

if(NOT OMR_OS_OSX)
# This flag isn't supported on OSX and on AIX while using openxl compiler,
# but it has this behaviour by default.
if(NOT OMR_OS_OSX AND NOT OMR_OS_AIX AND NOT CMAKE_C_COMPILER_IS_OPENXL)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think as this stands, this will also remove the flag for AIX when not using openxl. If you want to something like:

!(OSX || (AIX && OPENXL))

that expands (via a series of DeMorgan's) to

!OSX && (!AIX || !OPENXL)

Comment on lines +32 to +55
target_link_libraries(j9gcchk
PRIVATE
j9vm_interface
j9util
j9utilcore
j9avl
j9hashtable
j9thr
j9stackmap
j9pool
j9hookable
j9modronstartup

#TODO do we just want to link against j9gc?
j9gcbase
j9gcstructs
j9gcstats
j9gccheck
omrgc
)
endif()
#TODO do we just want to link against j9gc?
j9gcbase
j9gcstructs
j9gcstats
j9gccheck
omrgc
j9gctrc
j9gctrcstandard
j9gctrcvlhgc
j9gc
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to change what happens when !defined(DEFINED OMR_OS_AIX AND DEFINED CMAKE_C_COMPILER_IS_OPENXL); why are we modifying non AIX+OPENXL platforms?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the original intent was to limit this to AIX and OpenXL, but that condition was removed as requested on Ishita's old PR: #20045 (comment)

Comment on lines +71 to +94
target_link_libraries(j9gcchk_full
PRIVATE
j9vm_interface
j9util
j9utilcore
j9avl
j9hashtable
j9thr
j9stackmap
j9pool
j9hookable
j9modronstartup

#TODO do we just want to link against j9gc?
j9gcbase_full
j9gcstructs_full
j9gcstats_full
j9gccheck_full
omrgc_full
)
endif()
#TODO do we just want to link against j9gc?
j9gcbase_full
j9gcstructs_full
j9gcstats_full
j9gccheck_full
omrgc_full
j9gctrc_full
j9gctrcstandard_full
j9gctrcvlhgc_full
j9gc_full
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above.

@@ -35,6 +35,7 @@ target_link_libraries(constgen
j9vm_interface
omrport
j9thr
j9util
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the implications of this for non AIX+OPENXL?

Comment on lines 139 to +140
CXXFLAGS += -qxlcompatmacros -fno-rtti -fno-exceptions
CXXFLAGS += -qxlcompatmacros -fno-rtti -fno-exceptions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Repeated options.

Comment on lines +83 to +87
j9gc
j9gctrc
j9gcapi
j9gctrcstandard
j9gctrcvlhgc
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the implications of this for non AIX+OPENXL?

Comment on lines +146 to +150
j9gc
j9gctrc
j9gcapi
j9gctrcstandard
j9gctrcvlhgc
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the implications of this for non AIX+OPENXL?

@@ -137,6 +137,7 @@ SPDX-License-Identifier: EPL-2.0 OR Apache-2.0 OR GPL-2.0-only WITH Classpath-ex
<library name="j9modronstartup"/>
<library name="omrgcstartup" type="external"/>
<library name="omrgcverbose" type="external"/>
<library name="j9gc"/>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the implications of this for non AIX+OPENXL?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants