Regression: Expressions broken for Remote Java Application in recent Eclipse versions

[CalculonPrime]

Overview

We have a command-line build process for our framework not using Eclipse that generates several JARs that get deployed via Embedded Tomcat. In earlier Eclipse versions (circa 2017), I was able to create an Eclipse project, remove the "src" directory and replace it with a link to our app's source (outside the Eclipse project), and then attach to the running external Java app using a Remote Java Application launcher in Eclipse.

I also added the JARs generated by our build to the project, and all the 3rd party JARs used by the app.

With all of this in place, I could set breakpoints, step line-by-line, and most importantly, evaluate expressions in the expression pane.

However, with the most recent versions of Eclipse (2023-09 and 2023-12 tested), expressions don't work. This is horrific! Having to rebuild with logging or print commands makes things much more difficult. Expressions added show no output, and I believe this is connected to what happens when I press ctrl+shift+I after selecting a variable in the code near the breakpoint, which reports,

java.lang.reflect.InvocationTargetException: Unable to evaluate the selected expression:

To perform an evaluation, an expression must be compiled in the context of a Java project's build path. The current execution context is not associated with a Java project in the workspace.
	at org.eclipse.jdt.internal.debug.ui.actions.EvaluateAction$1.run(EvaluateAction.java:270)
	at org.eclipse.jface.operation.ModalContext$ModalContextThread.run(ModalContext.java:122)

The Java project for the new Eclipses was built in the same way, with the src directory in the project linked to the external Java source. All the suggested solutions for this problem I found on the web say to make sure that you're linking to the same source and libraries in the project, but as mentioned the app wasn't built with an Eclipse project in this case, as it's an external app. So this is not possible as a solution.

Reproduction

The most basic example of "remote debugging" (debugging an attached Java process outside the Eclipse project) will reproduce it.

I've attached two files:
javaTestApp.tar.gz
JavaHostProject.tar.gz

The first, javaTestApp, is a standalone Java app. It contains a build.sh and run.sh to build and run the app - just javac and java commands. The app takes a number and prints its square, in a loop. Hit return without typing anything to exit. The other tar file is the Eclipse project JavaHostProject, which links to the external javaTestApp java app.

To reproduce:

• Expand both of these tar files in your /tmp directory (Linux assumed).
• Build javaTestApp from the command line using build.sh, and run it with run.sh. You can type numbers into the app to see that it works. Just leave it running. We're going to attach to it in Eclipse.
• Open Eclipse, and import the project directory /tmp/JavaHostProject. The src directory in the project is a link to the src directory in the javaTestApp (which assumes you expanded it in /tmp). You can double click on Input.java in the project to open that source file in Eclipse.
• Attach to the running javaTestApp via Run >> Debug Configurations >> Remote Java Application. The run.sh script for the external app opens debug port 8000 which is the default for Remote Java Application
• Set a breakpoint somewhere inside the loop of Input.java. I will assume you set it at the line containing the System.out.println() call that prints the variable "square".
• Type another integer into the javaTestApp you left running. This will cause Eclipse to hit the breakpoint. Try hovering over the variable "square" in the source code. This will show the correct value.
• Now go to the expressions window and type square + " ok". In working versions of Eclipse you'll see the correct output. In non-working versions, there will be no output.
• Go back to the source file and select "square" with the mouse and press ctrl+shift+I. A dialog will pop up with the correct value in it in working versions of eclipse. In non working versions, you see the error message I originally reported:

To perform an evaluation, an expression must be compiled in the context of a Java project's build path. The current execution context is not associated with a Java project in the workspace`

Note that in non-working versions of Eclipse, when you hit the breakpoint, you'll be prompted to re-enter the source link to the external java app javaTestApp/src, even though it's already in the project. This doesn't happen in working versions of Eclipse, so that issue may be related to expressions not working. Re-entering the path doesn't fix expressions, but Eclipse requires it.

Working version: Oxygen.2 Release (4.7.2)
Broken versions: Eclipse 2023-09 and 2023-12

[srikanth-sankaran]

@iloveeclipse @jarthana , @mpalat : Looks like there is no current owner for jdt.debug - what is the recommendation for this ?

[iloveeclipse]

what is the recommendation for this

I've moved to jdt.debug.

We use remote debugging too in our application for years (and that with 4.21 / 4.25 / 4.30) and so far I haven't heard any complaints about broken breakpoint evaluation in this area.

So I assume whatever is broken here is "special" to the way the classpath and sourcepaths are set in the launch configuration, so someone who is interested on the fix should double check that all the dependencies required to resolve breakpoint related code are "visible" through the "Source lookup path" configuration tab.

Also one could play with "Java -> Debug -> Use advanced source lookup" that might be needed to be disabled or enabled depending on the use case.

[stephendnicholas]

I've raised a discussion thread which I think could be related: #406

[glhez]

what is the recommendation for this

I've moved to jdt.debug.

We use remote debugging too in our application for years (and that with 4.21 / 4.25 / 4.30) and so far I haven't heard any complaints about broken breakpoint evaluation in this area.

So I assume whatever is broken here is "special" to the way the classpath and sourcepaths are set in the launch configuration, so someone who is interested on the fix should double check that all the dependencies required to resolve breakpoint related code are "visible" through the "Source lookup path" configuration tab.

Also one could play with "Java -> Debug -> Use advanced source lookup" that might be needed to be disabled or enabled depending on the use case.

I'm pretty sure this still happens nowadays at least in JUnit 4/5 tests and Tomcat 8.5/9 Server : in fact, it is easy to track when it will fails, Eclipse will open the class in a "special" editor without no breadcrumb, no class name - only the current method.
In such case, the debug won't have any context and any expression/conditional breakpoint that rely on compiling will fail.
I also think that navigation between classes is affected too because the editor is kind of broken.

You can also see a simple example and some case where it failed in this issued I created : #86

[CalculonPrime]

This bug is serious enough that we're stuck using an older version of Eclipse without the issue so we can debug effectively, so I'm a bit surprised to see that it hasn't even been assigned to anyone or scheduled to be fixed in a particular release.

Eventually, we'll hit other issues using the older version, such as lack of support for newer JDKs, or perhaps incompatibilities with the OS. I like Eclipse so I filed this bug hoping it would be fixed before that happens, but if not we may shift to IntelliJ.

[srikanth-sankaran]

@SarikaSinha - Perhaps you have some volunteer cycles you can spare ?

[SarikaSinha]

@CalculonPrime Have you tested this in an Eclipse version of 2020, 2021, 2022 ?
Does it work and breaks only in 2023?
If you can test and let us know till when did it work, it will help us to find the change which has caused this.
We did not have any major change in Expression evaluation directly so knowing this will help us.