Performance Tests still relevant?

[laeubi]

I noticed today that https://ci.eclipse.org/releng/job/PerformanceTests/ blocks specific build executors, and it seem all of them are having failing tests.

Does anyone look at them (and take actions?) or are they maybe irrelevant these days and we could/should delete them? Especially on virtualisier hardware its questionable they give reliable/comparable results.

[akurtakov]

I am not aware of anyone doing so and since tests run in a container results are not reliable. E.g. https://download.eclipse.org/eclipse/downloads/drops4/S-4.33RC1-202408220100/performance/basicPerformance.php?name=org.eclipse.jdt.debug&build=I20240822-0100&baseline=R-4.32-202406010610 and https://download.eclipse.org/eclipse/downloads/drops4/I20240823-0600/performance/basicPerformance.php?name=org.eclipse.jdt.debug&build=I20240823-0600&baseline=R-4.32-202406010610 show quite different results without a change (https://download.eclipse.org/eclipse/downloads/drops4/I20240823-0600/gitLog.php) .
IMO we should remove them unless someone finds a way to hook real dedicated machine to run the tests so we get stable results.

[akurtakov]

I'll disable running them as said in #828 (comment)