Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] AutoTuner may not find the latest RAPIDS jar version #704

Closed
amahussein opened this issue Dec 26, 2023 · 0 comments · Fixed by #711
Closed

[BUG] AutoTuner may not find the latest RAPIDS jar version #704

amahussein opened this issue Dec 26, 2023 · 0 comments · Fixed by #711
Assignees
@amahussein
Labels
bug Something isn't working core_tools Scope the core module (scala)

Comments

@amahussein
Copy link
Collaborator

Describe the bug

The AutoTuner parses the HTML of https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/ to get the latest Spark RAPIDS jar to do a recommendation about updating the plugin jars.
Similar to #702 , that directory list may have some delays not showing the latest releases.
We need to fix the implementation to parse https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/ maven-metadata.xml instead. This guarantees that the AutoTuner always finds the latest version.
@amahussein amahussein added bug Something isn't working ? - Needs Triage core_tools Scope the core module (scala) labels Dec 26, 2023
@amahussein amahussein self-assigned this Dec 26, 2023
amahussein added a commit to amahussein/spark-rapids-tools that referenced this issue Dec 29, 2023
@amahussein
AutoTuner should poll maven-meta to retrieve the latest jar version
b1a2dfd 
Signed-off-by: Ahmed Hussein (amahussein) <[email protected]>

Fixes NVIDIA#704

The AutoTuner could show an old version if the list directory is not
updated.
This change is to make sure that the AutoTuner can find the latest
version by parsing maven-metadata.xml
@amahussein amahussein mentioned this issue Dec 29, 2023
Merged
amahussein added a commit that referenced this issue Jan 3, 2024
@amahussein
AutoTuner should poll maven-meta to retrieve the latest jar version (#…
c85f68c 
…711)

* AutoTuner should poll maven-meta to retrieve the latest jar version

Signed-off-by: Ahmed Hussein (amahussein) <[email protected]>

Fixes #704

The AutoTuner could show an old version if the list directory is not
updated.
This change is to make sure that the AutoTuner can find the latest
version by parsing maven-metadata.xml
---------

Signed-off-by: Ahmed Hussein (amahussein) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@amahussein amahussein

Labels
bug Something isn't working core_tools Scope the core module (scala)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

AutoTuner should poll maven-meta to retrieve the latest jar version amahussein/spark-rapids-tools
1 participant
@amahussein