Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove ReusedSubquery from SparkPlanGraph construction #741

Merged
merged 2 commits into from
Jan 26, 2024

Conversation

amahussein
Copy link
Collaborator

@amahussein amahussein commented Jan 25, 2024

Signed-off-by: Ahmed Hussein (amahussein) [email protected]

Fixes #718

  • Reused subqueries should be excluded from the metrics.
  • Added a common graph builder to be used for both CPU/GPU logs
  • Added the unit test for GPU eventlogs

I filed #742 as a followup to deal with code duplicates and investigate Qualification SQLPlanParser

Signed-off-by: Ahmed Hussein (amahussein) <[email protected]>

Fixes NVIDIA#718

- Reused subqueries should be excluded from the metrics.
- Added a common graph builder to be used for both CPU/GPU logs
- Added the unit test for GPU eventlogs
@amahussein amahussein added bug Something isn't working core_tools Scope the core module (scala) labels Jan 25, 2024
@amahussein amahussein self-assigned this Jan 25, 2024
@amahussein amahussein requested a review from nartal1 January 25, 2024 02:58
nartal1
nartal1 previously approved these changes Jan 26, 2024
Copy link
Collaborator

@nartal1 nartal1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @amahussein! LGTM. Just a nit to add comment.

Signed-off-by: Ahmed Hussein (amahussein) <[email protected]>
@amahussein amahussein merged commit 5bed34c into NVIDIA:dev Jan 26, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working core_tools Scope the core module (scala)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Profiler ignoring some reused subqueries
2 participants