Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable features via config that are off by default in the profiler AutoTuner #668

Merged
merged 5 commits into from
Dec 5, 2023
Merged

Enable features via config that are off by default in the profiler AutoTuner #668

merged 5 commits into from
Dec 5, 2023

Conversation

parthosa
Copy link
Collaborator

@parthosa parthosa commented Nov 21, 2023 •
edited
Loading

Fixes #577. This PR processed the driver logs and recommends experimental configs for unsupported operators in AutoTuner.

Changes:

  1. Process driver logs and collect unsupported operators.
  2. Provide these unsupported operators to AutoTuner and recommend config if it can run on GPU using experimental settings with a warning.

Note: We need to explicitly run AutoTuner if the event logs are not passed but driver logs are passed.

Testing:

Unit tests to cover cases:

  1. Recommendations generated for unsupported operators from driver logs only
  2. Recommendations generated for unsupported operators from driver and event logs
  3. Recommendations generated for empty unsupported operators from driver logs only
  4. Recommendations not generated for unsupported operators from driver logs
    • These operators do not have any recommendation

@parthosa parthosa self-assigned this Nov 21, 2023
@parthosa parthosa force-pushed the spark-rapids-tools-577 branch 2 times, most recently from 54749ac to 449fdaf Compare November 28, 2023 22:10
@parthosa parthosa force-pushed the spark-rapids-tools-577 branch from 449fdaf to a3c981f Compare November 29, 2023 00:50
@parthosa
Fix merge conflicts
42d44c6 
Signed-off-by: Partho Sarthi <[email protected]>
@parthosa parthosa marked this pull request as ready for review November 30, 2023 18:24
@parthosa parthosa added the core_tools Scope the core module (scala) label Nov 30, 2023
@parthosa
Fix missing methods due to merging
69fdd29 
Signed-off-by: Partho Sarthi <[email protected]>
@parthosa
Address review comments
ad7df5d 
Signed-off-by: Partho Sarthi <[email protected]>
cindyyuanjiang
cindyyuanjiang approved these changes Dec 5, 2023
View reviewed changes
Copy link
Collaborator

@cindyyuanjiang cindyyuanjiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @parthosa! LGTM.

@parthosa parthosa requested a review from nartal1 December 5, 2023 15:41
nartal1
nartal1 approved these changes Dec 5, 2023
View reviewed changes
Copy link
Collaborator

@nartal1 nartal1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @parthosa

@parthosa parthosa merged commit 6e8f1f5 into NVIDIA:dev Dec 5, 2023
9 checks passed
@parthosa parthosa deleted the spark-rapids-tools-577 branch December 5, 2023 17:38
@parthosa parthosa added the feature request New feature or request label Dec 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nartal1 nartal1 nartal1 approved these changes

@cindyyuanjiang cindyyuanjiang cindyyuanjiang approved these changes

@mattahrens mattahrens Awaiting requested review from mattahrens

Assignees

@parthosa parthosa

Labels
core_tools Scope the core module (scala) feature request New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[FEA] Enable features via config that are off by default in the profiler auto-tuner
3 participants
@parthosa @cindyyuanjiang @nartal1