Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Profiler tool: Get the stats of unsupported operators based on driver log and event logs. #641

Open
nartal1 opened this issue Oct 30, 2023 · 0 comments
Labels
core_tools Scope the core module (scala) feature request New feature or request

Comments

@nartal1
Copy link
Collaborator

nartal1 commented Oct 30, 2023

Is your feature request related to a problem? Please describe.
This is the follow on of issue - #580. Once we have support for driver log in Profiler tool, we should extend it to read both driver log and eventlogs and output the stats for unsupported operators. This could help in understanding why a certain query might be slow and also helps in prioritizing to add GPU support for unsupported operators.

For example: In a query if there are 2 unsupported operators, and when we look at the eventlogs one operator produces 100 rows(aggregate) and the other operator produces 10^9 rows(aggregate), the output should contain the aggregated number of rows for all the unsupported operators.

@nartal1 nartal1 added feature request New feature or request ? - Needs Triage core_tools Scope the core module (scala) labels Oct 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core_tools Scope the core module (scala) feature request New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants