Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Profiler ignores applications in incomplete eventlogs #685

Closed
amahussein opened this issue Dec 12, 2023 · 0 comments · Fixed by #686
Closed

[BUG] Profiler ignores applications in incomplete eventlogs #685

amahussein opened this issue Dec 12, 2023 · 0 comments · Fixed by #686
Assignees
@amahussein
Labels
bug Something isn't working core_tools Scope the core module (scala)

Comments

@amahussein
Copy link
Collaborator

Describe the bug

An "inprogress" eventlog throws an JSON exception that causes the entire application to be skipped.
We should ignore incomplete json files hopefully to do a best effort in gathering the events that we have.

Steps/Code to reproduce bug

  • generate an eventlog
  • modify the eventlog by removing some few characters from the tail
  • Run the profiler on the modified file, it will not be able to analyze the app. The app will be considered as N/A

Expected behavior

Show a warning message for unexpected Json EOF instead of throwing an exception.
At least the user is informed that the eventlog is incomplete.
@amahussein amahussein added bug Something isn't working ? - Needs Triage core_tools Scope the core module (scala) labels Dec 12, 2023
@amahussein amahussein self-assigned this Dec 12, 2023
amahussein added a commit to amahussein/spark-rapids-tools that referenced this issue Dec 12, 2023
@amahussein
Support parsing of inprogress eventlogs
5c4b67f 
Signed-off-by: Ahmed Hussein (amahussein) <[email protected]>

Fixes NVIDIA#685

* Added feature: support parsing of "inprogress" eventlogs
* Catches exception thrown by Json Parser when an unexpected EOF occurs
@amahussein amahussein mentioned this issue Dec 12, 2023
Merged
amahussein added a commit that referenced this issue Dec 13, 2023
@amahussein
Support parsing of inprogress eventlogs (#686)
e298c53 
* Support parsing of inprogress eventlogs

Fixes #685

* Added feature: support parsing of "inprogress" eventlogs
* Catches exception thrown by Json Parser when an unexpected EOF occurs

---------

Signed-off-by: Ahmed Hussein (amahussein) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@amahussein amahussein

Labels
bug Something isn't working core_tools Scope the core module (scala)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support parsing of inprogress eventlogs amahussein/spark-rapids-tools
1 participant
@amahussein