-
Notifications
You must be signed in to change notification settings - Fork 199
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Render a root-cause exception for dependency and join errors #3717
Open
benclifford
wants to merge
33
commits into
master
Choose a base branch
from
benc-dependency-error-rendering
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
was removed in #1773 then bad reintroduced into taskrecord as part of mypy work
…his still shows the error correctly
…or-rendering Conflicts: parsl/app/futures.py parsl/dataflow/errors.py
…or-rendering' into benc-dependency-error-rendering
…or-rendering Conflicts: parsl/dataflow/dflow.py
benclifford
changed the title
Print a root-cause exception for DependencyError
Render a root-cause exception for dependency and join errors
Jan 15, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR reworks two exception types, DependencyError and JoinError. Both of these exceptions report that a task failed because some other task/future failed - in the dependency case, because a task dependency failed, and in the join case because one of the tasks/futures being joined failed.
This PR introduces a common superclass
PropagatedException
to acknowledge that the meaning and behaviour of these two exceptions is very similar.PropagatedException
has a new implementation for reporting the failures that are being propagated. Parsl has tried a couple of ways to do this in the past:The implementation immediately before this PR reports only the immediate task IDs (or future reprs, for non-tasks) in the exception message. For details of the chain of exceptions and original/non-propagated exception, the user can examine the exception object via the
dependent_exceptions_tids
attribute.Prior to PR Make dependency exceptions only report task ID, not full exception tree #1802, the repr/str (and so the printed form) of dependency exceptions rendered the entire exception. In the case of deep dependency chains or where a dependency graph has many paths to a root cause, this resulted in extremely voluminous output with a lot of boiler plate dependency exception text.
The approach introduced by this current PR attempts a fusion of these two approaches:
(+ others)
__cause__
magic attribute which is usually populated byraise e1 from e2
. This PR populates that magic attribute at construction so that displaying the exception will show the cause using Python's native format.dependent_exceptions_tids
attribute for such introspection.A dependency or join error is now rendered by Python as exactly two exceptions next to each other:
Changed Behaviour
DependencyErrors and JoinErrors will render differently
Type of change