DAG Import Errors

[FredrikBakken]

Versions

Airflow: 2.3.4
LakeFS: 0.80.1
LakeFS Provider: 0.44.0

Error Message

Broken DAG: [/home/$USER/airflow/dags/ingest/ingest.py] Traceback (most recent call last):
  File "/home/$USER/.local/lib/python3.10/site-packages/airflow/models/taskmixin.py", line 244, in <listcomp>
    return [self.dag.get_task(tid) for tid in self.downstream_task_ids]
  File "/home/$USER/.local/lib/python3.10/site-packages/airflow/models/dag.py", line 2115, in get_task
    raise TaskNotFound(f"Task {task_id} not found")
airflow.exceptions.TaskNotFound: Task get_branch_commit not found

Operators with Error Message

Operator	Does it have the error?
LakeFSCommitOperator	✔️
LakeFSCreateBranchOperator
LakeFSGetCommitOperator	✔️
LakeFSGetObjectOperator
LakeFSMergeOperator	✔️
LakeFSUploadOperator	✔️

Correlation

The correlation between the operators with the error message and the operators that does not have the error message is that;

self.task_id = kwargs.get("task_id")

is noted in the __init__() function for all the operators with the DAG error. The two operators that does not have this error message, does not have self.task_id mentioned in the __init__() function. Removing this line from the operators that have the DAG error, makes them work as expected.

[FredrikBakken]

@arielshaqed , I've added a new comment to the closed PR (#35 (comment)). Please let me know if there is anything else I could help out with 😀