Skip to content

Commit

Permalink
Fix dramatiq implementation. Add tasks.
Browse files Browse the repository at this point in the history
  • Loading branch information
mgdaily committed Feb 7, 2024
1 parent efc53b7 commit 3b49c88
Showing 1 changed file with 13 additions and 0 deletions.
13 changes: 13 additions & 0 deletions datalab/datalab_session/tasks.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
import dramatiq
from datalab.datalab_session.data_operations.data_operation import available_operations

#TODO: Perhaps define a pipeline that can take the output of one data operation and upload to a s3 bucket, indicate success, etc...

@dramatiq.actor()
def execute_data_operation(data_operation_name: str, input_data: dict):
clz = available_operations().get(data_operation_name)
if clz is None:
raise NotImplementedError("Operation not implemented!")
else:
instance = clz()
instance.operate(input_data)

0 comments on commit 3b49c88

Please sign in to comment.