Add sub-workflow support

[samuell]

Allow chaining WorkflowTask tasks together. Might not be fully possible yet.

[proycon]

I'm looking for similar functionality and just stumbled upon the fact that this is not possible currently.

[samuell]

@proycon Thanks for letting us know! We should look at this soonish ... should not need too big a change.

[proycon]

Thanks, looking forward to it!

[ddd-bbb]

My solution to this (and #18) was to create a new SubWorkflowTask class that caches its workflow definition on initialization and so lets you specify your own out ports in case you want them available to other workflows:

class SubWorkflowTask(sciluigi.WorkflowTask):
    def __init__(self, *args, **kwargs):
        super(sciluigi.WorkflowTask, self).__init__(*args,**kwargs)
        self.cached_workflow = self.workflow_logic()

    def workflow(self):
        return self.cached_workflow

    def workflow_logic(self):
        """
        Contains all the logic that would normall go in self.workflow, but you
        can also define outputs (self.out_*) that will be visible elsewhere
        
        for example:

        my_task = self.new_task("do_something", DoSomething)
        self.out_foo = my_task.out_foo

        return my_task
        """
        raise Exception("workflow_logic not implemented")

Curious to hear what you think of this @samuell, not sure what issues I may be ignoring.