Subclassing LocalTarget and TargetInfo causing sciluigi run to fail

[BioComSoftware]

Hi,

A quick suggestion...

The method…

sciluigi.dependencies.DependencyHelpers._parse_outputitem()
(see below)

…is used to verify an 'out target' is either a 'TargetIfo' object a 'list' or a 'dict'.

But, since the target exists() method is used by (sci)luigi to decide if it will run the command (again) – AND exists() ONLY checks if the output directory 'exists' (not that it has useful data) I've found I often need to subclass the Target or LocalTarget class.

For example I subclassed the LUIGI LocalTarget class with 'class ToppasLocalTarget(LocalTarget)' which decides whether the out_target subdirectory structure matches the expected output of the TOPPAS file and – if it does not – returns False for exists().

Point being, the _parse_outputitem() does not recognize my subclass, and errors out the run with:

raise Exception('Input item is neither callable, TargetInfo, nor list: %s' % val)
Exception: Input item is neither callable, TargetInfo, nor list: <Bioproximity.common.luigi_extensions.local_target.ToppasLocalTarget object at 0x10d9d1190>

It might be useful to have it instead check the root parent class, instead of the class itself – otherwise my child class fails the sciluigi run because it doesn't recognize it as a valid target :)

Thanks :)

def _parse_outputitem(self, val, targets):
        '''
        Recursively loop through lists of TargetInfos, or
        callables returning TargetInfos, or lists of ...
        (repeat recursively) ... and return all targets.
        '''
        if callable(val):
            val = val()
        if isinstance(val, TargetInfo):
            targets.append(val.target)
        elif isinstance(val, list):
            for valitem in val:
                targets = self._parse_outputitem(valitem, targets)
        elif isinstance(val, dict):
            for _, valitem in iteritems(val):
                targets = self._parse_outputitem(valitem, targets)
        else:
            raise Exception('Input item is neither callable, TargetInfo, nor list: %s' % val)
        return targets

[BioComSoftware]

Ah. I just realized that TargetInfo is not a subclass of the luigi.Target classes. I think my answer is to subclass targetinfo and not luigi.LocalTarget