Adding more informative error report messages

[omar-rifai]

In the example below, the error message in _error.pklz does not relate to the actual problem encountered. Namely, for the following code:

from pydra import Workflow, Submitter
from pydra.tasks.nipype1.utils import Nipype1Task

from nipype.interfaces.ants.registration import RegistrationSynQuick


ants_reg = RegistrationSynQuick(transform_type="br", dimension=3)
in_t1w = "/path/to/t1/"
in_ref = "/path/to/ref/"

workflow = workflow = Workflow(
        name="test_workflow",
        input_spec=["input_dir"],
)

workflow.add(
     Nipype1Task(
         name="ants_reg",
         interface=ants_reg,
         moving_image=in_t1w,
         fixed_image=in_ref,
     )
 )

workflow.set_output(
        [ ("out_image", workflow.ants_reg.lzout.warped_image)]
    )

with Submitter("cf") as submitter:
    res = submitter(workflow)
    print(res)

The error reported in _error.pklz is the following:

raise BadGzipFile('Not a gzipped file (%r)' % magic)
gzip.BadGzipFile: Not a gzipped file (b'\x80\x05')

Which is generic and not related to the actual problem in this situation. The actual error message is printed when running the Nipype1Task interface ants_reg seperately:

OSError: No command "antsRegistrationSyNQuick.sh" found on host xxx-xxx. Please check that the corresponding package is installed.

It would be great if this error could be propagated to the workflow and outputted in the console or found in the relevant _error.pklz file

[effigies]

This might be more of an issue with Nipype1Task failing to proxy exceptions correctly.

[omar-rifai]

@effigies, should I create the issue there for reference?

[effigies]

Hmm. This is what I'm finding in the _error.pklz:

In [17]: import cloudpickle as cp

In [18]: with open('/tmp/tmp708qhxhb/Nipype1Task_73123a85d3daade462a868be921f7b62c0ac5b71e8b4216fc92d33f398696a1d/_error.pklz', 'rb') as fp:
    ...: 
    ...:     err = cp.load(fp)
    ...: 

In [19]: err
Out[19]: 
{'time of crash': '20220729-143942',
 'login name': 'chris',
 'name with checksum': 'Nipype1Task_73123a85d3daade462a868be921f7b62c0ac5b71e8b4216fc92d33f398696a1d',
 'error message': ['Traceback (most recent call last):\n',
  '  File "/home/chris/Projects/nipype2/pydra/pydra/engine/core.py", line 506, in _run\n    self._run_task()\n',
  '  File "/home/chris/Projects/nipype2/pydra-nipype1/pydra/tasks/nipype1/utils.py", line 74, in _run_task\n    res = node.run()\n',
  '  File "/home/chris/miniconda3/envs/pydra-dev/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 516, in run\n    result = self._run_interface(execute=True)\n',
  '  File "/home/chris/miniconda3/envs/pydra-dev/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 635, in _run_interface\n    return self._run_command(execute)\n',
  '  File "/home/chris/miniconda3/envs/pydra-dev/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 741, in _run_command\n    result = self._interface.run(cwd=outdir)\n',
  '  File "/home/chris/miniconda3/envs/pydra-dev/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 434, in run\n    runtime = self._run_interface(runtime)\n',
  '  File "/home/chris/miniconda3/envs/pydra-dev/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 815, in _run_interface\n    raise IOError(\n',
  'OSError: No command "antsRegistrationSyNQuick.sh" found on host lamarr. Please check that the corresponding package is installed.\n']}

Is it possible you're loading the wrong pickle file?

[omar-rifai]

I was just copy pasting the link in the error message but I was using nipypecli though, instead of loading the file with cloudpickle. Could that be it ? (I'm afk for now but I can check on monday)

[effigies]

If you're using nipype1's nipypecli, probably the failure you're seeing is a failure to load the pickle file, not its contents. Nipype uses pickle, while pydra uses cloudpickle, and it looks like we forgot to actually make .pklz be gzipped.

[omar-rifai]

Clear! Sorry about the confusion and thanks for the help :-)