Error during training on charades

[satoshinishimura2460]

When running train_net.py as of Stage 1 at Charades two-stage training in GETTING_STARTED.md, I met this error.

  [INFO: misc.py:  169]: Total conv/fc/matMul FLOPs: 190.646642688(e9)
  [INFO: misc.py:  170]: Total conv/fc params: 34.83232(e6)
  Segmentation fault
  (venv) [aac12391zj@g0034 video-long-term-feature-banks]$ Process PoolWorker-92:
  Traceback (most recent call last):
  File "/apps/centos7/python/2.7.18/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
  self.run()
  File "/apps/centos7/python/2.7.18/lib/python2.7/multiprocessing/process.py", line 114, in run
  self._target(*self._args, **self._kwargs)
  File "/apps/centos7/python/2.7.18/lib/python2.7/multiprocessing/pool.py", line 122, in worker
  put((job, i, (False, wrapped)))
  File "/apps/centos7/python/2.7.18/lib/python2.7/multiprocessing/queues.py", line 390, in put
  return send(obj)
  IOError: [Errno 32] Broken pipe

I used python 2.7.18, cuda 10.0.130.1, cudnn 7.6.5. And, I installed PyTorch 1.4 based on this issue.
I replaced both the GPU_NUM and BATCH_SIZE to 4 because of the limitation of our GPU setting.

I appreciate it if anyone provides any suggestion to solve this Segmentation fault and Broken pipe issue.
Thank you so much.

The detailed log is shown in the following.

(venv) [aac12391zj@g0034 video-long-term-feature-banks]$ python tools/train_net.py --config configs/charades_r50_baseline.yaml CHECKPOINT.DIR ./tmp/lfb-nl-step1-output
Ignoring @/caffe2/caffe2/contrib/gloo:gloo_ops as it is not a valid file.
Ignoring @/caffe2/caffe2/contrib/nccl:nccl_ops as it is not a valid file.
Ignoring @/caffe2/caffe2/contrib/gloo:gloo_ops_gpu as it is not a valid file.
Found Detectron ops lib: /groups/1/gac50586/migrated_from_SFA_GPFS/NISHIMURA/Charades/video-long-term-feature-banks/venv/lib/python2.7/site-packages/torch/lib/libcaffe2_detectron_ops_gpu.so
/home/aac12391zj/NISHIMURA/Charades/video-long-term-feature-banks/lib/core/config.py:427: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  yaml_config = AttrDict(yaml.load(fopen))
[INFO: config.py:  369]: Config:
[INFO: config.py:  370]: {'AVA': {'ANNOTATION_DIR': 'data/ava/annotations',
         'DETECTION_SCORE_THRESH_EVAL': [0.85],
         'DETECTION_SCORE_THRESH_TRAIN': 0.9,
         'FORCE_TEST_FLIP': False,
         'FPS': 30,
         'FRAME_LIST_DIR': 'data/ava/frame_lists',
         'FULL_EVAL_DURING_TRAINING': False,
         'LFB_DETECTION_SCORE_THRESH': 0.9,
         'LFB_MAX_NUM_FEAT_PER_STEP': 5,
         'TEST_BOX_LISTS': ['ava_val_predicted_boxes.csv'],
         'TEST_LFB_BOX_LISTS': ['ava_val_predicted_boxes.csv'],
         'TEST_LISTS': ['val.csv'],
         'TEST_MULTI_CROP': False,
         'TEST_MULTI_CROP_SCALES': [224, 256, 320],
         'TEST_ON_TEST_SET': False,
         'TRAIN_BOX_LISTS': ['ava_train_v2.1.csv',
                             'ava_train_predicted_boxes.csv'],
         'TRAIN_LFB_BOX_LISTS': ['ava_train_predicted_boxes.csv'],
         'TRAIN_LISTS': ['train.csv'],
         'TRAIN_ON_TRAIN_VAL': False},
 'CHARADES': {'FPS': 24,
              'FRAME_LIST_DIR': 'data/charades/frame_lists',
              'LFB_CLIPS_PER_SECOND': 2,
              'NUM_TEST_CLIPS_DURING_TRAINING': 9,
              'NUM_TEST_CLIPS_FINAL_EVAL': 30,
              'TEST_LISTS': ['val.csv'],
              'TRAIN_LISTS': ['train.csv']},
 'CHECKPOINT': {'CHECKPOINT_MODEL': True,
                'CHECKPOINT_PERIOD': 4000,
                'CONVERT_MODEL': True,
                'DIR': './tmp/lfb-nl-step1-output',
                'RESUME': True},
 'CUDNN_WORKSPACE_LIMIT': 256,
 'DATADIR': 'data/charades/frames',
 'DATALOADER': {'MAX_BAD_IMAGES': 100},
 'DATASET': 'charades',
 'DATA_MEAN': [0.45, 0.45, 0.45],
 'DATA_STD': [0.225, 0.225, 0.225],
 'DEBUG': False,
 'EPIC': {'ANNOTATIONS': 'EPIC_train_action_labels.csv',
          'ANNOTATION_DIR': 'data/epic/annotations',
          'CLASS_TYPE': '',
          'FPS': 30,
          'FRAME_LIST_DIR': 'data/epic/frame_lists',
          'MAX_NUM_FEATS_PER_NOUN_LFB_FRAME': 10,
          'NOUN_LFB_FRAMES_PER_SECOND': 1,
          'TEST_LISTS': ['val.csv'],
          'TRAIN_LISTS': ['train.csv'],
          'VERB_LFB_CLIPS_PER_SECOND': 1},
 'FBO_NL': {'DROPOUT_RATE': 0.2,
            'INPUT_DROPOUT_ON': True,
            'INPUT_REDUCE_DIM': True,
            'LATENT_DIM': 512,
            'LFB_DROPOUT_ON': True,
            'NL_DROPOUT_ON': True,
            'NUM_LAYERS': 2,
            'PRE_ACT': True,
            'PRE_ACT_LN': True,
            'SCALE': True},
 'GET_TRAIN_LFB': False,
 'IMG_LOAD_RETRY': 10,
 'INTERPOLATION': 'INTER_LINEAR',
 'LFB': {'ENABLED': False,
         'FBO_TYPE': 'nl',
         'LFB_DIM': 2048,
         'LOAD_LFB': False,
         'LOAD_LFB_PATH': '',
         'MODEL_PARAMS_FILE': '',
         'NUM_LFB_FEAT': 500,
         'WINDOW_SIZE': 100,
         'WRITE_LFB': False},
 'LOG_PERIOD': 10,
 'MINIBATCH_QUEUE_SIZE': 64,
 'MODEL': {'ALLOW_INPLACE_RELU': True,
           'ALLOW_INPLACE_RESHAPE': True,
           'ALLOW_INPLACE_SUM': True,
           'BN_EPSILON': 1.0000001e-05,
           'BN_INIT_GAMMA': 0.0,
           'BN_MOMENTUM': 0.9,
           'DEPTH': 50,
           'DILATIONS_AFTER_CONV5': True,
           'FC_INIT_STD': 0.01,
           'FREEZE_BACKBONE': False,
           'MEAN': 114.75,
           'MEMONGER': True,
           'MODEL_NAME': 'resnet_video',
           'MULTI_LABEL': True,
           'NUM_CLASSES': 157,
           'SAMPLE_THREADS': 8,
           'STD': 57.375,
           'USE_AFFINE': True,
           'USE_BGR': False,
           'VIDEO_ARC_CHOICE': 2},
 'NONLOCAL': {'BN_EPSILON': 1.0000001e-05,
              'BN_INIT_GAMMA': 0.0,
              'BN_MOMENTUM': 0.9,
              'CONV3_NONLOCAL': True,
              'CONV4_NONLOCAL': True,
              'CONV_INIT_STD': 0.01,
              'LAYER_MOD': 2,
              'NO_BIAS': 0,
              'USE_AFFINE': True,
              'USE_BN': False,
              'USE_MAXPOOL': True,
              'USE_SCALE': True,
              'USE_SOFTMAX': True,
              'USE_ZERO_INIT_CONV': True},
 'NUM_GPUS': 4,
 'PROF_DAG': False,
 'RESNETS': {'NUM_GROUPS': 1,
             'STRIDE_1X1': False,
             'TRANS_FUNC': 'bottleneck_transformation_3d',
             'WIDTH_PER_GROUP': 64},
 'RNG_SEED': 2,
 'ROI': {'SCALE_FACTOR': 16, 'XFORM_RESOLUTION': 7},
 'ROOT_GPU_ID': 0,
 'SOLVER': {'BASE_LR': 0.02,
            'GAMMA': 0.1,
            'LRS': [1, 0.1],
            'LR_POLICY': 'steps_with_relative_lrs',
            'MAX_ITER': 24000,
            'MOMENTUM': 0.9,
            'NESTEROV': True,
            'SCALE_MOMENTUM': True,
            'SCALE_MOMENTUM_THRESHOLD': 1.1,
            'STEPS': [0, 20000, 24000],
            'STEP_SIZES': [20000, 4000],
            'WARMUP': {'WARMUP_END_ITER': 5000,
                       'WARMUP_ON': False,
                       'WARMUP_START_LR': 0.1},
            'WEIGHT_DECAY': 1.25e-05,
            'WEIGHT_DECAY_BN': 0.0},
 'TEST': {'BATCH_SIZE': 4,
          'CROP_SHIFT': 1,
          'CROP_SIZE': 256,
          'DATASET_SIZE': 1814,
          'DATA_TYPE': 'val',
          'PARAMS_FILE': '',
          'SAMPLE_RATE': 4,
          'SCALE': 256,
          'VIDEO_LENGTH': 32},
 'TRAIN': {'BATCH_SIZE': 4,
           'COMPUTE_PRECISE_BN': False,
           'CROP_SIZE': 224,
           'DATASET_SIZE': 7811,
           'DATA_TYPE': 'train',
           'DROPOUT_RATE': 0.3,
           'EVAL_PERIOD': 4000,
           'ITER_COMPUTE_PRECISE_BN': 200,
           'JITTER_SCALES': [256, 320],
           'PARAMS_FILE': 'pretrained_weights/r50_k400_pretrained.pkl',
           'PCA_EIGVAL': [0.225, 0.224, 0.229],
           'PCA_EIGVEC': [[-0.5675, 0.7192, 0.4009],
                          [-0.5808, -0.0045, -0.814],
                          [-0.5836, -0.6948, 0.4203]],
           'RESET_START_ITER': True,
           'RESUME_FROM_BATCH_SIZE': -1,
           'SAMPLE_RATE': 4,
           'TEST_AFTER_TRAIN': True,
           'USE_COLOR_AUGMENTATION': False,
           'VIDEO_LENGTH': 32},
 'USE_CYTHON': False}
[E init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
[E init_intrinsics_check.cc:43] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
[E init_intrinsics_check.cc:43] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
[INFO: train_net.py:  104]: Checkpoint directory created: /groups/1/gac50586/migrated_from_SFA_GPFS/NISHIMURA/Charades/video-long-term-feature-banks/tmp/lfb-nl-step1-output/checkpoints
[WARNING: cnn.py:   25]: [====DEPRECATE WARNING====]: you are creating an object from CNNModelHelper class which will be deprecated soon. Please use ModelHelper object with brew module. For more information, please refer to caffe2.ai and python/brew.py, python/brew_test.py for more information.
[INFO: charades.py:  195]: === Charades dataset summary ===
[INFO: charades.py:  196]: Split: val
[INFO: charades.py:  197]: Use LFB? False
[INFO: charades.py:  198]: Spatial shift position: 1
[INFO: charades.py:  199]: Number of videos: 1814
[INFO: charades.py:  202]: Number of frames: 1254921
[INFO: charades_data_input.py:   51]: Creating the execution context for worker_ids: [100, 101, 102, 103], batch size: 1
[INFO: data_input_helper.py:  157]: CREATING EXECUTION CONTEXT
[INFO: data_input_helper.py:  164]: POOLS: {}
[INFO: data_input_helper.py:  165]: SHARED DATA LISTS: 0
[INFO: data_input_helper.py:  177]: worker_id: 100 list: 1
[INFO: data_input_helper.py:  179]: worker_id: 100 list keys: [100]
[INFO: data_input_helper.py:  177]: worker_id: 101 list: 2
[INFO: data_input_helper.py:  179]: worker_id: 101 list keys: [100, 101]
[INFO: data_input_helper.py:  177]: worker_id: 102 list: 3
[INFO: data_input_helper.py:  179]: worker_id: 102 list keys: [100, 101, 102]
[INFO: data_input_helper.py:  177]: worker_id: 103 list: 4
[INFO: data_input_helper.py:  179]: worker_id: 103 list keys: [100, 101, 102, 103]
[INFO: data_input_helper.py:  224]: CREATED POOL: {100: <multiprocessing.pool.Pool object at 0x2b4c57b23550>, 101: <multiprocessing.pool.Pool object at 0x2b4c57b2e850>, 102: <multiprocessing.pool.Pool object at 0x2b4c6371a990>, 103: <multiprocessing.pool.Pool object at 0x2b4c63724750>}
[INFO: data_input_helper.py:  225]: CREATED LISTS: 4
[INFO: data_input_helper.py:  226]: POOL keys: [100, 101, 102, 103]
[INFO: data_input_helper.py:  227]: LIST keys: [100, 101, 102, 103]
[INFO: data_parallel_model.py:  179]: Parallelizing model for devices: [0, 1, 2, 3]
[INFO: data_parallel_model.py:  202]: Create input and model training operators
[WARNING: data_parallel_model.py:  219]:
[WARNING: data_parallel_model.py:  220]: ############# WARNING #############
[WARNING: data_parallel_model.py:  222]: Model CNN/<models.model_builder_video.ModelBuilder object at 0x2b4c23886c90> is used for testing/validation but
[WARNING: data_parallel_model.py:  223]: has init_params=True!
[WARNING: data_parallel_model.py:  224]: This can conflict with model training.
[WARNING: data_parallel_model.py:  225]: Please ensure model = ModelHelper(init_params=False)
[WARNING: data_parallel_model.py:  226]: ####################################
[WARNING: data_parallel_model.py:  227]:
[INFO: data_parallel_model.py:  235]: Model for GPU : 0
[INFO: resnet_video.py:  150]: --------------- ResNet-50 1x64d-bottleneck_transformation_3d, charades, val, infer LFB? False, suffix: "_test" ---------------
[INFO: resnet_video.py:  166]: use_temp_convs_set: [[2], [1, 1, 1], [1, 0, 1, 0], [1, 0, 1, 0, 1, 0], [0, 1, 0]]
[INFO: resnet_video.py:  167]: temp_strides_set: [[1], [1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1, 1, 1], [1, 1, 1]]
[INFO: resnet_helper.py:   62]: res2_0 using dilation 1
[INFO: resnet_helper.py:   62]: res2_1 using dilation 1
[INFO: resnet_helper.py:   62]: res2_2 using dilation 1
[INFO: resnet_helper.py:   62]: res3_0 using dilation 1
[INFO: resnet_helper.py:   62]: res3_1 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res3_2 using dilation 1
[INFO: resnet_helper.py:   62]: res3_3 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_0 using dilation 1
[INFO: resnet_helper.py:   62]: res4_1 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_2 using dilation 1
[INFO: resnet_helper.py:   62]: res4_3 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_4 using dilation 1
[INFO: resnet_helper.py:   62]: res4_5 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res5_0 using dilation 2
[INFO: resnet_helper.py:   62]: res5_1 using dilation 2
[INFO: resnet_helper.py:   62]: res5_2 using dilation 2
[INFO: data_parallel_model.py:  235]: Model for GPU : 1
[INFO: resnet_video.py:  150]: --------------- ResNet-50 1x64d-bottleneck_transformation_3d, charades, val, infer LFB? False, suffix: "_test" ---------------
[INFO: resnet_video.py:  166]: use_temp_convs_set: [[2], [1, 1, 1], [1, 0, 1, 0], [1, 0, 1, 0, 1, 0], [0, 1, 0]]
[INFO: resnet_video.py:  167]: temp_strides_set: [[1], [1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1, 1, 1], [1, 1, 1]]
[INFO: resnet_helper.py:   62]: res2_0 using dilation 1
[INFO: resnet_helper.py:   62]: res2_1 using dilation 1
[INFO: resnet_helper.py:   62]: res2_2 using dilation 1
[INFO: resnet_helper.py:   62]: res3_0 using dilation 1
[INFO: resnet_helper.py:   62]: res3_1 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res3_2 using dilation 1
[INFO: resnet_helper.py:   62]: res3_3 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_0 using dilation 1
[INFO: resnet_helper.py:   62]: res4_1 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_2 using dilation 1
[INFO: resnet_helper.py:   62]: res4_3 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_4 using dilation 1
[INFO: resnet_helper.py:   62]: res4_5 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res5_0 using dilation 2
[INFO: resnet_helper.py:   62]: res5_1 using dilation 2
[INFO: resnet_helper.py:   62]: res5_2 using dilation 2
[INFO: data_parallel_model.py:  235]: Model for GPU : 2
[INFO: resnet_video.py:  150]: --------------- ResNet-50 1x64d-bottleneck_transformation_3d, charades, val, infer LFB? False, suffix: "_test" ---------------
[INFO: resnet_video.py:  166]: use_temp_convs_set: [[2], [1, 1, 1], [1, 0, 1, 0], [1, 0, 1, 0, 1, 0], [0, 1, 0]]
[INFO: resnet_video.py:  167]: temp_strides_set: [[1], [1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1, 1, 1], [1, 1, 1]]
[INFO: resnet_helper.py:   62]: res2_0 using dilation 1
[INFO: resnet_helper.py:   62]: res2_1 using dilation 1
[INFO: resnet_helper.py:   62]: res2_2 using dilation 1
[INFO: resnet_helper.py:   62]: res3_0 using dilation 1
[INFO: resnet_helper.py:   62]: res3_1 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res3_2 using dilation 1
[INFO: resnet_helper.py:   62]: res3_3 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_0 using dilation 1
[INFO: resnet_helper.py:   62]: res4_1 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_2 using dilation 1
[INFO: resnet_helper.py:   62]: res4_3 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_4 using dilation 1
[INFO: resnet_helper.py:   62]: res4_5 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res5_0 using dilation 2
[INFO: resnet_helper.py:   62]: res5_1 using dilation 2
[INFO: resnet_helper.py:   62]: res5_2 using dilation 2
[INFO: data_parallel_model.py:  235]: Model for GPU : 3
[INFO: resnet_video.py:  150]: --------------- ResNet-50 1x64d-bottleneck_transformation_3d, charades, val, infer LFB? False, suffix: "_test" ---------------
[INFO: resnet_video.py:  166]: use_temp_convs_set: [[2], [1, 1, 1], [1, 0, 1, 0], [1, 0, 1, 0, 1, 0], [0, 1, 0]]
[INFO: resnet_video.py:  167]: temp_strides_set: [[1], [1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1, 1, 1], [1, 1, 1]]
[INFO: resnet_helper.py:   62]: res2_0 using dilation 1
[INFO: resnet_helper.py:   62]: res2_1 using dilation 1
[INFO: resnet_helper.py:   62]: res2_2 using dilation 1
[INFO: resnet_helper.py:   62]: res3_0 using dilation 1
[INFO: resnet_helper.py:   62]: res3_1 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res3_2 using dilation 1
[INFO: resnet_helper.py:   62]: res3_3 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_0 using dilation 1
[INFO: resnet_helper.py:   62]: res4_1 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_2 using dilation 1
[INFO: resnet_helper.py:   62]: res4_3 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_4 using dilation 1
[INFO: resnet_helper.py:   62]: res4_5 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res5_0 using dilation 2
[INFO: resnet_helper.py:   62]: res5_1 using dilation 2
[INFO: resnet_helper.py:   62]: res5_2 using dilation 2
[INFO: data_parallel_model.py:  280]: Parameter update function not defined --> only forward
[I net_dag_utils.cc:102] Operator graph pruning prior to chain compute took: 0.000414924 secs
[INFO: model_builder_video.py:  160]: Starting data loader...
[INFO: dataloader.py:  366]: Pre-filling val minibatch queue
[INFO: dataloader.py:  369]: val minibatch queue pre-filled.
[INFO: metrics.py:  136]: Resetting val metrics...
[INFO: misc.py:   94]: CNN: Net proto saved to: /groups/1/gac50586/migrated_from_SFA_GPFS/NISHIMURA/Charades/video-long-term-feature-banks/tmp/lfb-nl-step1-output/CNN.pbtxt
[INFO: misc.py:   94]: CNN_init: Net proto saved to: /groups/1/gac50586/migrated_from_SFA_GPFS/NISHIMURA/Charades/video-long-term-feature-banks/tmp/lfb-nl-step1-output/CNN_init.pbtxt
[INFO: train_net.py:  124]: Test iters: 4082
[WARNING: cnn.py:   25]: [====DEPRECATE WARNING====]: you are creating an object from CNNModelHelper class which will be deprecated soon. Please use ModelHelper object with brew module. For more information, please refer to caffe2.ai and python/brew.py, python/brew_test.py for more information.
[INFO: charades.py:  195]: === Charades dataset summary ===
[INFO: charades.py:  196]: Split: train
[INFO: charades.py:  197]: Use LFB? False
[INFO: charades.py:  198]: Spatial shift position: 1
[INFO: charades.py:  199]: Number of videos: 7811
[INFO: charades.py:  202]: Number of frames: 5610191
[INFO: charades_data_input.py:   51]: Creating the execution context for worker_ids: [0, 1, 2, 3], batch size: 1
[INFO: data_input_helper.py:  157]: CREATING EXECUTION CONTEXT
[INFO: data_input_helper.py:  164]: POOLS: {100: <multiprocessing.pool.Pool object at 0x2b4c57b23550>, 101: <multiprocessing.pool.Pool object at 0x2b4c57b2e850>, 102: <multiprocessing.pool.Pool object at 0x2b4c6371a990>, 103: <multiprocessing.pool.Pool object at 0x2b4c63724750>}
[INFO: data_input_helper.py:  165]: SHARED DATA LISTS: 4
[INFO: data_input_helper.py:  177]: worker_id: 0 list: 5
[INFO: data_input_helper.py:  179]: worker_id: 0 list keys: [0, 100, 101, 102, 103]
[INFO: data_input_helper.py:  177]: worker_id: 1 list: 6
[INFO: data_input_helper.py:  179]: worker_id: 1 list keys: [0, 1, 100, 101, 102, 103]
[INFO: data_input_helper.py:  177]: worker_id: 2 list: 7
[INFO: data_input_helper.py:  179]: worker_id: 2 list keys: [0, 1, 2, 100, 101, 102, 103]
[INFO: data_input_helper.py:  177]: worker_id: 3 list: 8
[INFO: data_input_helper.py:  179]: worker_id: 3 list keys: [0, 1, 2, 3, 100, 101, 102, 103]
[INFO: data_input_helper.py:  224]: CREATED POOL: {0: <multiprocessing.pool.Pool object at 0x2b4c6392b9d0>, 1: <multiprocessing.pool.Pool object at 0x2b4e799b0510>, 2: <multiprocessing.pool.Pool object at 0x2b4e799b9510>, 3: <multiprocessing.pool.Pool object at 0x2b4e799bfed0>, 100: <multiprocessing.pool.Pool object at 0x2b4c57b23550>, 101: <multiprocessing.pool.Pool object at 0x2b4c57b2e850>, 102: <multiprocessing.pool.Pool object at 0x2b4c6371a990>, 103: <multiprocessing.pool.Pool object at 0x2b4c63724750>}
[INFO: data_input_helper.py:  225]: CREATED LISTS: 8
[INFO: data_input_helper.py:  226]: POOL keys: [0, 1, 2, 3, 100, 101, 102, 103]
[INFO: data_input_helper.py:  227]: LIST keys: [0, 1, 2, 3, 100, 101, 102, 103]
[INFO: data_parallel_model.py:  179]: Parallelizing model for devices: [0, 1, 2, 3]
[INFO: data_parallel_model.py:  202]: Create input and model training operators
[INFO: data_parallel_model.py:  235]: Model for GPU : 0
[INFO: resnet_video.py:  150]: --------------- ResNet-50 1x64d-bottleneck_transformation_3d, charades, train, infer LFB? False, suffix: "_train" ---------------
[INFO: resnet_video.py:  166]: use_temp_convs_set: [[2], [1, 1, 1], [1, 0, 1, 0], [1, 0, 1, 0, 1, 0], [0, 1, 0]]
[INFO: resnet_video.py:  167]: temp_strides_set: [[1], [1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1, 1, 1], [1, 1, 1]]
[INFO: resnet_helper.py:   62]: res2_0 using dilation 1
[INFO: resnet_helper.py:   62]: res2_1 using dilation 1
[INFO: resnet_helper.py:   62]: res2_2 using dilation 1
[INFO: resnet_helper.py:   62]: res3_0 using dilation 1
[INFO: resnet_helper.py:   62]: res3_1 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res3_2 using dilation 1
[INFO: resnet_helper.py:   62]: res3_3 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_0 using dilation 1
[INFO: resnet_helper.py:   62]: res4_1 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_2 using dilation 1
[INFO: resnet_helper.py:   62]: res4_3 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_4 using dilation 1
[INFO: resnet_helper.py:   62]: res4_5 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res5_0 using dilation 2
[INFO: resnet_helper.py:   62]: res5_1 using dilation 2
[INFO: resnet_helper.py:   62]: res5_2 using dilation 2
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: SigmoidCrossEntropyLoss.
[INFO: data_parallel_model.py:  235]: Model for GPU : 1
[INFO: resnet_video.py:  150]: --------------- ResNet-50 1x64d-bottleneck_transformation_3d, charades, train, infer LFB? False, suffix: "_train" ---------------
[INFO: resnet_video.py:  166]: use_temp_convs_set: [[2], [1, 1, 1], [1, 0, 1, 0], [1, 0, 1, 0, 1, 0], [0, 1, 0]]
[INFO: resnet_video.py:  167]: temp_strides_set: [[1], [1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1, 1, 1], [1, 1, 1]]
[INFO: resnet_helper.py:   62]: res2_0 using dilation 1
[INFO: resnet_helper.py:   62]: res2_1 using dilation 1
[INFO: resnet_helper.py:   62]: res2_2 using dilation 1
[INFO: resnet_helper.py:   62]: res3_0 using dilation 1
[INFO: resnet_helper.py:   62]: res3_1 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res3_2 using dilation 1
[INFO: resnet_helper.py:   62]: res3_3 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_0 using dilation 1
[INFO: resnet_helper.py:   62]: res4_1 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_2 using dilation 1
[INFO: resnet_helper.py:   62]: res4_3 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_4 using dilation 1
[INFO: resnet_helper.py:   62]: res4_5 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res5_0 using dilation 2
[INFO: resnet_helper.py:   62]: res5_1 using dilation 2
[INFO: resnet_helper.py:   62]: res5_2 using dilation 2
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: SigmoidCrossEntropyLoss.
[INFO: data_parallel_model.py:  235]: Model for GPU : 2
[INFO: resnet_video.py:  150]: --------------- ResNet-50 1x64d-bottleneck_transformation_3d, charades, train, infer LFB? False, suffix: "_train" ---------------
[INFO: resnet_video.py:  166]: use_temp_convs_set: [[2], [1, 1, 1], [1, 0, 1, 0], [1, 0, 1, 0, 1, 0], [0, 1, 0]]
[INFO: resnet_video.py:  167]: temp_strides_set: [[1], [1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1, 1, 1], [1, 1, 1]]
[INFO: resnet_helper.py:   62]: res2_0 using dilation 1
[INFO: resnet_helper.py:   62]: res2_1 using dilation 1
[INFO: resnet_helper.py:   62]: res2_2 using dilation 1
[INFO: resnet_helper.py:   62]: res3_0 using dilation 1
[INFO: resnet_helper.py:   62]: res3_1 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res3_2 using dilation 1
[INFO: resnet_helper.py:   62]: res3_3 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_0 using dilation 1
[INFO: resnet_helper.py:   62]: res4_1 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_2 using dilation 1
[INFO: resnet_helper.py:   62]: res4_3 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_4 using dilation 1
[INFO: resnet_helper.py:   62]: res4_5 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res5_0 using dilation 2
[INFO: resnet_helper.py:   62]: res5_1 using dilation 2
[INFO: resnet_helper.py:   62]: res5_2 using dilation 2
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: SigmoidCrossEntropyLoss.
[INFO: data_parallel_model.py:  235]: Model for GPU : 3
[INFO: resnet_video.py:  150]: --------------- ResNet-50 1x64d-bottleneck_transformation_3d, charades, train, infer LFB? False, suffix: "_train" ---------------
[INFO: resnet_video.py:  166]: use_temp_convs_set: [[2], [1, 1, 1], [1, 0, 1, 0], [1, 0, 1, 0, 1, 0], [0, 1, 0]]
[INFO: resnet_video.py:  167]: temp_strides_set: [[1], [1, 1, 1], [1, 1, 1, 1], [1, 1, 1, 1, 1, 1], [1, 1, 1]]
[INFO: resnet_helper.py:   62]: res2_0 using dilation 1
[INFO: resnet_helper.py:   62]: res2_1 using dilation 1
[INFO: resnet_helper.py:   62]: res2_2 using dilation 1
[INFO: resnet_helper.py:   62]: res3_0 using dilation 1
[INFO: resnet_helper.py:   62]: res3_1 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res3_2 using dilation 1
[INFO: resnet_helper.py:   62]: res3_3 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_0 using dilation 1
[INFO: resnet_helper.py:   62]: res4_1 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_2 using dilation 1
[INFO: resnet_helper.py:   62]: res4_3 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res4_4 using dilation 1
[INFO: resnet_helper.py:   62]: res4_5 using dilation 1
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: Reshape.
[INFO: resnet_helper.py:   62]: res5_0 using dilation 2
[INFO: resnet_helper.py:   62]: res5_1 using dilation 2
[INFO: resnet_helper.py:   62]: res5_2 using dilation 2
[WARNING: model_helper.py:  442]: You are creating an op that the ModelHelper does not recognize: SigmoidCrossEntropyLoss.
[INFO: data_parallel_model.py:  269]: Adding gradient operators
[INFO: data_parallel_model.py:  313]: Add gradient all-reduces for SyncSGD
[INFO: data_parallel_model.py:  333]: Post-iteration operators for updating params
[INFO: data_parallel_model.py:  372]: Add initial parameter sync
[WARNING: data_parallel_model.py: 1737]: ------- DEPRECATED API, please use data_parallel_model.OptimizeGradientMemory() -----
[WARNING: memonger.py:   55]: NOTE: Executing memonger to optimize gradient memory
[I memonger.cc:236] Remapping 151 using 22 shared blobs.
[INFO: memonger.py:   97]: Memonger memory optimization took 0.0479099750519 secs
[WARNING: memonger.py:   55]: NOTE: Executing memonger to optimize gradient memory
[I memonger.cc:236] Remapping 151 using 22 shared blobs.
[INFO: memonger.py:   97]: Memonger memory optimization took 0.0419881343842 secs
[WARNING: memonger.py:   55]: NOTE: Executing memonger to optimize gradient memory
[I memonger.cc:236] Remapping 151 using 22 shared blobs.
[INFO: memonger.py:   97]: Memonger memory optimization took 0.0381062030792 secs
[WARNING: memonger.py:   55]: NOTE: Executing memonger to optimize gradient memory
[I memonger.cc:236] Remapping 151 using 22 shared blobs.
[INFO: memonger.py:   97]: Memonger memory optimization took 0.0379178524017 secs
[I net_dag_utils.cc:102] Operator graph pruning prior to chain compute took: 0.00156603 secs
[INFO: model_builder_video.py:  160]: Starting data loader...
[INFO: dataloader.py:  366]: Pre-filling train minibatch queue
[INFO: dataloader.py:  369]: train minibatch queue pre-filled.
[INFO: metrics.py:  136]: Resetting train metrics...
[INFO: misc.py:   94]: CNN_1: Net proto saved to: /groups/1/gac50586/migrated_from_SFA_GPFS/NISHIMURA/Charades/video-long-term-feature-banks/tmp/lfb-nl-step1-output/CNN_1.pbtxt
[INFO: misc.py:   94]: CNN_init_1: Net proto saved to: /groups/1/gac50586/migrated_from_SFA_GPFS/NISHIMURA/Charades/video-long-term-feature-banks/tmp/lfb-nl-step1-output/CNN_init_1.pbtxt
[INFO: checkpoints.py:  198]: Resume training: True
[INFO: checkpoints.py:  150]: Coverting model : pretrained_weights/r50_k400_pretrained.pkl
[INFO: checkpoints.py:  204]: Checkpoint model converted
[INFO: checkpoints.py:  208]: Initializing from pre-trained file...
[INFO: checkpoints.py:  413]: Initializing model params from file: /groups/1/gac50586/migrated_from_SFA_GPFS/NISHIMURA/Charades/video-long-term-feature-banks/tmp/lfb-nl-step1-output/checkpoints/converted_model.pkl
[INFO: checkpoints.py:  274]: Initializing model params from file: /groups/1/gac50586/migrated_from_SFA_GPFS/NISHIMURA/Charades/video-long-term-feature-banks/tmp/lfb-nl-step1-output/checkpoints/converted_model.pkl
[INFO: checkpoints.py:  368]: conv1_w loaded from weights file into: gpu_0/conv1_w (64, 3, 5, 7, 7)
[INFO: checkpoints.py:  368]: res_conv1_bn_s loaded from weights file into: gpu_0/res_conv1_bn_s (64,)
[INFO: checkpoints.py:  368]: res_conv1_bn_b loaded from weights file into: gpu_0/res_conv1_bn_b (64,)
[INFO: checkpoints.py:  368]: res2_0_branch2a_w loaded from weights file into: gpu_0/res2_0_branch2a_w (64, 64, 3, 1, 1)
[INFO: checkpoints.py:  368]: res2_0_branch2a_bn_s loaded from weights file into: gpu_0/res2_0_branch2a_bn_s (64,)
[INFO: checkpoints.py:  368]: res2_0_branch2a_bn_b loaded from weights file into: gpu_0/res2_0_branch2a_bn_b (64,)
[INFO: checkpoints.py:  368]: res2_0_branch2b_w loaded from weights file into: gpu_0/res2_0_branch2b_w (64, 64, 1, 3, 3)
[INFO: checkpoints.py:  368]: res2_0_branch2b_bn_s loaded from weights file into: gpu_0/res2_0_branch2b_bn_s (64,)
[INFO: checkpoints.py:  368]: res2_0_branch2b_bn_b loaded from weights file into: gpu_0/res2_0_branch2b_bn_b (64,)
[INFO: checkpoints.py:  368]: res2_0_branch2c_w loaded from weights file into: gpu_0/res2_0_branch2c_w (256, 64, 1, 1, 1)
[INFO: checkpoints.py:  368]: res2_0_branch2c_bn_s loaded from weights file into: gpu_0/res2_0_branch2c_bn_s (256,)
[INFO: checkpoints.py:  368]: res2_0_branch2c_bn_b loaded from weights file into: gpu_0/res2_0_branch2c_bn_b (256,)
[INFO: checkpoints.py:  368]: res2_0_branch1_w loaded from weights file into: gpu_0/res2_0_branch1_w (256, 64, 1, 1, 1)
[INFO: checkpoints.py:  368]: res2_0_branch1_bn_s loaded from weights file into: gpu_0/res2_0_branch1_bn_s (256,)
[INFO: checkpoints.py:  368]: res2_0_branch1_bn_b loaded from weights file into: gpu_0/res2_0_branch1_bn_b (256,)
[INFO: checkpoints.py:  368]: res2_1_branch2a_w loaded from weights file into: gpu_0/res2_1_branch2a_w (64, 256, 3, 1, 1)
[INFO: checkpoints.py:  368]: res2_1_branch2a_bn_s loaded from weights file into: gpu_0/res2_1_branch2a_bn_s (64,)
[INFO: checkpoints.py:  368]: res2_1_branch2a_bn_b loaded from weights file into: gpu_0/res2_1_branch2a_bn_b (64,)
[INFO: checkpoints.py:  368]: res2_1_branch2b_w loaded from weights file into: gpu_0/res2_1_branch2b_w (64, 64, 1, 3, 3)
[INFO: checkpoints.py:  368]: res2_1_branch2b_bn_s loaded from weights file into: gpu_0/res2_1_branch2b_bn_s (64,)
[INFO: checkpoints.py:  368]: res2_1_branch2b_bn_b loaded from weights file into: gpu_0/res2_1_branch2b_bn_b (64,)
[INFO: checkpoints.py:  368]: res2_1_branch2c_w loaded from weights file into: gpu_0/res2_1_branch2c_w (256, 64, 1, 1, 1)
[INFO: checkpoints.py:  368]: res2_1_branch2c_bn_s loaded from weights file into: gpu_0/res2_1_branch2c_bn_s (256,)
[INFO: checkpoints.py:  368]: res2_1_branch2c_bn_b loaded from weights file into: gpu_0/res2_1_branch2c_bn_b (256,)
[INFO: checkpoints.py:  368]: res2_2_branch2a_w loaded from weights file into: gpu_0/res2_2_branch2a_w (64, 256, 3, 1, 1)
[INFO: checkpoints.py:  368]: res2_2_branch2a_bn_s loaded from weights file into: gpu_0/res2_2_branch2a_bn_s (64,)
[INFO: checkpoints.py:  368]: res2_2_branch2a_bn_b loaded from weights file into: gpu_0/res2_2_branch2a_bn_b (64,)
[INFO: checkpoints.py:  368]: res2_2_branch2b_w loaded from weights file into: gpu_0/res2_2_branch2b_w (64, 64, 1, 3, 3)
[INFO: checkpoints.py:  368]: res2_2_branch2b_bn_s loaded from weights file into: gpu_0/res2_2_branch2b_bn_s (64,)
[INFO: checkpoints.py:  368]: res2_2_branch2b_bn_b loaded from weights file into: gpu_0/res2_2_branch2b_bn_b (64,)
[INFO: checkpoints.py:  368]: res2_2_branch2c_w loaded from weights file into: gpu_0/res2_2_branch2c_w (256, 64, 1, 1, 1)
[INFO: checkpoints.py:  368]: res2_2_branch2c_bn_s loaded from weights file into: gpu_0/res2_2_branch2c_bn_s (256,)
[INFO: checkpoints.py:  368]: res2_2_branch2c_bn_b loaded from weights file into: gpu_0/res2_2_branch2c_bn_b (256,)
[INFO: checkpoints.py:  368]: res3_0_branch2a_w loaded from weights file into: gpu_0/res3_0_branch2a_w (128, 256, 3, 1, 1)
[INFO: checkpoints.py:  368]: res3_0_branch2a_bn_s loaded from weights file into: gpu_0/res3_0_branch2a_bn_s (128,)
[INFO: checkpoints.py:  368]: res3_0_branch2a_bn_b loaded from weights file into: gpu_0/res3_0_branch2a_bn_b (128,)
[INFO: checkpoints.py:  368]: res3_0_branch2b_w loaded from weights file into: gpu_0/res3_0_branch2b_w (128, 128, 1, 3, 3)
[INFO: checkpoints.py:  368]: res3_0_branch2b_bn_s loaded from weights file into: gpu_0/res3_0_branch2b_bn_s (128,)
[INFO: checkpoints.py:  368]: res3_0_branch2b_bn_b loaded from weights file into: gpu_0/res3_0_branch2b_bn_b (128,)
[INFO: checkpoints.py:  368]: res3_0_branch2c_w loaded from weights file into: gpu_0/res3_0_branch2c_w (512, 128, 1, 1, 1)
[INFO: checkpoints.py:  368]: res3_0_branch2c_bn_s loaded from weights file into: gpu_0/res3_0_branch2c_bn_s (512,)
[INFO: checkpoints.py:  368]: res3_0_branch2c_bn_b loaded from weights file into: gpu_0/res3_0_branch2c_bn_b (512,)
[INFO: checkpoints.py:  368]: res3_0_branch1_w loaded from weights file into: gpu_0/res3_0_branch1_w (512, 256, 1, 1, 1)
[INFO: checkpoints.py:  368]: res3_0_branch1_bn_s loaded from weights file into: gpu_0/res3_0_branch1_bn_s (512,)
[INFO: checkpoints.py:  368]: res3_0_branch1_bn_b loaded from weights file into: gpu_0/res3_0_branch1_bn_b (512,)
[INFO: checkpoints.py:  368]: res3_1_branch2a_w loaded from weights file into: gpu_0/res3_1_branch2a_w (128, 512, 1, 1, 1)
[INFO: checkpoints.py:  368]: res3_1_branch2a_bn_s loaded from weights file into: gpu_0/res3_1_branch2a_bn_s (128,)
[INFO: checkpoints.py:  368]: res3_1_branch2a_bn_b loaded from weights file into: gpu_0/res3_1_branch2a_bn_b (128,)
[INFO: checkpoints.py:  368]: res3_1_branch2b_w loaded from weights file into: gpu_0/res3_1_branch2b_w (128, 128, 1, 3, 3)
[INFO: checkpoints.py:  368]: res3_1_branch2b_bn_s loaded from weights file into: gpu_0/res3_1_branch2b_bn_s (128,)
[INFO: checkpoints.py:  368]: res3_1_branch2b_bn_b loaded from weights file into: gpu_0/res3_1_branch2b_bn_b (128,)
[INFO: checkpoints.py:  368]: res3_1_branch2c_w loaded from weights file into: gpu_0/res3_1_branch2c_w (512, 128, 1, 1, 1)
[INFO: checkpoints.py:  368]: res3_1_branch2c_bn_s loaded from weights file into: gpu_0/res3_1_branch2c_bn_s (512,)
[INFO: checkpoints.py:  368]: res3_1_branch2c_bn_b loaded from weights file into: gpu_0/res3_1_branch2c_bn_b (512,)
[INFO: checkpoints.py:  368]: nonlocal_conv3_1_theta_w loaded from weights file into: gpu_0/nonlocal_conv3_1_theta_w (256, 512, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv3_1_theta_b loaded from weights file into: gpu_0/nonlocal_conv3_1_theta_b (256,)
[INFO: checkpoints.py:  368]: nonlocal_conv3_1_phi_w loaded from weights file into: gpu_0/nonlocal_conv3_1_phi_w (256, 512, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv3_1_phi_b loaded from weights file into: gpu_0/nonlocal_conv3_1_phi_b (256,)
[INFO: checkpoints.py:  368]: nonlocal_conv3_1_g_w loaded from weights file into: gpu_0/nonlocal_conv3_1_g_w (256, 512, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv3_1_g_b loaded from weights file into: gpu_0/nonlocal_conv3_1_g_b (256,)
[INFO: checkpoints.py:  368]: nonlocal_conv3_1_out_w loaded from weights file into: gpu_0/nonlocal_conv3_1_out_w (512, 256, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv3_1_out_b loaded from weights file into: gpu_0/nonlocal_conv3_1_out_b (512,)
[INFO: checkpoints.py:  368]: nonlocal_conv3_1_bn_s loaded from weights file into: gpu_0/nonlocal_conv3_1_bn_s (512,)
[INFO: checkpoints.py:  368]: nonlocal_conv3_1_bn_b loaded from weights file into: gpu_0/nonlocal_conv3_1_bn_b (512,)
[INFO: checkpoints.py:  368]: res3_2_branch2a_w loaded from weights file into: gpu_0/res3_2_branch2a_w (128, 512, 3, 1, 1)
[INFO: checkpoints.py:  368]: res3_2_branch2a_bn_s loaded from weights file into: gpu_0/res3_2_branch2a_bn_s (128,)
[INFO: checkpoints.py:  368]: res3_2_branch2a_bn_b loaded from weights file into: gpu_0/res3_2_branch2a_bn_b (128,)
[INFO: checkpoints.py:  368]: res3_2_branch2b_w loaded from weights file into: gpu_0/res3_2_branch2b_w (128, 128, 1, 3, 3)
[INFO: checkpoints.py:  368]: res3_2_branch2b_bn_s loaded from weights file into: gpu_0/res3_2_branch2b_bn_s (128,)
[INFO: checkpoints.py:  368]: res3_2_branch2b_bn_b loaded from weights file into: gpu_0/res3_2_branch2b_bn_b (128,)
[INFO: checkpoints.py:  368]: res3_2_branch2c_w loaded from weights file into: gpu_0/res3_2_branch2c_w (512, 128, 1, 1, 1)
[INFO: checkpoints.py:  368]: res3_2_branch2c_bn_s loaded from weights file into: gpu_0/res3_2_branch2c_bn_s (512,)
[INFO: checkpoints.py:  368]: res3_2_branch2c_bn_b loaded from weights file into: gpu_0/res3_2_branch2c_bn_b (512,)
[INFO: checkpoints.py:  368]: res3_3_branch2a_w loaded from weights file into: gpu_0/res3_3_branch2a_w (128, 512, 1, 1, 1)
[INFO: checkpoints.py:  368]: res3_3_branch2a_bn_s loaded from weights file into: gpu_0/res3_3_branch2a_bn_s (128,)
[INFO: checkpoints.py:  368]: res3_3_branch2a_bn_b loaded from weights file into: gpu_0/res3_3_branch2a_bn_b (128,)
[INFO: checkpoints.py:  368]: res3_3_branch2b_w loaded from weights file into: gpu_0/res3_3_branch2b_w (128, 128, 1, 3, 3)
[INFO: checkpoints.py:  368]: res3_3_branch2b_bn_s loaded from weights file into: gpu_0/res3_3_branch2b_bn_s (128,)
[INFO: checkpoints.py:  368]: res3_3_branch2b_bn_b loaded from weights file into: gpu_0/res3_3_branch2b_bn_b (128,)
[INFO: checkpoints.py:  368]: res3_3_branch2c_w loaded from weights file into: gpu_0/res3_3_branch2c_w (512, 128, 1, 1, 1)
[INFO: checkpoints.py:  368]: res3_3_branch2c_bn_s loaded from weights file into: gpu_0/res3_3_branch2c_bn_s (512,)
[INFO: checkpoints.py:  368]: res3_3_branch2c_bn_b loaded from weights file into: gpu_0/res3_3_branch2c_bn_b (512,)
[INFO: checkpoints.py:  368]: nonlocal_conv3_3_theta_w loaded from weights file into: gpu_0/nonlocal_conv3_3_theta_w (256, 512, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv3_3_theta_b loaded from weights file into: gpu_0/nonlocal_conv3_3_theta_b (256,)
[INFO: checkpoints.py:  368]: nonlocal_conv3_3_phi_w loaded from weights file into: gpu_0/nonlocal_conv3_3_phi_w (256, 512, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv3_3_phi_b loaded from weights file into: gpu_0/nonlocal_conv3_3_phi_b (256,)
[INFO: checkpoints.py:  368]: nonlocal_conv3_3_g_w loaded from weights file into: gpu_0/nonlocal_conv3_3_g_w (256, 512, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv3_3_g_b loaded from weights file into: gpu_0/nonlocal_conv3_3_g_b (256,)
[INFO: checkpoints.py:  368]: nonlocal_conv3_3_out_w loaded from weights file into: gpu_0/nonlocal_conv3_3_out_w (512, 256, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv3_3_out_b loaded from weights file into: gpu_0/nonlocal_conv3_3_out_b (512,)
[INFO: checkpoints.py:  368]: nonlocal_conv3_3_bn_s loaded from weights file into: gpu_0/nonlocal_conv3_3_bn_s (512,)
[INFO: checkpoints.py:  368]: nonlocal_conv3_3_bn_b loaded from weights file into: gpu_0/nonlocal_conv3_3_bn_b (512,)
[INFO: checkpoints.py:  368]: res4_0_branch2a_w loaded from weights file into: gpu_0/res4_0_branch2a_w (256, 512, 3, 1, 1)
[INFO: checkpoints.py:  368]: res4_0_branch2a_bn_s loaded from weights file into: gpu_0/res4_0_branch2a_bn_s (256,)
[INFO: checkpoints.py:  368]: res4_0_branch2a_bn_b loaded from weights file into: gpu_0/res4_0_branch2a_bn_b (256,)
[INFO: checkpoints.py:  368]: res4_0_branch2b_w loaded from weights file into: gpu_0/res4_0_branch2b_w (256, 256, 1, 3, 3)
[INFO: checkpoints.py:  368]: res4_0_branch2b_bn_s loaded from weights file into: gpu_0/res4_0_branch2b_bn_s (256,)
[INFO: checkpoints.py:  368]: res4_0_branch2b_bn_b loaded from weights file into: gpu_0/res4_0_branch2b_bn_b (256,)
[INFO: checkpoints.py:  368]: res4_0_branch2c_w loaded from weights file into: gpu_0/res4_0_branch2c_w (1024, 256, 1, 1, 1)
[INFO: checkpoints.py:  368]: res4_0_branch2c_bn_s loaded from weights file into: gpu_0/res4_0_branch2c_bn_s (1024,)
[INFO: checkpoints.py:  368]: res4_0_branch2c_bn_b loaded from weights file into: gpu_0/res4_0_branch2c_bn_b (1024,)
[INFO: checkpoints.py:  368]: res4_0_branch1_w loaded from weights file into: gpu_0/res4_0_branch1_w (1024, 512, 1, 1, 1)
[INFO: checkpoints.py:  368]: res4_0_branch1_bn_s loaded from weights file into: gpu_0/res4_0_branch1_bn_s (1024,)
[INFO: checkpoints.py:  368]: res4_0_branch1_bn_b loaded from weights file into: gpu_0/res4_0_branch1_bn_b (1024,)
[INFO: checkpoints.py:  368]: res4_1_branch2a_w loaded from weights file into: gpu_0/res4_1_branch2a_w (256, 1024, 1, 1, 1)
[INFO: checkpoints.py:  368]: res4_1_branch2a_bn_s loaded from weights file into: gpu_0/res4_1_branch2a_bn_s (256,)
[INFO: checkpoints.py:  368]: res4_1_branch2a_bn_b loaded from weights file into: gpu_0/res4_1_branch2a_bn_b (256,)
[INFO: checkpoints.py:  368]: res4_1_branch2b_w loaded from weights file into: gpu_0/res4_1_branch2b_w (256, 256, 1, 3, 3)
[INFO: checkpoints.py:  368]: res4_1_branch2b_bn_s loaded from weights file into: gpu_0/res4_1_branch2b_bn_s (256,)
[INFO: checkpoints.py:  368]: res4_1_branch2b_bn_b loaded from weights file into: gpu_0/res4_1_branch2b_bn_b (256,)
[INFO: checkpoints.py:  368]: res4_1_branch2c_w loaded from weights file into: gpu_0/res4_1_branch2c_w (1024, 256, 1, 1, 1)
[INFO: checkpoints.py:  368]: res4_1_branch2c_bn_s loaded from weights file into: gpu_0/res4_1_branch2c_bn_s (1024,)
[INFO: checkpoints.py:  368]: res4_1_branch2c_bn_b loaded from weights file into: gpu_0/res4_1_branch2c_bn_b (1024,)
[INFO: checkpoints.py:  368]: nonlocal_conv4_1_theta_w loaded from weights file into: gpu_0/nonlocal_conv4_1_theta_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv4_1_theta_b loaded from weights file into: gpu_0/nonlocal_conv4_1_theta_b (512,)
[INFO: checkpoints.py:  368]: nonlocal_conv4_1_phi_w loaded from weights file into: gpu_0/nonlocal_conv4_1_phi_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv4_1_phi_b loaded from weights file into: gpu_0/nonlocal_conv4_1_phi_b (512,)
[INFO: checkpoints.py:  368]: nonlocal_conv4_1_g_w loaded from weights file into: gpu_0/nonlocal_conv4_1_g_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv4_1_g_b loaded from weights file into: gpu_0/nonlocal_conv4_1_g_b (512,)
[INFO: checkpoints.py:  368]: nonlocal_conv4_1_out_w loaded from weights file into: gpu_0/nonlocal_conv4_1_out_w (1024, 512, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv4_1_out_b loaded from weights file into: gpu_0/nonlocal_conv4_1_out_b (1024,)
[INFO: checkpoints.py:  368]: nonlocal_conv4_1_bn_s loaded from weights file into: gpu_0/nonlocal_conv4_1_bn_s (1024,)
[INFO: checkpoints.py:  368]: nonlocal_conv4_1_bn_b loaded from weights file into: gpu_0/nonlocal_conv4_1_bn_b (1024,)
[INFO: checkpoints.py:  368]: res4_2_branch2a_w loaded from weights file into: gpu_0/res4_2_branch2a_w (256, 1024, 3, 1, 1)
[INFO: checkpoints.py:  368]: res4_2_branch2a_bn_s loaded from weights file into: gpu_0/res4_2_branch2a_bn_s (256,)
[INFO: checkpoints.py:  368]: res4_2_branch2a_bn_b loaded from weights file into: gpu_0/res4_2_branch2a_bn_b (256,)
[INFO: checkpoints.py:  368]: res4_2_branch2b_w loaded from weights file into: gpu_0/res4_2_branch2b_w (256, 256, 1, 3, 3)
[INFO: checkpoints.py:  368]: res4_2_branch2b_bn_s loaded from weights file into: gpu_0/res4_2_branch2b_bn_s (256,)
[INFO: checkpoints.py:  368]: res4_2_branch2b_bn_b loaded from weights file into: gpu_0/res4_2_branch2b_bn_b (256,)
[INFO: checkpoints.py:  368]: res4_2_branch2c_w loaded from weights file into: gpu_0/res4_2_branch2c_w (1024, 256, 1, 1, 1)
[INFO: checkpoints.py:  368]: res4_2_branch2c_bn_s loaded from weights file into: gpu_0/res4_2_branch2c_bn_s (1024,)
[INFO: checkpoints.py:  368]: res4_2_branch2c_bn_b loaded from weights file into: gpu_0/res4_2_branch2c_bn_b (1024,)
[INFO: checkpoints.py:  368]: res4_3_branch2a_w loaded from weights file into: gpu_0/res4_3_branch2a_w (256, 1024, 1, 1, 1)
[INFO: checkpoints.py:  368]: res4_3_branch2a_bn_s loaded from weights file into: gpu_0/res4_3_branch2a_bn_s (256,)
[INFO: checkpoints.py:  368]: res4_3_branch2a_bn_b loaded from weights file into: gpu_0/res4_3_branch2a_bn_b (256,)
[INFO: checkpoints.py:  368]: res4_3_branch2b_w loaded from weights file into: gpu_0/res4_3_branch2b_w (256, 256, 1, 3, 3)
[INFO: checkpoints.py:  368]: res4_3_branch2b_bn_s loaded from weights file into: gpu_0/res4_3_branch2b_bn_s (256,)
[INFO: checkpoints.py:  368]: res4_3_branch2b_bn_b loaded from weights file into: gpu_0/res4_3_branch2b_bn_b (256,)
[INFO: checkpoints.py:  368]: res4_3_branch2c_w loaded from weights file into: gpu_0/res4_3_branch2c_w (1024, 256, 1, 1, 1)
[INFO: checkpoints.py:  368]: res4_3_branch2c_bn_s loaded from weights file into: gpu_0/res4_3_branch2c_bn_s (1024,)
[INFO: checkpoints.py:  368]: res4_3_branch2c_bn_b loaded from weights file into: gpu_0/res4_3_branch2c_bn_b (1024,)
[INFO: checkpoints.py:  368]: nonlocal_conv4_3_theta_w loaded from weights file into: gpu_0/nonlocal_conv4_3_theta_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv4_3_theta_b loaded from weights file into: gpu_0/nonlocal_conv4_3_theta_b (512,)
[INFO: checkpoints.py:  368]: nonlocal_conv4_3_phi_w loaded from weights file into: gpu_0/nonlocal_conv4_3_phi_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv4_3_phi_b loaded from weights file into: gpu_0/nonlocal_conv4_3_phi_b (512,)
[INFO: checkpoints.py:  368]: nonlocal_conv4_3_g_w loaded from weights file into: gpu_0/nonlocal_conv4_3_g_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv4_3_g_b loaded from weights file into: gpu_0/nonlocal_conv4_3_g_b (512,)
[INFO: checkpoints.py:  368]: nonlocal_conv4_3_out_w loaded from weights file into: gpu_0/nonlocal_conv4_3_out_w (1024, 512, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv4_3_out_b loaded from weights file into: gpu_0/nonlocal_conv4_3_out_b (1024,)
[INFO: checkpoints.py:  368]: nonlocal_conv4_3_bn_s loaded from weights file into: gpu_0/nonlocal_conv4_3_bn_s (1024,)
[INFO: checkpoints.py:  368]: nonlocal_conv4_3_bn_b loaded from weights file into: gpu_0/nonlocal_conv4_3_bn_b (1024,)
[INFO: checkpoints.py:  368]: res4_4_branch2a_w loaded from weights file into: gpu_0/res4_4_branch2a_w (256, 1024, 3, 1, 1)
[INFO: checkpoints.py:  368]: res4_4_branch2a_bn_s loaded from weights file into: gpu_0/res4_4_branch2a_bn_s (256,)
[INFO: checkpoints.py:  368]: res4_4_branch2a_bn_b loaded from weights file into: gpu_0/res4_4_branch2a_bn_b (256,)
[INFO: checkpoints.py:  368]: res4_4_branch2b_w loaded from weights file into: gpu_0/res4_4_branch2b_w (256, 256, 1, 3, 3)
[INFO: checkpoints.py:  368]: res4_4_branch2b_bn_s loaded from weights file into: gpu_0/res4_4_branch2b_bn_s (256,)
[INFO: checkpoints.py:  368]: res4_4_branch2b_bn_b loaded from weights file into: gpu_0/res4_4_branch2b_bn_b (256,)
[INFO: checkpoints.py:  368]: res4_4_branch2c_w loaded from weights file into: gpu_0/res4_4_branch2c_w (1024, 256, 1, 1, 1)
[INFO: checkpoints.py:  368]: res4_4_branch2c_bn_s loaded from weights file into: gpu_0/res4_4_branch2c_bn_s (1024,)
[INFO: checkpoints.py:  368]: res4_4_branch2c_bn_b loaded from weights file into: gpu_0/res4_4_branch2c_bn_b (1024,)
[INFO: checkpoints.py:  368]: res4_5_branch2a_w loaded from weights file into: gpu_0/res4_5_branch2a_w (256, 1024, 1, 1, 1)
[INFO: checkpoints.py:  368]: res4_5_branch2a_bn_s loaded from weights file into: gpu_0/res4_5_branch2a_bn_s (256,)
[INFO: checkpoints.py:  368]: res4_5_branch2a_bn_b loaded from weights file into: gpu_0/res4_5_branch2a_bn_b (256,)
[INFO: checkpoints.py:  368]: res4_5_branch2b_w loaded from weights file into: gpu_0/res4_5_branch2b_w (256, 256, 1, 3, 3)
[INFO: checkpoints.py:  368]: res4_5_branch2b_bn_s loaded from weights file into: gpu_0/res4_5_branch2b_bn_s (256,)
[INFO: checkpoints.py:  368]: res4_5_branch2b_bn_b loaded from weights file into: gpu_0/res4_5_branch2b_bn_b (256,)
[INFO: checkpoints.py:  368]: res4_5_branch2c_w loaded from weights file into: gpu_0/res4_5_branch2c_w (1024, 256, 1, 1, 1)
[INFO: checkpoints.py:  368]: res4_5_branch2c_bn_s loaded from weights file into: gpu_0/res4_5_branch2c_bn_s (1024,)
[INFO: checkpoints.py:  368]: res4_5_branch2c_bn_b loaded from weights file into: gpu_0/res4_5_branch2c_bn_b (1024,)
[INFO: checkpoints.py:  368]: nonlocal_conv4_5_theta_w loaded from weights file into: gpu_0/nonlocal_conv4_5_theta_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv4_5_theta_b loaded from weights file into: gpu_0/nonlocal_conv4_5_theta_b (512,)
[INFO: checkpoints.py:  368]: nonlocal_conv4_5_phi_w loaded from weights file into: gpu_0/nonlocal_conv4_5_phi_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv4_5_phi_b loaded from weights file into: gpu_0/nonlocal_conv4_5_phi_b (512,)
[INFO: checkpoints.py:  368]: nonlocal_conv4_5_g_w loaded from weights file into: gpu_0/nonlocal_conv4_5_g_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv4_5_g_b loaded from weights file into: gpu_0/nonlocal_conv4_5_g_b (512,)
[INFO: checkpoints.py:  368]: nonlocal_conv4_5_out_w loaded from weights file into: gpu_0/nonlocal_conv4_5_out_w (1024, 512, 1, 1, 1)
[INFO: checkpoints.py:  368]: nonlocal_conv4_5_out_b loaded from weights file into: gpu_0/nonlocal_conv4_5_out_b (1024,)
[INFO: checkpoints.py:  368]: nonlocal_conv4_5_bn_s loaded from weights file into: gpu_0/nonlocal_conv4_5_bn_s (1024,)
[INFO: checkpoints.py:  368]: nonlocal_conv4_5_bn_b loaded from weights file into: gpu_0/nonlocal_conv4_5_bn_b (1024,)
[INFO: checkpoints.py:  368]: res5_0_branch2a_w loaded from weights file into: gpu_0/res5_0_branch2a_w (512, 1024, 1, 1, 1)
[INFO: checkpoints.py:  368]: res5_0_branch2a_bn_s loaded from weights file into: gpu_0/res5_0_branch2a_bn_s (512,)
[INFO: checkpoints.py:  368]: res5_0_branch2a_bn_b loaded from weights file into: gpu_0/res5_0_branch2a_bn_b (512,)
[INFO: checkpoints.py:  368]: res5_0_branch2b_w loaded from weights file into: gpu_0/res5_0_branch2b_w (512, 512, 1, 3, 3)
[INFO: checkpoints.py:  368]: res5_0_branch2b_bn_s loaded from weights file into: gpu_0/res5_0_branch2b_bn_s (512,)
[INFO: checkpoints.py:  368]: res5_0_branch2b_bn_b loaded from weights file into: gpu_0/res5_0_branch2b_bn_b (512,)
[INFO: checkpoints.py:  368]: res5_0_branch2c_w loaded from weights file into: gpu_0/res5_0_branch2c_w (2048, 512, 1, 1, 1)
[INFO: checkpoints.py:  368]: res5_0_branch2c_bn_s loaded from weights file into: gpu_0/res5_0_branch2c_bn_s (2048,)
[INFO: checkpoints.py:  368]: res5_0_branch2c_bn_b loaded from weights file into: gpu_0/res5_0_branch2c_bn_b (2048,)
[INFO: checkpoints.py:  368]: res5_0_branch1_w loaded from weights file into: gpu_0/res5_0_branch1_w (2048, 1024, 1, 1, 1)
[INFO: checkpoints.py:  368]: res5_0_branch1_bn_s loaded from weights file into: gpu_0/res5_0_branch1_bn_s (2048,)
[INFO: checkpoints.py:  368]: res5_0_branch1_bn_b loaded from weights file into: gpu_0/res5_0_branch1_bn_b (2048,)
[INFO: checkpoints.py:  368]: res5_1_branch2a_w loaded from weights file into: gpu_0/res5_1_branch2a_w (512, 2048, 3, 1, 1)
[INFO: checkpoints.py:  368]: res5_1_branch2a_bn_s loaded from weights file into: gpu_0/res5_1_branch2a_bn_s (512,)
[INFO: checkpoints.py:  368]: res5_1_branch2a_bn_b loaded from weights file into: gpu_0/res5_1_branch2a_bn_b (512,)
[INFO: checkpoints.py:  368]: res5_1_branch2b_w loaded from weights file into: gpu_0/res5_1_branch2b_w (512, 512, 1, 3, 3)
[INFO: checkpoints.py:  368]: res5_1_branch2b_bn_s loaded from weights file into: gpu_0/res5_1_branch2b_bn_s (512,)
[INFO: checkpoints.py:  368]: res5_1_branch2b_bn_b loaded from weights file into: gpu_0/res5_1_branch2b_bn_b (512,)
[INFO: checkpoints.py:  368]: res5_1_branch2c_w loaded from weights file into: gpu_0/res5_1_branch2c_w (2048, 512, 1, 1, 1)
[INFO: checkpoints.py:  368]: res5_1_branch2c_bn_s loaded from weights file into: gpu_0/res5_1_branch2c_bn_s (2048,)
[INFO: checkpoints.py:  368]: res5_1_branch2c_bn_b loaded from weights file into: gpu_0/res5_1_branch2c_bn_b (2048,)
[INFO: checkpoints.py:  368]: res5_2_branch2a_w loaded from weights file into: gpu_0/res5_2_branch2a_w (512, 2048, 1, 1, 1)
[INFO: checkpoints.py:  368]: res5_2_branch2a_bn_s loaded from weights file into: gpu_0/res5_2_branch2a_bn_s (512,)
[INFO: checkpoints.py:  368]: res5_2_branch2a_bn_b loaded from weights file into: gpu_0/res5_2_branch2a_bn_b (512,)
[INFO: checkpoints.py:  368]: res5_2_branch2b_w loaded from weights file into: gpu_0/res5_2_branch2b_w (512, 512, 1, 3, 3)
[INFO: checkpoints.py:  368]: res5_2_branch2b_bn_s loaded from weights file into: gpu_0/res5_2_branch2b_bn_s (512,)
[INFO: checkpoints.py:  368]: res5_2_branch2b_bn_b loaded from weights file into: gpu_0/res5_2_branch2b_bn_b (512,)
[INFO: checkpoints.py:  368]: res5_2_branch2c_w loaded from weights file into: gpu_0/res5_2_branch2c_w (2048, 512, 1, 1, 1)
[INFO: checkpoints.py:  368]: res5_2_branch2c_bn_s loaded from weights file into: gpu_0/res5_2_branch2c_bn_s (2048,)
[INFO: checkpoints.py:  368]: res5_2_branch2c_bn_b loaded from weights file into: gpu_0/res5_2_branch2c_bn_b (2048,)
[INFO: checkpoints.py:  316]: pred_w not found
[INFO: checkpoints.py:  316]: pred_b not found
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/conv1_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/conv1_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/conv1_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/conv1_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res_conv1_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res_conv1_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res_conv1_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res_conv1_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res_conv1_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res_conv1_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res_conv1_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res_conv1_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_0_branch2a_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_0_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_0_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_0_branch2a_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_0_branch2a_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_0_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_0_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_0_branch2a_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_0_branch2a_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_0_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_0_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_0_branch2a_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_0_branch2b_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_0_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_0_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_0_branch2b_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_0_branch2b_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_0_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_0_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_0_branch2b_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_0_branch2b_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_0_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_0_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_0_branch2b_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_0_branch2c_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_0_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_0_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_0_branch2c_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_0_branch2c_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_0_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_0_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_0_branch2c_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_0_branch2c_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_0_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_0_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_0_branch2c_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_0_branch1_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_0_branch1_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_0_branch1_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_0_branch1_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_0_branch1_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_0_branch1_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_0_branch1_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_0_branch1_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_0_branch1_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_0_branch1_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_0_branch1_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_0_branch1_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_1_branch2a_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_1_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_1_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_1_branch2a_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_1_branch2a_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_1_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_1_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_1_branch2a_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_1_branch2a_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_1_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_1_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_1_branch2a_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_1_branch2b_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_1_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_1_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_1_branch2b_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_1_branch2b_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_1_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_1_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_1_branch2b_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_1_branch2b_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_1_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_1_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_1_branch2b_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_1_branch2c_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_1_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_1_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_1_branch2c_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_1_branch2c_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_1_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_1_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_1_branch2c_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_1_branch2c_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_1_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_1_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_1_branch2c_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_2_branch2a_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_2_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_2_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_2_branch2a_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_2_branch2a_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_2_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_2_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_2_branch2a_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_2_branch2a_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_2_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_2_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_2_branch2a_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_2_branch2b_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_2_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_2_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_2_branch2b_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_2_branch2b_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_2_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_2_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_2_branch2b_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_2_branch2b_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_2_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_2_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_2_branch2b_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_2_branch2c_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_2_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_2_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_2_branch2c_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_2_branch2c_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_2_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_2_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_2_branch2c_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_2_branch2c_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_2_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_2_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_2_branch2c_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_0_branch2a_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_0_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_0_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_0_branch2a_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_0_branch2a_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_0_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_0_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_0_branch2a_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_0_branch2a_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_0_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_0_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_0_branch2a_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_0_branch2b_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_0_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_0_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_0_branch2b_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_0_branch2b_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_0_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_0_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_0_branch2b_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_0_branch2b_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_0_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_0_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_0_branch2b_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_0_branch2c_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_0_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_0_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_0_branch2c_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_0_branch2c_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_0_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_0_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_0_branch2c_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_0_branch2c_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_0_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_0_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_0_branch2c_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_0_branch1_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_0_branch1_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_0_branch1_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_0_branch1_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_0_branch1_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_0_branch1_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_0_branch1_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_0_branch1_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_0_branch1_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_0_branch1_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_0_branch1_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_0_branch1_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_1_branch2a_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_1_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_1_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_1_branch2a_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_1_branch2a_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_1_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_1_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_1_branch2a_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_1_branch2a_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_1_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_1_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_1_branch2a_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_1_branch2b_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_1_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_1_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_1_branch2b_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_1_branch2b_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_1_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_1_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_1_branch2b_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_1_branch2b_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_1_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_1_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_1_branch2b_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_1_branch2c_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_1_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_1_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_1_branch2c_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_1_branch2c_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_1_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_1_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_1_branch2c_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_1_branch2c_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_1_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_1_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_1_branch2c_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_1_theta_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_1_theta_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_1_theta_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_1_theta_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_1_theta_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_1_theta_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_1_theta_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_1_theta_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_1_phi_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_1_phi_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_1_phi_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_1_phi_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_1_phi_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_1_phi_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_1_phi_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_1_phi_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_1_g_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_1_g_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_1_g_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_1_g_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_1_g_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_1_g_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_1_g_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_1_g_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_1_out_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_1_out_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_1_out_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_1_out_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_1_out_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_1_out_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_1_out_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_1_out_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_1_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_1_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_1_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_1_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_1_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_1_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_1_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_1_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_2_branch2a_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_2_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_2_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_2_branch2a_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_2_branch2a_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_2_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_2_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_2_branch2a_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_2_branch2a_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_2_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_2_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_2_branch2a_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_2_branch2b_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_2_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_2_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_2_branch2b_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_2_branch2b_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_2_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_2_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_2_branch2b_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_2_branch2b_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_2_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_2_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_2_branch2b_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_2_branch2c_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_2_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_2_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_2_branch2c_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_2_branch2c_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_2_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_2_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_2_branch2c_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_2_branch2c_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_2_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_2_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_2_branch2c_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_3_branch2a_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_3_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_3_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_3_branch2a_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_3_branch2a_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_3_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_3_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_3_branch2a_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_3_branch2a_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_3_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_3_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_3_branch2a_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_3_branch2b_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_3_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_3_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_3_branch2b_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_3_branch2b_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_3_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_3_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_3_branch2b_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_3_branch2b_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_3_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_3_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_3_branch2b_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_3_branch2c_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_3_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_3_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_3_branch2c_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_3_branch2c_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_3_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_3_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_3_branch2c_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_3_branch2c_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_3_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_3_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_3_branch2c_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_3_theta_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_3_theta_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_3_theta_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_3_theta_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_3_theta_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_3_theta_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_3_theta_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_3_theta_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_3_phi_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_3_phi_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_3_phi_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_3_phi_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_3_phi_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_3_phi_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_3_phi_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_3_phi_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_3_g_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_3_g_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_3_g_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_3_g_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_3_g_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_3_g_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_3_g_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_3_g_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_3_out_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_3_out_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_3_out_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_3_out_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_3_out_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_3_out_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_3_out_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_3_out_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_3_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_3_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_3_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_3_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_3_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_3_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_3_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_3_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_0_branch2a_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_0_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_0_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_0_branch2a_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_0_branch2a_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_0_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_0_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_0_branch2a_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_0_branch2a_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_0_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_0_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_0_branch2a_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_0_branch2b_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_0_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_0_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_0_branch2b_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_0_branch2b_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_0_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_0_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_0_branch2b_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_0_branch2b_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_0_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_0_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_0_branch2b_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_0_branch2c_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_0_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_0_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_0_branch2c_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_0_branch2c_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_0_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_0_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_0_branch2c_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_0_branch2c_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_0_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_0_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_0_branch2c_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_0_branch1_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_0_branch1_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_0_branch1_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_0_branch1_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_0_branch1_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_0_branch1_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_0_branch1_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_0_branch1_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_0_branch1_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_0_branch1_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_0_branch1_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_0_branch1_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_1_branch2a_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_1_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_1_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_1_branch2a_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_1_branch2a_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_1_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_1_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_1_branch2a_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_1_branch2a_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_1_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_1_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_1_branch2a_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_1_branch2b_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_1_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_1_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_1_branch2b_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_1_branch2b_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_1_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_1_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_1_branch2b_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_1_branch2b_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_1_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_1_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_1_branch2b_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_1_branch2c_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_1_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_1_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_1_branch2c_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_1_branch2c_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_1_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_1_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_1_branch2c_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_1_branch2c_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_1_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_1_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_1_branch2c_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_1_theta_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_1_theta_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_1_theta_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_1_theta_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_1_theta_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_1_theta_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_1_theta_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_1_theta_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_1_phi_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_1_phi_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_1_phi_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_1_phi_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_1_phi_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_1_phi_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_1_phi_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_1_phi_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_1_g_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_1_g_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_1_g_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_1_g_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_1_g_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_1_g_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_1_g_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_1_g_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_1_out_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_1_out_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_1_out_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_1_out_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_1_out_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_1_out_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_1_out_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_1_out_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_1_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_1_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_1_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_1_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_1_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_1_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_1_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_1_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_2_branch2a_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_2_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_2_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_2_branch2a_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_2_branch2a_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_2_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_2_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_2_branch2a_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_2_branch2a_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_2_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_2_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_2_branch2a_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_2_branch2b_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_2_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_2_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_2_branch2b_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_2_branch2b_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_2_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_2_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_2_branch2b_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_2_branch2b_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_2_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_2_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_2_branch2b_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_2_branch2c_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_2_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_2_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_2_branch2c_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_2_branch2c_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_2_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_2_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_2_branch2c_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_2_branch2c_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_2_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_2_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_2_branch2c_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_3_branch2a_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_3_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_3_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_3_branch2a_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_3_branch2a_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_3_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_3_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_3_branch2a_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_3_branch2a_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_3_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_3_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_3_branch2a_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_3_branch2b_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_3_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_3_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_3_branch2b_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_3_branch2b_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_3_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_3_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_3_branch2b_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_3_branch2b_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_3_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_3_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_3_branch2b_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_3_branch2c_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_3_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_3_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_3_branch2c_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_3_branch2c_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_3_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_3_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_3_branch2c_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_3_branch2c_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_3_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_3_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_3_branch2c_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_3_theta_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_3_theta_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_3_theta_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_3_theta_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_3_theta_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_3_theta_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_3_theta_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_3_theta_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_3_phi_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_3_phi_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_3_phi_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_3_phi_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_3_phi_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_3_phi_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_3_phi_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_3_phi_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_3_g_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_3_g_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_3_g_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_3_g_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_3_g_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_3_g_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_3_g_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_3_g_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_3_out_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_3_out_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_3_out_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_3_out_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_3_out_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_3_out_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_3_out_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_3_out_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_3_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_3_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_3_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_3_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_3_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_3_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_3_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_3_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_4_branch2a_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_4_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_4_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_4_branch2a_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_4_branch2a_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_4_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_4_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_4_branch2a_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_4_branch2a_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_4_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_4_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_4_branch2a_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_4_branch2b_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_4_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_4_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_4_branch2b_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_4_branch2b_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_4_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_4_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_4_branch2b_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_4_branch2b_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_4_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_4_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_4_branch2b_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_4_branch2c_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_4_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_4_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_4_branch2c_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_4_branch2c_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_4_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_4_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_4_branch2c_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_4_branch2c_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_4_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_4_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_4_branch2c_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_5_branch2a_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_5_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_5_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_5_branch2a_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_5_branch2a_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_5_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_5_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_5_branch2a_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_5_branch2a_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_5_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_5_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_5_branch2a_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_5_branch2b_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_5_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_5_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_5_branch2b_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_5_branch2b_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_5_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_5_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_5_branch2b_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_5_branch2b_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_5_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_5_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_5_branch2b_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_5_branch2c_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_5_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_5_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_5_branch2c_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_5_branch2c_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_5_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_5_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_5_branch2c_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_5_branch2c_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_5_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_5_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_5_branch2c_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_5_theta_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_5_theta_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_5_theta_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_5_theta_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_5_theta_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_5_theta_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_5_theta_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_5_theta_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_5_phi_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_5_phi_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_5_phi_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_5_phi_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_5_phi_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_5_phi_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_5_phi_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_5_phi_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_5_g_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_5_g_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_5_g_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_5_g_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_5_g_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_5_g_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_5_g_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_5_g_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_5_out_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_5_out_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_5_out_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_5_out_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_5_out_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_5_out_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_5_out_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_5_out_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_5_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_5_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_5_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_5_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_5_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_5_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_5_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_5_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_0_branch2a_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_0_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_0_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_0_branch2a_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_0_branch2a_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_0_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_0_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_0_branch2a_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_0_branch2a_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_0_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_0_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_0_branch2a_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_0_branch2b_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_0_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_0_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_0_branch2b_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_0_branch2b_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_0_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_0_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_0_branch2b_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_0_branch2b_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_0_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_0_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_0_branch2b_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_0_branch2c_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_0_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_0_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_0_branch2c_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_0_branch2c_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_0_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_0_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_0_branch2c_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_0_branch2c_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_0_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_0_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_0_branch2c_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_0_branch1_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_0_branch1_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_0_branch1_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_0_branch1_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_0_branch1_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_0_branch1_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_0_branch1_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_0_branch1_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_0_branch1_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_0_branch1_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_0_branch1_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_0_branch1_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_1_branch2a_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_1_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_1_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_1_branch2a_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_1_branch2a_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_1_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_1_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_1_branch2a_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_1_branch2a_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_1_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_1_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_1_branch2a_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_1_branch2b_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_1_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_1_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_1_branch2b_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_1_branch2b_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_1_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_1_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_1_branch2b_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_1_branch2b_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_1_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_1_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_1_branch2b_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_1_branch2c_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_1_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_1_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_1_branch2c_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_1_branch2c_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_1_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_1_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_1_branch2c_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_1_branch2c_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_1_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_1_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_1_branch2c_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_2_branch2a_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_2_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_2_branch2a_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_2_branch2a_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_2_branch2a_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_2_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_2_branch2a_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_2_branch2a_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_2_branch2a_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_2_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_2_branch2a_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_2_branch2a_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_2_branch2b_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_2_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_2_branch2b_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_2_branch2b_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_2_branch2b_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_2_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_2_branch2b_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_2_branch2b_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_2_branch2b_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_2_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_2_branch2b_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_2_branch2b_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_2_branch2c_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_2_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_2_branch2c_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_2_branch2c_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_2_branch2c_bn_s to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_2_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_2_branch2c_bn_s
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_2_branch2c_bn_s
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_2_branch2c_bn_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_2_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_2_branch2c_bn_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_2_branch2c_bn_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/pred_w to
[INFO: checkpoints.py:  406]:  |-> gpu_1/pred_w
[INFO: checkpoints.py:  406]:  |-> gpu_2/pred_w
[INFO: checkpoints.py:  406]:  |-> gpu_3/pred_w
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/pred_b to
[INFO: checkpoints.py:  406]:  |-> gpu_1/pred_b
[INFO: checkpoints.py:  406]:  |-> gpu_2/pred_b
[INFO: checkpoints.py:  406]:  |-> gpu_3/pred_b
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/conv1_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/conv1_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/conv1_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/conv1_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_0_branch2a_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_0_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_0_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_0_branch2a_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_0_branch2b_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_0_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_0_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_0_branch2b_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_0_branch2c_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_0_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_0_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_0_branch2c_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_0_branch1_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_0_branch1_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_0_branch1_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_0_branch1_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_1_branch2a_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_1_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_1_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_1_branch2a_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_1_branch2b_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_1_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_1_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_1_branch2b_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_1_branch2c_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_1_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_1_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_1_branch2c_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_2_branch2a_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_2_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_2_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_2_branch2a_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_2_branch2b_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_2_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_2_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_2_branch2b_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res2_2_branch2c_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res2_2_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res2_2_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res2_2_branch2c_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_0_branch2a_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_0_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_0_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_0_branch2a_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_0_branch2b_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_0_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_0_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_0_branch2b_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_0_branch2c_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_0_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_0_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_0_branch2c_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_0_branch1_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_0_branch1_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_0_branch1_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_0_branch1_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_1_branch2a_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_1_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_1_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_1_branch2a_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_1_branch2b_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_1_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_1_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_1_branch2b_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_1_branch2c_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_1_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_1_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_1_branch2c_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_1_theta_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_1_theta_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_1_theta_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_1_theta_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_1_theta_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_1_theta_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_1_theta_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_1_theta_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_1_phi_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_1_phi_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_1_phi_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_1_phi_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_1_phi_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_1_phi_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_1_phi_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_1_phi_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_1_g_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_1_g_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_1_g_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_1_g_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_1_g_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_1_g_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_1_g_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_1_g_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_1_out_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_1_out_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_1_out_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_1_out_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_1_out_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_1_out_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_1_out_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_1_out_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_2_branch2a_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_2_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_2_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_2_branch2a_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_2_branch2b_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_2_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_2_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_2_branch2b_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_2_branch2c_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_2_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_2_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_2_branch2c_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_3_branch2a_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_3_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_3_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_3_branch2a_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_3_branch2b_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_3_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_3_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_3_branch2b_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res3_3_branch2c_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res3_3_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res3_3_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res3_3_branch2c_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_3_theta_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_3_theta_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_3_theta_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_3_theta_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_3_theta_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_3_theta_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_3_theta_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_3_theta_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_3_phi_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_3_phi_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_3_phi_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_3_phi_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_3_phi_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_3_phi_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_3_phi_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_3_phi_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_3_g_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_3_g_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_3_g_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_3_g_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_3_g_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_3_g_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_3_g_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_3_g_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_3_out_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_3_out_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_3_out_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_3_out_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv3_3_out_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv3_3_out_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv3_3_out_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv3_3_out_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_0_branch2a_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_0_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_0_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_0_branch2a_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_0_branch2b_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_0_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_0_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_0_branch2b_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_0_branch2c_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_0_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_0_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_0_branch2c_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_0_branch1_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_0_branch1_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_0_branch1_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_0_branch1_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_1_branch2a_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_1_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_1_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_1_branch2a_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_1_branch2b_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_1_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_1_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_1_branch2b_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_1_branch2c_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_1_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_1_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_1_branch2c_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_1_theta_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_1_theta_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_1_theta_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_1_theta_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_1_theta_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_1_theta_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_1_theta_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_1_theta_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_1_phi_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_1_phi_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_1_phi_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_1_phi_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_1_phi_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_1_phi_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_1_phi_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_1_phi_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_1_g_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_1_g_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_1_g_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_1_g_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_1_g_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_1_g_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_1_g_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_1_g_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_1_out_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_1_out_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_1_out_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_1_out_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_1_out_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_1_out_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_1_out_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_1_out_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_2_branch2a_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_2_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_2_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_2_branch2a_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_2_branch2b_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_2_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_2_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_2_branch2b_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_2_branch2c_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_2_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_2_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_2_branch2c_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_3_branch2a_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_3_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_3_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_3_branch2a_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_3_branch2b_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_3_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_3_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_3_branch2b_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_3_branch2c_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_3_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_3_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_3_branch2c_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_3_theta_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_3_theta_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_3_theta_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_3_theta_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_3_theta_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_3_theta_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_3_theta_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_3_theta_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_3_phi_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_3_phi_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_3_phi_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_3_phi_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_3_phi_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_3_phi_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_3_phi_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_3_phi_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_3_g_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_3_g_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_3_g_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_3_g_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_3_g_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_3_g_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_3_g_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_3_g_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_3_out_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_3_out_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_3_out_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_3_out_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_3_out_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_3_out_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_3_out_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_3_out_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_4_branch2a_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_4_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_4_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_4_branch2a_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_4_branch2b_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_4_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_4_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_4_branch2b_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_4_branch2c_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_4_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_4_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_4_branch2c_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_5_branch2a_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_5_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_5_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_5_branch2a_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_5_branch2b_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_5_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_5_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_5_branch2b_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res4_5_branch2c_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res4_5_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res4_5_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res4_5_branch2c_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_5_theta_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_5_theta_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_5_theta_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_5_theta_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_5_theta_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_5_theta_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_5_theta_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_5_theta_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_5_phi_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_5_phi_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_5_phi_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_5_phi_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_5_phi_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_5_phi_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_5_phi_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_5_phi_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_5_g_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_5_g_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_5_g_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_5_g_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_5_g_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_5_g_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_5_g_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_5_g_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_5_out_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_5_out_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_5_out_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_5_out_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/nonlocal_conv4_5_out_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/nonlocal_conv4_5_out_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/nonlocal_conv4_5_out_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/nonlocal_conv4_5_out_b_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_0_branch2a_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_0_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_0_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_0_branch2a_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_0_branch2b_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_0_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_0_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_0_branch2b_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_0_branch2c_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_0_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_0_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_0_branch2c_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_0_branch1_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_0_branch1_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_0_branch1_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_0_branch1_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_1_branch2a_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_1_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_1_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_1_branch2a_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_1_branch2b_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_1_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_1_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_1_branch2b_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_1_branch2c_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_1_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_1_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_1_branch2c_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_2_branch2a_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_2_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_2_branch2a_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_2_branch2a_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_2_branch2b_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_2_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_2_branch2b_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_2_branch2b_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/res5_2_branch2c_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/res5_2_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/res5_2_branch2c_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/res5_2_branch2c_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/pred_w_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/pred_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/pred_w_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/pred_w_momentum
[INFO: checkpoints.py:  401]: Broadcasting gpu_0/pred_b_momentum to
[INFO: checkpoints.py:  406]:  |-> gpu_1/pred_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_2/pred_b_momentum
[INFO: checkpoints.py:  406]:  |-> gpu_3/pred_b_momentum
[INFO: checkpoints.py:  214]: Loaded: start_model_iter: 0; prev_lr: 0.00125000
[INFO: train_net.py:  143]: ------------- Training model... -------------
[INFO: metrics.py:  136]: Resetting train metrics...
[INFO: model_builder_video.py:  268]: Setting learning rate to 0.020000 at iteration 0
[INFO: model_builder_video.py:  300]: Scaling update history by 16.000000 (new/old lr)
[I net_async_base.h:207] Using specified CPU pool size: 16; device id: -1
[I net_async_base.h:212] Created new CPU pool, size: 16; device id: -1
[I cuda_nccl_gpu.cc:91] Creating NCCLContext for key: 0:0,1,2,3,
[INFO: misc.py:  114]: Printing Model: CNN_1
[INFO: misc.py:  142]: data_train                  : (1, 3, 32, 224, 224) => data_train                          : (1, 3, 32, 224, 224) ------- (op: StopGradient)
[INFO: misc.py:  142]: data_train                  : (1, 3, 32, 224, 224) => conv1                               : (1, 64, 32, 112, 112) ------- (op: Conv)
[INFO: misc.py:  142]: conv1                       : (1, 64, 32, 112, 112) => res_conv1_bn                        : (1, 64, 32, 112, 112) ------- (op: AffineNd)
[INFO: misc.py:  142]: res_conv1_bn                : (1, 64, 32, 112, 112) => res_conv1_bn                        : (1, 64, 32, 112, 112) ------- (op: Relu)
[INFO: misc.py:  142]: res_conv1_bn                : (1, 64, 32, 112, 112) => pool1                               : (1, 64, 32, 56, 56)  ------- (op: MaxPool)
[INFO: misc.py:  142]: pool1                       : (1, 64, 32, 56, 56)  => res2_0_branch2a                     : (1, 64, 32, 56, 56)  ------- (op: Conv)
[INFO: misc.py:  142]: res2_0_branch2a             : (1, 64, 32, 56, 56)  => res2_0_branch2a_bn                  : (1, 64, 32, 56, 56)  ------- (op: AffineNd)
[INFO: misc.py:  142]: res2_0_branch2a_bn          : (1, 64, 32, 56, 56)  => res2_0_branch2a_bn                  : (1, 64, 32, 56, 56)  ------- (op: Relu)
[INFO: misc.py:  142]: res2_0_branch2a_bn          : (1, 64, 32, 56, 56)  => res2_0_branch2b                     : (1, 64, 32, 56, 56)  ------- (op: Conv)
[INFO: misc.py:  142]: res2_0_branch2b             : (1, 64, 32, 56, 56)  => res2_0_branch2b_bn                  : (1, 64, 32, 56, 56)  ------- (op: AffineNd)
[INFO: misc.py:  142]: res2_0_branch2b_bn          : (1, 64, 32, 56, 56)  => res2_0_branch2b_bn                  : (1, 64, 32, 56, 56)  ------- (op: Relu)
[INFO: misc.py:  142]: res2_0_branch2b_bn          : (1, 64, 32, 56, 56)  => res2_0_branch2c                     : (1, 256, 32, 56, 56) ------- (op: Conv)
[INFO: misc.py:  142]: res2_0_branch2c             : (1, 256, 32, 56, 56) => res2_0_branch2c_bn                  : (1, 256, 32, 56, 56) ------- (op: AffineNd)
[INFO: misc.py:  142]: pool1                       : (1, 64, 32, 56, 56)  => res2_0_branch1                      : (1, 256, 32, 56, 56) ------- (op: Conv)
[INFO: misc.py:  142]: res2_0_branch1              : (1, 256, 32, 56, 56) => res2_0_branch1_bn                   : (1, 256, 32, 56, 56) ------- (op: AffineNd)
[INFO: misc.py:  142]: res2_0_branch2c_bn          : (1, 256, 32, 56, 56) => res2_0_branch2c_bn                  : (1, 256, 32, 56, 56) ------- (op: Sum)
[INFO: misc.py:  142]: res2_0_branch1_bn           : (1, 256, 32, 56, 56) => res2_0_branch2c_bn                  : (1, 256, 32, 56, 56) ------|
[INFO: misc.py:  142]: res2_0_branch2c_bn          : (1, 256, 32, 56, 56) => res2_0_branch2c_bn                  : (1, 256, 32, 56, 56) ------- (op: Relu)
[INFO: misc.py:  142]: res2_0_branch2c_bn          : (1, 256, 32, 56, 56) => res2_1_branch2a                     : (1, 64, 32, 56, 56)  ------- (op: Conv)
[INFO: misc.py:  142]: res2_1_branch2a             : (1, 64, 32, 56, 56)  => res2_1_branch2a_bn                  : (1, 64, 32, 56, 56)  ------- (op: AffineNd)
[INFO: misc.py:  142]: res2_1_branch2a_bn          : (1, 64, 32, 56, 56)  => res2_1_branch2a_bn                  : (1, 64, 32, 56, 56)  ------- (op: Relu)
[INFO: misc.py:  142]: res2_1_branch2a_bn          : (1, 64, 32, 56, 56)  => res2_1_branch2b                     : (1, 64, 32, 56, 56)  ------- (op: Conv)
[INFO: misc.py:  142]: res2_1_branch2b             : (1, 64, 32, 56, 56)  => res2_1_branch2b_bn                  : (1, 64, 32, 56, 56)  ------- (op: AffineNd)
[INFO: misc.py:  142]: res2_1_branch2b_bn          : (1, 64, 32, 56, 56)  => res2_1_branch2b_bn                  : (1, 64, 32, 56, 56)  ------- (op: Relu)
[INFO: misc.py:  142]: res2_1_branch2b_bn          : (1, 64, 32, 56, 56)  => res2_1_branch2c                     : (1, 256, 32, 56, 56) ------- (op: Conv)
[INFO: misc.py:  142]: res2_1_branch2c             : (1, 256, 32, 56, 56) => res2_1_branch2c_bn                  : (1, 256, 32, 56, 56) ------- (op: AffineNd)
[INFO: misc.py:  142]: res2_1_branch2c_bn          : (1, 256, 32, 56, 56) => res2_1_branch2c_bn                  : (1, 256, 32, 56, 56) ------- (op: Sum)
[INFO: misc.py:  142]: res2_0_branch2c_bn          : (1, 256, 32, 56, 56) => res2_1_branch2c_bn                  : (1, 256, 32, 56, 56) ------|
[INFO: misc.py:  142]: res2_1_branch2c_bn          : (1, 256, 32, 56, 56) => res2_1_branch2c_bn                  : (1, 256, 32, 56, 56) ------- (op: Relu)
[INFO: misc.py:  142]: res2_1_branch2c_bn          : (1, 256, 32, 56, 56) => res2_2_branch2a                     : (1, 64, 32, 56, 56)  ------- (op: Conv)
[INFO: misc.py:  142]: res2_2_branch2a             : (1, 64, 32, 56, 56)  => res2_2_branch2a_bn                  : (1, 64, 32, 56, 56)  ------- (op: AffineNd)
[INFO: misc.py:  142]: res2_2_branch2a_bn          : (1, 64, 32, 56, 56)  => res2_2_branch2a_bn                  : (1, 64, 32, 56, 56)  ------- (op: Relu)
[INFO: misc.py:  142]: res2_2_branch2a_bn          : (1, 64, 32, 56, 56)  => res2_2_branch2b                     : (1, 64, 32, 56, 56)  ------- (op: Conv)
[INFO: misc.py:  142]: res2_2_branch2b             : (1, 64, 32, 56, 56)  => res2_2_branch2b_bn                  : (1, 64, 32, 56, 56)  ------- (op: AffineNd)
[INFO: misc.py:  142]: res2_2_branch2b_bn          : (1, 64, 32, 56, 56)  => res2_2_branch2b_bn                  : (1, 64, 32, 56, 56)  ------- (op: Relu)
[INFO: misc.py:  142]: res2_2_branch2b_bn          : (1, 64, 32, 56, 56)  => res2_2_branch2c                     : (1, 256, 32, 56, 56) ------- (op: Conv)
[INFO: misc.py:  142]: res2_2_branch2c             : (1, 256, 32, 56, 56) => res2_2_branch2c_bn                  : (1, 256, 32, 56, 56) ------- (op: AffineNd)
[INFO: misc.py:  142]: res2_2_branch2c_bn          : (1, 256, 32, 56, 56) => res2_2_branch2c_bn                  : (1, 256, 32, 56, 56) ------- (op: Sum)
[INFO: misc.py:  142]: res2_1_branch2c_bn          : (1, 256, 32, 56, 56) => res2_2_branch2c_bn                  : (1, 256, 32, 56, 56) ------|
[INFO: misc.py:  142]: res2_2_branch2c_bn          : (1, 256, 32, 56, 56) => res2_2_branch2c_bn                  : (1, 256, 32, 56, 56) ------- (op: Relu)
[INFO: misc.py:  142]: res2_2_branch2c_bn          : (1, 256, 32, 56, 56) => pool2                               : (1, 256, 16, 56, 56) ------- (op: MaxPool)
[INFO: misc.py:  142]: pool2                       : (1, 256, 16, 56, 56) => res3_0_branch2a                     : (1, 128, 16, 56, 56) ------- (op: Conv)
[INFO: misc.py:  142]: res3_0_branch2a             : (1, 128, 16, 56, 56) => res3_0_branch2a_bn                  : (1, 128, 16, 56, 56) ------- (op: AffineNd)
[INFO: misc.py:  142]: res3_0_branch2a_bn          : (1, 128, 16, 56, 56) => res3_0_branch2a_bn                  : (1, 128, 16, 56, 56) ------- (op: Relu)
[INFO: misc.py:  142]: res3_0_branch2a_bn          : (1, 128, 16, 56, 56) => res3_0_branch2b                     : (1, 128, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py:  142]: res3_0_branch2b             : (1, 128, 16, 28, 28) => res3_0_branch2b_bn                  : (1, 128, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py:  142]: res3_0_branch2b_bn          : (1, 128, 16, 28, 28) => res3_0_branch2b_bn                  : (1, 128, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py:  142]: res3_0_branch2b_bn          : (1, 128, 16, 28, 28) => res3_0_branch2c                     : (1, 512, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py:  142]: res3_0_branch2c             : (1, 512, 16, 28, 28) => res3_0_branch2c_bn                  : (1, 512, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py:  142]: pool2                       : (1, 256, 16, 56, 56) => res3_0_branch1                      : (1, 512, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py:  142]: res3_0_branch1              : (1, 512, 16, 28, 28) => res3_0_branch1_bn                   : (1, 512, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py:  142]: res3_0_branch2c_bn          : (1, 512, 16, 28, 28) => res3_0_branch2c_bn                  : (1, 512, 16, 28, 28) ------- (op: Sum)
[INFO: misc.py:  142]: res3_0_branch1_bn           : (1, 512, 16, 28, 28) => res3_0_branch2c_bn                  : (1, 512, 16, 28, 28) ------|
[INFO: misc.py:  142]: res3_0_branch2c_bn          : (1, 512, 16, 28, 28) => res3_0_branch2c_bn                  : (1, 512, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py:  142]: res3_0_branch2c_bn          : (1, 512, 16, 28, 28) => res3_1_branch2a                     : (1, 128, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py:  142]: res3_1_branch2a             : (1, 128, 16, 28, 28) => res3_1_branch2a_bn                  : (1, 128, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py:  142]: res3_1_branch2a_bn          : (1, 128, 16, 28, 28) => res3_1_branch2a_bn                  : (1, 128, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py:  142]: res3_1_branch2a_bn          : (1, 128, 16, 28, 28) => res3_1_branch2b                     : (1, 128, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py:  142]: res3_1_branch2b             : (1, 128, 16, 28, 28) => res3_1_branch2b_bn                  : (1, 128, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py:  142]: res3_1_branch2b_bn          : (1, 128, 16, 28, 28) => res3_1_branch2b_bn                  : (1, 128, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py:  142]: res3_1_branch2b_bn          : (1, 128, 16, 28, 28) => res3_1_branch2c                     : (1, 512, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py:  142]: res3_1_branch2c             : (1, 512, 16, 28, 28) => res3_1_branch2c_bn                  : (1, 512, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py:  142]: res3_1_branch2c_bn          : (1, 512, 16, 28, 28) => res3_1_branch2c_bn                  : (1, 512, 16, 28, 28) ------- (op: Sum)
[INFO: misc.py:  142]: res3_0_branch2c_bn          : (1, 512, 16, 28, 28) => res3_1_branch2c_bn                  : (1, 512, 16, 28, 28) ------|
[INFO: misc.py:  142]: res3_1_branch2c_bn          : (1, 512, 16, 28, 28) => res3_1_branch2c_bn                  : (1, 512, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py:  142]: res3_1_branch2c_bn          : (1, 512, 16, 28, 28) => res3_1_branch2c_bn_trans            : (4, 4, 512, 28, 28)  ------- (op: Transpose)
[INFO: misc.py:  142]: res3_1_branch2c_bn_trans    : (4, 4, 512, 28, 28)  => res3_1_branch2c_bn_trans            : (4, 4, 512, 28, 28)  ------- (op: Reshape)
[INFO: misc.py:  142]: res3_1_branch2c_bn_trans    : (4, 4, 512, 28, 28)  => res3_1_branch2c_bn_trans_trans      : (4, 512, 4, 28, 28)  ------- (op: Transpose)
[INFO: misc.py:  142]: res3_1_branch2c_bn_trans_trans: (4, 512, 4, 28, 28)  => nonlocal_conv3_1_theta              : (4, 256, 3136)       ------- (op: Conv)
[INFO: misc.py:  142]: res3_1_branch2c_bn_trans_trans: (4, 512, 4, 28, 28)  => nonlocal_conv3_1_pool               : (4, 512, 4, 14, 14)  ------- (op: MaxPool)
[INFO: misc.py:  142]: nonlocal_conv3_1_pool       : (4, 512, 4, 14, 14)  => nonlocal_conv3_1_phi                : (4, 256, 784)        ------- (op: Conv)
[INFO: misc.py:  142]: nonlocal_conv3_1_pool       : (4, 512, 4, 14, 14)  => nonlocal_conv3_1_g                  : (4, 256, 784)        ------- (op: Conv)
[INFO: misc.py:  142]: nonlocal_conv3_1_theta      : (4, 256, 3136)       => nonlocal_conv3_1_theta              : (4, 256, 3136)       ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv3_1_phi        : (4, 256, 784)        => nonlocal_conv3_1_phi                : (4, 256, 784)        ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv3_1_g          : (4, 256, 784)        => nonlocal_conv3_1_g                  : (4, 256, 784)        ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv3_1_theta      : (4, 256, 3136)       => nonlocal_conv3_1_affinity           : (4, 3136, 784)       ------- (op: BatchMatMul)
[INFO: misc.py:  142]: nonlocal_conv3_1_phi        : (4, 256, 784)        => nonlocal_conv3_1_affinity           : (4, 3136, 784)       ------|
[INFO: misc.py:  142]: nonlocal_conv3_1_affinity   : (4, 3136, 784)       => nonlocal_conv3_1_affinity           : (4, 3136, 784)       ------- (op: Scale)
[INFO: misc.py:  142]: nonlocal_conv3_1_affinity   : (4, 3136, 784)       => nonlocal_conv3_1_affinity_prob      : (4, 3136, 784)       ------- (op: Softmax)
[INFO: misc.py:  142]: nonlocal_conv3_1_g          : (4, 256, 784)        => nonlocal_conv3_1_y                  : (4, 256, 4, 28, 28)  ------- (op: BatchMatMul)
[INFO: misc.py:  142]: nonlocal_conv3_1_affinity_prob: (4, 3136, 784)       => nonlocal_conv3_1_y                  : (4, 256, 4, 28, 28)  ------|
[INFO: misc.py:  142]: nonlocal_conv3_1_y          : (4, 256, 4, 28, 28)  => nonlocal_conv3_1_y                  : (4, 256, 4, 28, 28)  ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv3_1_theta_shape5d: (5,)                 => nonlocal_conv3_1_y                  : (4, 256, 4, 28, 28)  ------|
[INFO: misc.py:  142]: nonlocal_conv3_1_y          : (4, 256, 4, 28, 28)  => nonlocal_conv3_1_out                : (4, 512, 4, 28, 28)  ------- (op: Conv)
[INFO: misc.py:  142]: nonlocal_conv3_1_out        : (4, 512, 4, 28, 28)  => nonlocal_conv3_1_bn                 : (4, 512, 4, 28, 28)  ------- (op: AffineNd)
[INFO: misc.py:  142]: res3_1_branch2c_bn_trans_trans: (4, 512, 4, 28, 28)  => nonlocal_conv3_1_sum                : (4, 512, 4, 28, 28)  ------- (op: Sum)
[INFO: misc.py:  142]: nonlocal_conv3_1_bn         : (4, 512, 4, 28, 28)  => nonlocal_conv3_1_sum                : (4, 512, 4, 28, 28)  ------|
[INFO: misc.py:  142]: nonlocal_conv3_1_sum        : (4, 512, 4, 28, 28)  => nonlocal_conv3_1_sum_trans          : (1, 16, 512, 28, 28) ------- (op: Transpose)
[INFO: misc.py:  142]: nonlocal_conv3_1_sum_trans  : (1, 16, 512, 28, 28) => nonlocal_conv3_1_sum_trans          : (1, 16, 512, 28, 28) ------- (op: Reshape)
[INFO: misc.py:  142]: res3_1_branch2c_bn_trans_shape5d: (5,)                 => nonlocal_conv3_1_sum_trans          : (1, 16, 512, 28, 28) ------|
[INFO: misc.py:  142]: nonlocal_conv3_1_sum_trans  : (1, 16, 512, 28, 28) => nonlocal_conv3_1_sum_trans_trans    : (1, 512, 16, 28, 28) ------- (op: Transpose)
[INFO: misc.py:  142]: nonlocal_conv3_1_sum_trans_trans: (1, 512, 16, 28, 28) => res3_2_branch2a                     : (1, 128, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py:  142]: res3_2_branch2a             : (1, 128, 16, 28, 28) => res3_2_branch2a_bn                  : (1, 128, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py:  142]: res3_2_branch2a_bn          : (1, 128, 16, 28, 28) => res3_2_branch2a_bn                  : (1, 128, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py:  142]: res3_2_branch2a_bn          : (1, 128, 16, 28, 28) => res3_2_branch2b                     : (1, 128, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py:  142]: res3_2_branch2b             : (1, 128, 16, 28, 28) => res3_2_branch2b_bn                  : (1, 128, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py:  142]: res3_2_branch2b_bn          : (1, 128, 16, 28, 28) => res3_2_branch2b_bn                  : (1, 128, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py:  142]: res3_2_branch2b_bn          : (1, 128, 16, 28, 28) => res3_2_branch2c                     : (1, 512, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py:  142]: res3_2_branch2c             : (1, 512, 16, 28, 28) => res3_2_branch2c_bn                  : (1, 512, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py:  142]: res3_2_branch2c_bn          : (1, 512, 16, 28, 28) => res3_2_branch2c_bn                  : (1, 512, 16, 28, 28) ------- (op: Sum)
[INFO: misc.py:  142]: nonlocal_conv3_1_sum_trans_trans: (1, 512, 16, 28, 28) => res3_2_branch2c_bn                  : (1, 512, 16, 28, 28) ------|
[INFO: misc.py:  142]: res3_2_branch2c_bn          : (1, 512, 16, 28, 28) => res3_2_branch2c_bn                  : (1, 512, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py:  142]: res3_2_branch2c_bn          : (1, 512, 16, 28, 28) => res3_3_branch2a                     : (1, 128, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py:  142]: res3_3_branch2a             : (1, 128, 16, 28, 28) => res3_3_branch2a_bn                  : (1, 128, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py:  142]: res3_3_branch2a_bn          : (1, 128, 16, 28, 28) => res3_3_branch2a_bn                  : (1, 128, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py:  142]: res3_3_branch2a_bn          : (1, 128, 16, 28, 28) => res3_3_branch2b                     : (1, 128, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py:  142]: res3_3_branch2b             : (1, 128, 16, 28, 28) => res3_3_branch2b_bn                  : (1, 128, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py:  142]: res3_3_branch2b_bn          : (1, 128, 16, 28, 28) => res3_3_branch2b_bn                  : (1, 128, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py:  142]: res3_3_branch2b_bn          : (1, 128, 16, 28, 28) => res3_3_branch2c                     : (1, 512, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py:  142]: res3_3_branch2c             : (1, 512, 16, 28, 28) => res3_3_branch2c_bn                  : (1, 512, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py:  142]: res3_3_branch2c_bn          : (1, 512, 16, 28, 28) => res3_3_branch2c_bn                  : (1, 512, 16, 28, 28) ------- (op: Sum)
[INFO: misc.py:  142]: res3_2_branch2c_bn          : (1, 512, 16, 28, 28) => res3_3_branch2c_bn                  : (1, 512, 16, 28, 28) ------|
[INFO: misc.py:  142]: res3_3_branch2c_bn          : (1, 512, 16, 28, 28) => res3_3_branch2c_bn                  : (1, 512, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py:  142]: res3_3_branch2c_bn          : (1, 512, 16, 28, 28) => res3_3_branch2c_bn_trans            : (4, 4, 512, 28, 28)  ------- (op: Transpose)
[INFO: misc.py:  142]: res3_3_branch2c_bn_trans    : (4, 4, 512, 28, 28)  => res3_3_branch2c_bn_trans            : (4, 4, 512, 28, 28)  ------- (op: Reshape)
[INFO: misc.py:  142]: res3_3_branch2c_bn_trans    : (4, 4, 512, 28, 28)  => res3_3_branch2c_bn_trans_trans      : (4, 512, 4, 28, 28)  ------- (op: Transpose)
[INFO: misc.py:  142]: res3_3_branch2c_bn_trans_trans: (4, 512, 4, 28, 28)  => nonlocal_conv3_3_theta              : (4, 256, 3136)       ------- (op: Conv)
[INFO: misc.py:  142]: res3_3_branch2c_bn_trans_trans: (4, 512, 4, 28, 28)  => nonlocal_conv3_3_pool               : (4, 512, 4, 14, 14)  ------- (op: MaxPool)
[INFO: misc.py:  142]: nonlocal_conv3_3_pool       : (4, 512, 4, 14, 14)  => nonlocal_conv3_3_phi                : (4, 256, 784)        ------- (op: Conv)
[INFO: misc.py:  142]: nonlocal_conv3_3_pool       : (4, 512, 4, 14, 14)  => nonlocal_conv3_3_g                  : (4, 256, 784)        ------- (op: Conv)
[INFO: misc.py:  142]: nonlocal_conv3_3_theta      : (4, 256, 3136)       => nonlocal_conv3_3_theta              : (4, 256, 3136)       ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv3_3_phi        : (4, 256, 784)        => nonlocal_conv3_3_phi                : (4, 256, 784)        ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv3_3_g          : (4, 256, 784)        => nonlocal_conv3_3_g                  : (4, 256, 784)        ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv3_3_theta      : (4, 256, 3136)       => nonlocal_conv3_3_affinity           : (4, 3136, 784)       ------- (op: BatchMatMul)
[INFO: misc.py:  142]: nonlocal_conv3_3_phi        : (4, 256, 784)        => nonlocal_conv3_3_affinity           : (4, 3136, 784)       ------|
[INFO: misc.py:  142]: nonlocal_conv3_3_affinity   : (4, 3136, 784)       => nonlocal_conv3_3_affinity           : (4, 3136, 784)       ------- (op: Scale)
[INFO: misc.py:  142]: nonlocal_conv3_3_affinity   : (4, 3136, 784)       => nonlocal_conv3_3_affinity_prob      : (4, 3136, 784)       ------- (op: Softmax)
[INFO: misc.py:  142]: nonlocal_conv3_3_g          : (4, 256, 784)        => nonlocal_conv3_3_y                  : (4, 256, 4, 28, 28)  ------- (op: BatchMatMul)
[INFO: misc.py:  142]: nonlocal_conv3_3_affinity_prob: (4, 3136, 784)       => nonlocal_conv3_3_y                  : (4, 256, 4, 28, 28)  ------|
[INFO: misc.py:  142]: nonlocal_conv3_3_y          : (4, 256, 4, 28, 28)  => nonlocal_conv3_3_y                  : (4, 256, 4, 28, 28)  ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv3_3_theta_shape5d: (5,)                 => nonlocal_conv3_3_y                  : (4, 256, 4, 28, 28)  ------|
[INFO: misc.py:  142]: nonlocal_conv3_3_y          : (4, 256, 4, 28, 28)  => nonlocal_conv3_3_out                : (4, 512, 4, 28, 28)  ------- (op: Conv)
[INFO: misc.py:  142]: nonlocal_conv3_3_out        : (4, 512, 4, 28, 28)  => nonlocal_conv3_3_bn                 : (4, 512, 4, 28, 28)  ------- (op: AffineNd)
[INFO: misc.py:  142]: res3_3_branch2c_bn_trans_trans: (4, 512, 4, 28, 28)  => nonlocal_conv3_3_sum                : (4, 512, 4, 28, 28)  ------- (op: Sum)
[INFO: misc.py:  142]: nonlocal_conv3_3_bn         : (4, 512, 4, 28, 28)  => nonlocal_conv3_3_sum                : (4, 512, 4, 28, 28)  ------|
[INFO: misc.py:  142]: nonlocal_conv3_3_sum        : (4, 512, 4, 28, 28)  => nonlocal_conv3_3_sum_trans          : (1, 16, 512, 28, 28) ------- (op: Transpose)
[INFO: misc.py:  142]: nonlocal_conv3_3_sum_trans  : (1, 16, 512, 28, 28) => nonlocal_conv3_3_sum_trans          : (1, 16, 512, 28, 28) ------- (op: Reshape)
[INFO: misc.py:  142]: res3_3_branch2c_bn_trans_shape5d: (5,)                 => nonlocal_conv3_3_sum_trans          : (1, 16, 512, 28, 28) ------|
[INFO: misc.py:  142]: nonlocal_conv3_3_sum_trans  : (1, 16, 512, 28, 28) => nonlocal_conv3_3_sum_trans_trans    : (1, 512, 16, 28, 28) ------- (op: Transpose)
[INFO: misc.py:  142]: nonlocal_conv3_3_sum_trans_trans: (1, 512, 16, 28, 28) => res4_0_branch2a                     : (1, 256, 16, 28, 28) ------- (op: Conv)
[INFO: misc.py:  142]: res4_0_branch2a             : (1, 256, 16, 28, 28) => res4_0_branch2a_bn                  : (1, 256, 16, 28, 28) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_0_branch2a_bn          : (1, 256, 16, 28, 28) => res4_0_branch2a_bn                  : (1, 256, 16, 28, 28) ------- (op: Relu)
[INFO: misc.py:  142]: res4_0_branch2a_bn          : (1, 256, 16, 28, 28) => res4_0_branch2b                     : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res4_0_branch2b             : (1, 256, 16, 14, 14) => res4_0_branch2b_bn                  : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_0_branch2b_bn          : (1, 256, 16, 14, 14) => res4_0_branch2b_bn                  : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res4_0_branch2b_bn          : (1, 256, 16, 14, 14) => res4_0_branch2c                     : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res4_0_branch2c             : (1, 1024, 16, 14, 14) => res4_0_branch2c_bn                  : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: nonlocal_conv3_3_sum_trans_trans: (1, 512, 16, 28, 28) => res4_0_branch1                      : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res4_0_branch1              : (1, 1024, 16, 14, 14) => res4_0_branch1_bn                   : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_0_branch2c_bn          : (1, 1024, 16, 14, 14) => res4_0_branch2c_bn                  : (1, 1024, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py:  142]: res4_0_branch1_bn           : (1, 1024, 16, 14, 14) => res4_0_branch2c_bn                  : (1, 1024, 16, 14, 14) ------|
[INFO: misc.py:  142]: res4_0_branch2c_bn          : (1, 1024, 16, 14, 14) => res4_0_branch2c_bn                  : (1, 1024, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res4_0_branch2c_bn          : (1, 1024, 16, 14, 14) => res4_1_branch2a                     : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res4_1_branch2a             : (1, 256, 16, 14, 14) => res4_1_branch2a_bn                  : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_1_branch2a_bn          : (1, 256, 16, 14, 14) => res4_1_branch2a_bn                  : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res4_1_branch2a_bn          : (1, 256, 16, 14, 14) => res4_1_branch2b                     : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res4_1_branch2b             : (1, 256, 16, 14, 14) => res4_1_branch2b_bn                  : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_1_branch2b_bn          : (1, 256, 16, 14, 14) => res4_1_branch2b_bn                  : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res4_1_branch2b_bn          : (1, 256, 16, 14, 14) => res4_1_branch2c                     : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res4_1_branch2c             : (1, 1024, 16, 14, 14) => res4_1_branch2c_bn                  : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_1_branch2c_bn          : (1, 1024, 16, 14, 14) => res4_1_branch2c_bn                  : (1, 1024, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py:  142]: res4_0_branch2c_bn          : (1, 1024, 16, 14, 14) => res4_1_branch2c_bn                  : (1, 1024, 16, 14, 14) ------|
[INFO: misc.py:  142]: res4_1_branch2c_bn          : (1, 1024, 16, 14, 14) => res4_1_branch2c_bn                  : (1, 1024, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res4_1_branch2c_bn          : (1, 1024, 16, 14, 14) => nonlocal_conv4_1_theta              : (1, 512, 3136)       ------- (op: Conv)
[INFO: misc.py:  142]: res4_1_branch2c_bn          : (1, 1024, 16, 14, 14) => nonlocal_conv4_1_pool               : (1, 1024, 16, 7, 7)  ------- (op: MaxPool)
[INFO: misc.py:  142]: nonlocal_conv4_1_pool       : (1, 1024, 16, 7, 7)  => nonlocal_conv4_1_phi                : (1, 512, 784)        ------- (op: Conv)
[INFO: misc.py:  142]: nonlocal_conv4_1_pool       : (1, 1024, 16, 7, 7)  => nonlocal_conv4_1_g                  : (1, 512, 784)        ------- (op: Conv)
[INFO: misc.py:  142]: nonlocal_conv4_1_theta      : (1, 512, 3136)       => nonlocal_conv4_1_theta              : (1, 512, 3136)       ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv4_1_phi        : (1, 512, 784)        => nonlocal_conv4_1_phi                : (1, 512, 784)        ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv4_1_g          : (1, 512, 784)        => nonlocal_conv4_1_g                  : (1, 512, 784)        ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv4_1_theta      : (1, 512, 3136)       => nonlocal_conv4_1_affinity           : (1, 3136, 784)       ------- (op: BatchMatMul)
[INFO: misc.py:  142]: nonlocal_conv4_1_phi        : (1, 512, 784)        => nonlocal_conv4_1_affinity           : (1, 3136, 784)       ------|
[INFO: misc.py:  142]: nonlocal_conv4_1_affinity   : (1, 3136, 784)       => nonlocal_conv4_1_affinity           : (1, 3136, 784)       ------- (op: Scale)
[INFO: misc.py:  142]: nonlocal_conv4_1_affinity   : (1, 3136, 784)       => nonlocal_conv4_1_affinity_prob      : (1, 3136, 784)       ------- (op: Softmax)
[INFO: misc.py:  142]: nonlocal_conv4_1_g          : (1, 512, 784)        => nonlocal_conv4_1_y                  : (1, 512, 16, 14, 14) ------- (op: BatchMatMul)
[INFO: misc.py:  142]: nonlocal_conv4_1_affinity_prob: (1, 3136, 784)       => nonlocal_conv4_1_y                  : (1, 512, 16, 14, 14) ------|
[INFO: misc.py:  142]: nonlocal_conv4_1_y          : (1, 512, 16, 14, 14) => nonlocal_conv4_1_y                  : (1, 512, 16, 14, 14) ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv4_1_theta_shape5d: (5,)                 => nonlocal_conv4_1_y                  : (1, 512, 16, 14, 14) ------|
[INFO: misc.py:  142]: nonlocal_conv4_1_y          : (1, 512, 16, 14, 14) => nonlocal_conv4_1_out                : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: nonlocal_conv4_1_out        : (1, 1024, 16, 14, 14) => nonlocal_conv4_1_bn                 : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_1_branch2c_bn          : (1, 1024, 16, 14, 14) => nonlocal_conv4_1_sum                : (1, 1024, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py:  142]: nonlocal_conv4_1_bn         : (1, 1024, 16, 14, 14) => nonlocal_conv4_1_sum                : (1, 1024, 16, 14, 14) ------|
[INFO: misc.py:  142]: nonlocal_conv4_1_sum        : (1, 1024, 16, 14, 14) => res4_2_branch2a                     : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res4_2_branch2a             : (1, 256, 16, 14, 14) => res4_2_branch2a_bn                  : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_2_branch2a_bn          : (1, 256, 16, 14, 14) => res4_2_branch2a_bn                  : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res4_2_branch2a_bn          : (1, 256, 16, 14, 14) => res4_2_branch2b                     : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res4_2_branch2b             : (1, 256, 16, 14, 14) => res4_2_branch2b_bn                  : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_2_branch2b_bn          : (1, 256, 16, 14, 14) => res4_2_branch2b_bn                  : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res4_2_branch2b_bn          : (1, 256, 16, 14, 14) => res4_2_branch2c                     : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res4_2_branch2c             : (1, 1024, 16, 14, 14) => res4_2_branch2c_bn                  : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_2_branch2c_bn          : (1, 1024, 16, 14, 14) => res4_2_branch2c_bn                  : (1, 1024, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py:  142]: nonlocal_conv4_1_sum        : (1, 1024, 16, 14, 14) => res4_2_branch2c_bn                  : (1, 1024, 16, 14, 14) ------|
[INFO: misc.py:  142]: res4_2_branch2c_bn          : (1, 1024, 16, 14, 14) => res4_2_branch2c_bn                  : (1, 1024, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res4_2_branch2c_bn          : (1, 1024, 16, 14, 14) => res4_3_branch2a                     : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res4_3_branch2a             : (1, 256, 16, 14, 14) => res4_3_branch2a_bn                  : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_3_branch2a_bn          : (1, 256, 16, 14, 14) => res4_3_branch2a_bn                  : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res4_3_branch2a_bn          : (1, 256, 16, 14, 14) => res4_3_branch2b                     : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res4_3_branch2b             : (1, 256, 16, 14, 14) => res4_3_branch2b_bn                  : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_3_branch2b_bn          : (1, 256, 16, 14, 14) => res4_3_branch2b_bn                  : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res4_3_branch2b_bn          : (1, 256, 16, 14, 14) => res4_3_branch2c                     : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res4_3_branch2c             : (1, 1024, 16, 14, 14) => res4_3_branch2c_bn                  : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_3_branch2c_bn          : (1, 1024, 16, 14, 14) => res4_3_branch2c_bn                  : (1, 1024, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py:  142]: res4_2_branch2c_bn          : (1, 1024, 16, 14, 14) => res4_3_branch2c_bn                  : (1, 1024, 16, 14, 14) ------|
[INFO: misc.py:  142]: res4_3_branch2c_bn          : (1, 1024, 16, 14, 14) => res4_3_branch2c_bn                  : (1, 1024, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res4_3_branch2c_bn          : (1, 1024, 16, 14, 14) => nonlocal_conv4_3_theta              : (1, 512, 3136)       ------- (op: Conv)
[INFO: misc.py:  142]: res4_3_branch2c_bn          : (1, 1024, 16, 14, 14) => nonlocal_conv4_3_pool               : (1, 1024, 16, 7, 7)  ------- (op: MaxPool)
[INFO: misc.py:  142]: nonlocal_conv4_3_pool       : (1, 1024, 16, 7, 7)  => nonlocal_conv4_3_phi                : (1, 512, 784)        ------- (op: Conv)
[INFO: misc.py:  142]: nonlocal_conv4_3_pool       : (1, 1024, 16, 7, 7)  => nonlocal_conv4_3_g                  : (1, 512, 784)        ------- (op: Conv)
[INFO: misc.py:  142]: nonlocal_conv4_3_theta      : (1, 512, 3136)       => nonlocal_conv4_3_theta              : (1, 512, 3136)       ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv4_3_phi        : (1, 512, 784)        => nonlocal_conv4_3_phi                : (1, 512, 784)        ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv4_3_g          : (1, 512, 784)        => nonlocal_conv4_3_g                  : (1, 512, 784)        ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv4_3_theta      : (1, 512, 3136)       => nonlocal_conv4_3_affinity           : (1, 3136, 784)       ------- (op: BatchMatMul)
[INFO: misc.py:  142]: nonlocal_conv4_3_phi        : (1, 512, 784)        => nonlocal_conv4_3_affinity           : (1, 3136, 784)       ------|
[INFO: misc.py:  142]: nonlocal_conv4_3_affinity   : (1, 3136, 784)       => nonlocal_conv4_3_affinity           : (1, 3136, 784)       ------- (op: Scale)
[INFO: misc.py:  142]: nonlocal_conv4_3_affinity   : (1, 3136, 784)       => nonlocal_conv4_3_affinity_prob      : (1, 3136, 784)       ------- (op: Softmax)
[INFO: misc.py:  142]: nonlocal_conv4_3_g          : (1, 512, 784)        => nonlocal_conv4_3_y                  : (1, 512, 16, 14, 14) ------- (op: BatchMatMul)
[INFO: misc.py:  142]: nonlocal_conv4_3_affinity_prob: (1, 3136, 784)       => nonlocal_conv4_3_y                  : (1, 512, 16, 14, 14) ------|
[INFO: misc.py:  142]: nonlocal_conv4_3_y          : (1, 512, 16, 14, 14) => nonlocal_conv4_3_y                  : (1, 512, 16, 14, 14) ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv4_3_theta_shape5d: (5,)                 => nonlocal_conv4_3_y                  : (1, 512, 16, 14, 14) ------|
[INFO: misc.py:  142]: nonlocal_conv4_3_y          : (1, 512, 16, 14, 14) => nonlocal_conv4_3_out                : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: nonlocal_conv4_3_out        : (1, 1024, 16, 14, 14) => nonlocal_conv4_3_bn                 : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_3_branch2c_bn          : (1, 1024, 16, 14, 14) => nonlocal_conv4_3_sum                : (1, 1024, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py:  142]: nonlocal_conv4_3_bn         : (1, 1024, 16, 14, 14) => nonlocal_conv4_3_sum                : (1, 1024, 16, 14, 14) ------|
[INFO: misc.py:  142]: nonlocal_conv4_3_sum        : (1, 1024, 16, 14, 14) => res4_4_branch2a                     : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res4_4_branch2a             : (1, 256, 16, 14, 14) => res4_4_branch2a_bn                  : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_4_branch2a_bn          : (1, 256, 16, 14, 14) => res4_4_branch2a_bn                  : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res4_4_branch2a_bn          : (1, 256, 16, 14, 14) => res4_4_branch2b                     : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res4_4_branch2b             : (1, 256, 16, 14, 14) => res4_4_branch2b_bn                  : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_4_branch2b_bn          : (1, 256, 16, 14, 14) => res4_4_branch2b_bn                  : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res4_4_branch2b_bn          : (1, 256, 16, 14, 14) => res4_4_branch2c                     : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res4_4_branch2c             : (1, 1024, 16, 14, 14) => res4_4_branch2c_bn                  : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_4_branch2c_bn          : (1, 1024, 16, 14, 14) => res4_4_branch2c_bn                  : (1, 1024, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py:  142]: nonlocal_conv4_3_sum        : (1, 1024, 16, 14, 14) => res4_4_branch2c_bn                  : (1, 1024, 16, 14, 14) ------|
[INFO: misc.py:  142]: res4_4_branch2c_bn          : (1, 1024, 16, 14, 14) => res4_4_branch2c_bn                  : (1, 1024, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res4_4_branch2c_bn          : (1, 1024, 16, 14, 14) => res4_5_branch2a                     : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res4_5_branch2a             : (1, 256, 16, 14, 14) => res4_5_branch2a_bn                  : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_5_branch2a_bn          : (1, 256, 16, 14, 14) => res4_5_branch2a_bn                  : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res4_5_branch2a_bn          : (1, 256, 16, 14, 14) => res4_5_branch2b                     : (1, 256, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res4_5_branch2b             : (1, 256, 16, 14, 14) => res4_5_branch2b_bn                  : (1, 256, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_5_branch2b_bn          : (1, 256, 16, 14, 14) => res4_5_branch2b_bn                  : (1, 256, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res4_5_branch2b_bn          : (1, 256, 16, 14, 14) => res4_5_branch2c                     : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res4_5_branch2c             : (1, 1024, 16, 14, 14) => res4_5_branch2c_bn                  : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_5_branch2c_bn          : (1, 1024, 16, 14, 14) => res4_5_branch2c_bn                  : (1, 1024, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py:  142]: res4_4_branch2c_bn          : (1, 1024, 16, 14, 14) => res4_5_branch2c_bn                  : (1, 1024, 16, 14, 14) ------|
[INFO: misc.py:  142]: res4_5_branch2c_bn          : (1, 1024, 16, 14, 14) => res4_5_branch2c_bn                  : (1, 1024, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res4_5_branch2c_bn          : (1, 1024, 16, 14, 14) => nonlocal_conv4_5_theta              : (1, 512, 3136)       ------- (op: Conv)
[INFO: misc.py:  142]: res4_5_branch2c_bn          : (1, 1024, 16, 14, 14) => nonlocal_conv4_5_pool               : (1, 1024, 16, 7, 7)  ------- (op: MaxPool)
[INFO: misc.py:  142]: nonlocal_conv4_5_pool       : (1, 1024, 16, 7, 7)  => nonlocal_conv4_5_phi                : (1, 512, 784)        ------- (op: Conv)
[INFO: misc.py:  142]: nonlocal_conv4_5_pool       : (1, 1024, 16, 7, 7)  => nonlocal_conv4_5_g                  : (1, 512, 784)        ------- (op: Conv)
[INFO: misc.py:  142]: nonlocal_conv4_5_theta      : (1, 512, 3136)       => nonlocal_conv4_5_theta              : (1, 512, 3136)       ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv4_5_phi        : (1, 512, 784)        => nonlocal_conv4_5_phi                : (1, 512, 784)        ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv4_5_g          : (1, 512, 784)        => nonlocal_conv4_5_g                  : (1, 512, 784)        ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv4_5_theta      : (1, 512, 3136)       => nonlocal_conv4_5_affinity           : (1, 3136, 784)       ------- (op: BatchMatMul)
[INFO: misc.py:  142]: nonlocal_conv4_5_phi        : (1, 512, 784)        => nonlocal_conv4_5_affinity           : (1, 3136, 784)       ------|
[INFO: misc.py:  142]: nonlocal_conv4_5_affinity   : (1, 3136, 784)       => nonlocal_conv4_5_affinity           : (1, 3136, 784)       ------- (op: Scale)
[INFO: misc.py:  142]: nonlocal_conv4_5_affinity   : (1, 3136, 784)       => nonlocal_conv4_5_affinity_prob      : (1, 3136, 784)       ------- (op: Softmax)
[INFO: misc.py:  142]: nonlocal_conv4_5_g          : (1, 512, 784)        => nonlocal_conv4_5_y                  : (1, 512, 16, 14, 14) ------- (op: BatchMatMul)
[INFO: misc.py:  142]: nonlocal_conv4_5_affinity_prob: (1, 3136, 784)       => nonlocal_conv4_5_y                  : (1, 512, 16, 14, 14) ------|
[INFO: misc.py:  142]: nonlocal_conv4_5_y          : (1, 512, 16, 14, 14) => nonlocal_conv4_5_y                  : (1, 512, 16, 14, 14) ------- (op: Reshape)
[INFO: misc.py:  142]: nonlocal_conv4_5_theta_shape5d: (5,)                 => nonlocal_conv4_5_y                  : (1, 512, 16, 14, 14) ------|
[INFO: misc.py:  142]: nonlocal_conv4_5_y          : (1, 512, 16, 14, 14) => nonlocal_conv4_5_out                : (1, 1024, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: nonlocal_conv4_5_out        : (1, 1024, 16, 14, 14) => nonlocal_conv4_5_bn                 : (1, 1024, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res4_5_branch2c_bn          : (1, 1024, 16, 14, 14) => nonlocal_conv4_5_sum                : (1, 1024, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py:  142]: nonlocal_conv4_5_bn         : (1, 1024, 16, 14, 14) => nonlocal_conv4_5_sum                : (1, 1024, 16, 14, 14) ------|
[INFO: misc.py:  142]: nonlocal_conv4_5_sum        : (1, 1024, 16, 14, 14) => res5_0_branch2a                     : (1, 512, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res5_0_branch2a             : (1, 512, 16, 14, 14) => res5_0_branch2a_bn                  : (1, 512, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res5_0_branch2a_bn          : (1, 512, 16, 14, 14) => res5_0_branch2a_bn                  : (1, 512, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res5_0_branch2a_bn          : (1, 512, 16, 14, 14) => res5_0_branch2b                     : (1, 512, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res5_0_branch2b             : (1, 512, 16, 14, 14) => res5_0_branch2b_bn                  : (1, 512, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res5_0_branch2b_bn          : (1, 512, 16, 14, 14) => res5_0_branch2b_bn                  : (1, 512, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res5_0_branch2b_bn          : (1, 512, 16, 14, 14) => res5_0_branch2c                     : (1, 2048, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res5_0_branch2c             : (1, 2048, 16, 14, 14) => res5_0_branch2c_bn                  : (1, 2048, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: nonlocal_conv4_5_sum        : (1, 1024, 16, 14, 14) => res5_0_branch1                      : (1, 2048, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res5_0_branch1              : (1, 2048, 16, 14, 14) => res5_0_branch1_bn                   : (1, 2048, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res5_0_branch2c_bn          : (1, 2048, 16, 14, 14) => res5_0_branch2c_bn                  : (1, 2048, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py:  142]: res5_0_branch1_bn           : (1, 2048, 16, 14, 14) => res5_0_branch2c_bn                  : (1, 2048, 16, 14, 14) ------|
[INFO: misc.py:  142]: res5_0_branch2c_bn          : (1, 2048, 16, 14, 14) => res5_0_branch2c_bn                  : (1, 2048, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res5_0_branch2c_bn          : (1, 2048, 16, 14, 14) => res5_1_branch2a                     : (1, 512, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res5_1_branch2a             : (1, 512, 16, 14, 14) => res5_1_branch2a_bn                  : (1, 512, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res5_1_branch2a_bn          : (1, 512, 16, 14, 14) => res5_1_branch2a_bn                  : (1, 512, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res5_1_branch2a_bn          : (1, 512, 16, 14, 14) => res5_1_branch2b                     : (1, 512, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res5_1_branch2b             : (1, 512, 16, 14, 14) => res5_1_branch2b_bn                  : (1, 512, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res5_1_branch2b_bn          : (1, 512, 16, 14, 14) => res5_1_branch2b_bn                  : (1, 512, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res5_1_branch2b_bn          : (1, 512, 16, 14, 14) => res5_1_branch2c                     : (1, 2048, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res5_1_branch2c             : (1, 2048, 16, 14, 14) => res5_1_branch2c_bn                  : (1, 2048, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res5_1_branch2c_bn          : (1, 2048, 16, 14, 14) => res5_1_branch2c_bn                  : (1, 2048, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py:  142]: res5_0_branch2c_bn          : (1, 2048, 16, 14, 14) => res5_1_branch2c_bn                  : (1, 2048, 16, 14, 14) ------|
[INFO: misc.py:  142]: res5_1_branch2c_bn          : (1, 2048, 16, 14, 14) => res5_1_branch2c_bn                  : (1, 2048, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res5_1_branch2c_bn          : (1, 2048, 16, 14, 14) => res5_2_branch2a                     : (1, 512, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res5_2_branch2a             : (1, 512, 16, 14, 14) => res5_2_branch2a_bn                  : (1, 512, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res5_2_branch2a_bn          : (1, 512, 16, 14, 14) => res5_2_branch2a_bn                  : (1, 512, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res5_2_branch2a_bn          : (1, 512, 16, 14, 14) => res5_2_branch2b                     : (1, 512, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res5_2_branch2b             : (1, 512, 16, 14, 14) => res5_2_branch2b_bn                  : (1, 512, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res5_2_branch2b_bn          : (1, 512, 16, 14, 14) => res5_2_branch2b_bn                  : (1, 512, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res5_2_branch2b_bn          : (1, 512, 16, 14, 14) => res5_2_branch2c                     : (1, 2048, 16, 14, 14) ------- (op: Conv)
[INFO: misc.py:  142]: res5_2_branch2c             : (1, 2048, 16, 14, 14) => res5_2_branch2c_bn                  : (1, 2048, 16, 14, 14) ------- (op: AffineNd)
[INFO: misc.py:  142]: res5_2_branch2c_bn          : (1, 2048, 16, 14, 14) => res5_2_branch2c_bn                  : (1, 2048, 16, 14, 14) ------- (op: Sum)
[INFO: misc.py:  142]: res5_1_branch2c_bn          : (1, 2048, 16, 14, 14) => res5_2_branch2c_bn                  : (1, 2048, 16, 14, 14) ------|
[INFO: misc.py:  142]: res5_2_branch2c_bn          : (1, 2048, 16, 14, 14) => res5_2_branch2c_bn                  : (1, 2048, 16, 14, 14) ------- (op: Relu)
[INFO: misc.py:  142]: res5_2_branch2c_bn          : (1, 2048, 16, 14, 14) => res5_2_branch2c_bn_pooled           : (1, 2048, 1, 1, 1)   ------- (op: AveragePool)
[INFO: misc.py:  142]: res5_2_branch2c_bn_pooled   : (1, 2048, 1, 1, 1)   => pool5                               : (1, 2048, 1, 1, 1)   ------- (op: Concat)
[INFO: misc.py:  142]: pool5                       : (1, 2048, 1, 1, 1)   => pool5_dropout                       : (1, 2048, 1, 1, 1)   ------- (op: Dropout)
[INFO: misc.py:  142]: pool5_dropout               : (1, 2048, 1, 1, 1)   => pred                                : (1, 157)             ------- (op: FC)
[INFO: misc.py:  142]: pred                        : (1, 157)             => prob                                : (1, 157)             ------- (op: Sigmoid)
[INFO: misc.py:  142]: pred                        : (1, 157)             => loss                                : ()                   ------- (op: SigmoidCrossEntropyLoss)
[INFO: misc.py:  142]: labels_train                : (1, 157)             => loss                                : ()                   ------|
[INFO: misc.py:  142]: pred                        : (1, 157)             => __m14_shared                        : (1, 2048, 16, 14, 14) ------- (op: SigmoidCrossEntropyLossGradient)
[INFO: misc.py:  142]: labels_train                : (1, 157)             => __m14_shared                        : (1, 2048, 16, 14, 14) ------|
[INFO: misc.py:  142]: __m6_shared                 : (1, 3, 32, 224, 224) => __m14_shared                        : (1, 2048, 16, 14, 14) ------|
[INFO: misc.py:  147]: End of model: CNN_1
Mon Aug  2 10:38:38 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.32.03    Driver Version: 460.32.03    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla V100-SXM2...  On   | 00000000:3D:00.0 Off |                    0 |
| N/A   39C    P0    58W / 300W |  13178MiB / 16160MiB |      6%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  Tesla V100-SXM2...  On   | 00000000:3E:00.0 Off |                    0 |
| N/A   38C    P0    59W / 300W |  13134MiB / 16160MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   2  Tesla V100-SXM2...  On   | 00000000:B1:00.0 Off |                    0 |
| N/A   38C    P0    58W / 300W |  13134MiB / 16160MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   3  Tesla V100-SXM2...  On   | 00000000:B2:00.0 Off |                    0 |
| N/A   38C    P0    59W / 300W |  13134MiB / 16160MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     41058      C   python                          13175MiB |
|    1   N/A  N/A     41058      C   python                          13131MiB |
|    2   N/A  N/A     41058      C   python                          13131MiB |
|    3   N/A  N/A     41058      C   python                          13131MiB |
+-----------------------------------------------------------------------------+
[INFO: misc.py:  268]: layer gpu_0/conv1 (1) FLOPs: 18882.23 M PARAMs: 47.04 K
[INFO: misc.py:  268]: layer gpu_0/res2_0_branch2a (1) FLOPs: 1233.13 M PARAMs: 12.29 K
[INFO: misc.py:  268]: layer gpu_0/res2_0_branch2b (1) FLOPs: 3699.38 M PARAMs: 36.86 K
[INFO: misc.py:  268]: layer gpu_0/res2_0_branch2c (1) FLOPs: 1644.17 M PARAMs: 16.38 K
[INFO: misc.py:  268]: layer gpu_0/res2_0_branch1 (1) FLOPs: 1644.17 M PARAMs: 16.38 K
[INFO: misc.py:  268]: layer gpu_0/res2_1_branch2a (1) FLOPs: 4932.50 M PARAMs: 49.15 K
[INFO: misc.py:  268]: layer gpu_0/res2_1_branch2b (1) FLOPs: 3699.38 M PARAMs: 36.86 K
[INFO: misc.py:  268]: layer gpu_0/res2_1_branch2c (1) FLOPs: 1644.17 M PARAMs: 16.38 K
[INFO: misc.py:  268]: layer gpu_0/res2_2_branch2a (1) FLOPs: 4932.50 M PARAMs: 49.15 K
[INFO: misc.py:  268]: layer gpu_0/res2_2_branch2b (1) FLOPs: 3699.38 M PARAMs: 36.86 K
[INFO: misc.py:  268]: layer gpu_0/res2_2_branch2c (1) FLOPs: 1644.17 M PARAMs: 16.38 K
[INFO: misc.py:  268]: layer gpu_0/res3_0_branch2a (1) FLOPs: 4932.50 M PARAMs: 98.30 K
[INFO: misc.py:  268]: layer gpu_0/res3_0_branch2b (1) FLOPs: 1849.69 M PARAMs: 147.46 K
[INFO: misc.py:  268]: layer gpu_0/res3_0_branch2c (1) FLOPs: 822.08 M PARAMs: 65.54 K
[INFO: misc.py:  268]: layer gpu_0/res3_0_branch1 (1) FLOPs: 1644.17 M PARAMs: 131.07 K
[INFO: misc.py:  268]: layer gpu_0/res3_1_branch2a (1) FLOPs: 822.08 M PARAMs: 65.54 K
[INFO: misc.py:  268]: layer gpu_0/res3_1_branch2b (1) FLOPs: 1849.69 M PARAMs: 147.46 K
[INFO: misc.py:  268]: layer gpu_0/res3_1_branch2c (1) FLOPs: 822.08 M PARAMs: 65.54 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv3_1_theta (4) FLOPs: 1644.17 M PARAMs: 131.07 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv3_1_phi (4) FLOPs: 411.04 M PARAMs: 131.07 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv3_1_g (4) FLOPs: 411.04 M PARAMs: 131.07 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv3_1_affinity (4) FLOPs: 2517.63 M PARAMs: 0.00 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv3_1_y (4) FLOPs: 2517.63 M PARAMs: 0.00 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv3_1_out (4) FLOPs: 1644.17 M PARAMs: 131.07 K
[INFO: misc.py:  268]: layer gpu_0/res3_2_branch2a (1) FLOPs: 2466.25 M PARAMs: 196.61 K
[INFO: misc.py:  268]: layer gpu_0/res3_2_branch2b (1) FLOPs: 1849.69 M PARAMs: 147.46 K
[INFO: misc.py:  268]: layer gpu_0/res3_2_branch2c (1) FLOPs: 822.08 M PARAMs: 65.54 K
[INFO: misc.py:  268]: layer gpu_0/res3_3_branch2a (1) FLOPs: 822.08 M PARAMs: 65.54 K
[INFO: misc.py:  268]: layer gpu_0/res3_3_branch2b (1) FLOPs: 1849.69 M PARAMs: 147.46 K
[INFO: misc.py:  268]: layer gpu_0/res3_3_branch2c (1) FLOPs: 822.08 M PARAMs: 65.54 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv3_3_theta (4) FLOPs: 1644.17 M PARAMs: 131.07 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv3_3_phi (4) FLOPs: 411.04 M PARAMs: 131.07 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv3_3_g (4) FLOPs: 411.04 M PARAMs: 131.07 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv3_3_affinity (4) FLOPs: 2517.63 M PARAMs: 0.00 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv3_3_y (4) FLOPs: 2517.63 M PARAMs: 0.00 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv3_3_out (4) FLOPs: 1644.17 M PARAMs: 131.07 K
[INFO: misc.py:  268]: layer gpu_0/res4_0_branch2a (1) FLOPs: 4932.50 M PARAMs: 393.22 K
[INFO: misc.py:  268]: layer gpu_0/res4_0_branch2b (1) FLOPs: 1849.69 M PARAMs: 589.82 K
[INFO: misc.py:  268]: layer gpu_0/res4_0_branch2c (1) FLOPs: 822.08 M PARAMs: 262.14 K
[INFO: misc.py:  268]: layer gpu_0/res4_0_branch1 (1) FLOPs: 1644.17 M PARAMs: 524.29 K
[INFO: misc.py:  268]: layer gpu_0/res4_1_branch2a (1) FLOPs: 822.08 M PARAMs: 262.14 K
[INFO: misc.py:  268]: layer gpu_0/res4_1_branch2b (1) FLOPs: 1849.69 M PARAMs: 589.82 K
[INFO: misc.py:  268]: layer gpu_0/res4_1_branch2c (1) FLOPs: 822.08 M PARAMs: 262.14 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv4_1_theta (1) FLOPs: 1644.17 M PARAMs: 524.29 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv4_1_phi (1) FLOPs: 411.04 M PARAMs: 524.29 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv4_1_g (1) FLOPs: 411.04 M PARAMs: 524.29 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv4_1_affinity (1) FLOPs: 1258.82 M PARAMs: 0.00 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv4_1_y (1) FLOPs: 1258.82 M PARAMs: 0.00 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv4_1_out (1) FLOPs: 1644.17 M PARAMs: 524.29 K
[INFO: misc.py:  268]: layer gpu_0/res4_2_branch2a (1) FLOPs: 2466.25 M PARAMs: 786.43 K
[INFO: misc.py:  268]: layer gpu_0/res4_2_branch2b (1) FLOPs: 1849.69 M PARAMs: 589.82 K
[INFO: misc.py:  268]: layer gpu_0/res4_2_branch2c (1) FLOPs: 822.08 M PARAMs: 262.14 K
[INFO: misc.py:  268]: layer gpu_0/res4_3_branch2a (1) FLOPs: 822.08 M PARAMs: 262.14 K
[INFO: misc.py:  268]: layer gpu_0/res4_3_branch2b (1) FLOPs: 1849.69 M PARAMs: 589.82 K
[INFO: misc.py:  268]: layer gpu_0/res4_3_branch2c (1) FLOPs: 822.08 M PARAMs: 262.14 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv4_3_theta (1) FLOPs: 1644.17 M PARAMs: 524.29 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv4_3_phi (1) FLOPs: 411.04 M PARAMs: 524.29 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv4_3_g (1) FLOPs: 411.04 M PARAMs: 524.29 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv4_3_affinity (1) FLOPs: 1258.82 M PARAMs: 0.00 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv4_3_y (1) FLOPs: 1258.82 M PARAMs: 0.00 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv4_3_out (1) FLOPs: 1644.17 M PARAMs: 524.29 K
[INFO: misc.py:  268]: layer gpu_0/res4_4_branch2a (1) FLOPs: 2466.25 M PARAMs: 786.43 K
[INFO: misc.py:  268]: layer gpu_0/res4_4_branch2b (1) FLOPs: 1849.69 M PARAMs: 589.82 K
[INFO: misc.py:  268]: layer gpu_0/res4_4_branch2c (1) FLOPs: 822.08 M PARAMs: 262.14 K
[INFO: misc.py:  268]: layer gpu_0/res4_5_branch2a (1) FLOPs: 822.08 M PARAMs: 262.14 K
[INFO: misc.py:  268]: layer gpu_0/res4_5_branch2b (1) FLOPs: 1849.69 M PARAMs: 589.82 K
[INFO: misc.py:  268]: layer gpu_0/res4_5_branch2c (1) FLOPs: 822.08 M PARAMs: 262.14 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv4_5_theta (1) FLOPs: 1644.17 M PARAMs: 524.29 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv4_5_phi (1) FLOPs: 411.04 M PARAMs: 524.29 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv4_5_g (1) FLOPs: 411.04 M PARAMs: 524.29 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv4_5_affinity (1) FLOPs: 1258.82 M PARAMs: 0.00 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv4_5_y (1) FLOPs: 1258.82 M PARAMs: 0.00 K
[INFO: misc.py:  268]: layer gpu_0/nonlocal_conv4_5_out (1) FLOPs: 1644.17 M PARAMs: 524.29 K
[INFO: misc.py:  268]: layer gpu_0/res5_0_branch2a (1) FLOPs: 1644.17 M PARAMs: 524.29 K
[INFO: misc.py:  268]: layer gpu_0/res5_0_branch2b (1) FLOPs: 7398.75 M PARAMs: 2359.30 K
[INFO: misc.py:  268]: layer gpu_0/res5_0_branch2c (1) FLOPs: 3288.33 M PARAMs: 1048.58 K
[INFO: misc.py:  268]: layer gpu_0/res5_0_branch1 (1) FLOPs: 6576.67 M PARAMs: 2097.15 K
[INFO: misc.py:  268]: layer gpu_0/res5_1_branch2a (1) FLOPs: 9865.00 M PARAMs: 3145.73 K
[INFO: misc.py:  268]: layer gpu_0/res5_1_branch2b (1) FLOPs: 7398.75 M PARAMs: 2359.30 K
[INFO: misc.py:  268]: layer gpu_0/res5_1_branch2c (1) FLOPs: 3288.33 M PARAMs: 1048.58 K
[INFO: misc.py:  268]: layer gpu_0/res5_2_branch2a (1) FLOPs: 3288.33 M PARAMs: 1048.58 K
[INFO: misc.py:  268]: layer gpu_0/res5_2_branch2b (1) FLOPs: 7398.75 M PARAMs: 2359.30 K
[INFO: misc.py:  268]: layer gpu_0/res5_2_branch2c (1) FLOPs: 3288.33 M PARAMs: 1048.58 K
[INFO: misc.py:  268]: layer gpu_0/pred (1) FLOPs: 0.32 M PARAMs: 321.54 K
[INFO: misc.py:  169]: Total conv/fc/matMul FLOPs: 190.646642688(e9)
[INFO: misc.py:  170]: Total conv/fc params: 34.83232(e6)
Segmentation fault
(venv) [aac12391zj@g0034 video-long-term-feature-banks]$ Process PoolWorker-92:
Traceback (most recent call last):
  File "/apps/centos7/python/2.7.18/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
    self.run()
  File "/apps/centos7/python/2.7.18/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/apps/centos7/python/2.7.18/lib/python2.7/multiprocessing/pool.py", line 122, in worker
    put((job, i, (False, wrapped)))
  File "/apps/centos7/python/2.7.18/lib/python2.7/multiprocessing/queues.py", line 390, in put
    return send(obj)
IOError: [Errno 32] Broken pipe