backwardProp函数

[cong-code]

您好！作者，我想问一下，为什么我重新编译后，发现不能调用backwardProp()函数

[LiuLiluZJU]

backwardProp()是用在另一个项目里的，用于可微渲染和三位重建，这个项目用不到。需要在pytorch里把上游梯度算出来传入这个函数，代码示例如下。个人建议有相关需求可以参考NeRF系列

# Projection Process AP
class ProjectionProcessAP(Function):
  @staticmethod
  def forward(ctx, movImgArray_Gpu):
    # load files
    DRRPhy_array = np.load('/root/X2CT-master/3DGAN/lib/model/loss/DRRPhy_array_AP_256_728699_big_inverse.npy')
    source_forGpu = np.load('/root/X2CT-master/3DGAN/lib/model/loss/source_forGpu_AP_256_728699_big_inverse.npy')

    # preprocess CT
    movImgArray_Gpu = movImgArray_Gpu.detach()
    movImgArray = movImgArray_Gpu.data.clone().cpu().numpy()
    movImgArray = movImgArray[:, ::-1, :, :]
    movImgArray = movImgArray * 2500
    batch = np.size(movImgArray, 0)
    output_final = []

    for x in range(batch):
      # extract ct
      movImgArray_x = movImgArray[x]
      movImgArray_x_1d = np.ravel(movImgArray_x, order = 'C')

      # define projector
      projector = pySiddonGpu(np.array([16, 16, 1], dtype=np.int32),
                              movImgArray_x_1d.astype(np.float32),
                              np.array([128, 128, 128], dtype=np.int32),
                              np.array([350 / 128, 350 / 128, 350 / 128], dtype=np.float32),
                              np.float32(-350 / 256), np.float32(-350 / 256), np.float32(-350 / 256),
                              np.array([512, 512, 1], dtype=np.int32))

      # generate DRR
      output = projector.generateDRR(source_forGpu, DRRPhy_array)
      projector.computeMetric(np.asarray(output))  # Free DRR GPU memory
      projector.delete()  # Destruct object, free GPU memory
      output_reshaped = np.reshape(output, (512, 512), order='C')
      output_reshaped = (output_reshaped - np.min(output_reshaped)) / (np.max(output_reshaped) - np.min(output_reshaped))  # Normalization
      output_final.append(torch.Tensor(output_reshaped))
    
    return torch.stack(output_final).cuda()

  @staticmethod
  def backward(ctx, grad_upstream):
    # load files
    DRRPhy_array = np.load('/root/X2CT-master/3DGAN/lib/model/loss/DRRPhy_array_AP_256_728699_big_inverse.npy')
    source_forGpu = np.load('/root/X2CT-master/3DGAN/lib/model/loss/source_forGpu_AP_256_728699_big_inverse.npy')
    
    # preprocess upstream grad
    grad_upstream = grad_upstream.detach()
    grad_upstream_array = grad_upstream.cpu().numpy()
    batch = np.size(grad_upstream_array, 0)
    grad_downstream = []
    
    for x in range(batch):
      # extract grad
      grad_upstream_x = grad_upstream_array[x]
      grad_upstream_x_1d = np.ravel(grad_upstream_x, order='C').astype(np.float32)
      projector = pySiddonGpu(np.array([16, 16, 1], dtype=np.int32),
                              np.zeros(128 * 128 * 128, dtype=np.float32),  # zero matrix save output gradient
                              np.array([128, 128, 128], dtype=np.int32),
                              np.array([350 / 128, 350 / 128, 350 / 128], dtype=np.float32),
                              np.float32(-350 / 256), np.float32(-350 / 256), np.float32(-350 / 256),
                              np.array([512, 512, 1], dtype=np.int32))
      
      output = projector.backwardProp(source_forGpu, DRRPhy_array, grad_upstream_x_1d)
      projector.delete()  # Destruct object, free GPU memory
      output_reshaped = np.reshape(output, (128, 128, 128), order='C')
      output_reshaped = output_reshaped[::-1, :, :]
      output_reshaped = output_reshaped.copy()
      grad_downstream.append(torch.Tensor(output_reshaped))
    
    return torch.stack(grad_downstream).cuda()

[cong-code]

我想用在三维重建上，这个函数是可以直接从DRR及对应的位姿参数重建CT？传入的梯度这个参数代表什么？

[LiuLiluZJU]

这个函数是可微渲染的梯度反传部分，不是直接用于三维重建的函数，传入的梯度是上游传进来的梯度，不需要自己给，执行loss.backward()会自动计算。关于X光到CT的三维重建我建议看这篇文章：https://openaccess.thecvf.com/content_CVPR_2019/papers/Ying_X2CT-GAN_Reconstructing_CT_From_Biplanar_X-Rays_With_Generative_Adversarial_Networks_CVPR_2019_paper.pdf 你也可以看看我们的一个小工作，基于重建的配准，我们加了可微渲染loss，但实际上对重建质量的提升不算大 https://www.mdpi.com/2072-666X/12/7/844

[cong-code]

好的谢谢作者，但是我想做的是直接简单基于传统的方式反投影到三维（较粗糙的也行），而不需要基于深度学习

[cong-code]

XraySyn: Realistic View Synthesis From a Single Radiograph Through CT Priors类似这篇文章的前向投影和反向投影，不知道作者编译的这个.so的backwardProp可否直接反投影，我试了一下好像可以直接从一张DRR反投影到3D：
projector = pySiddonGpu(initial_params['NumThreadsPerBlock'],
np.ravel(pre_ct_array.copy(), order='C').astype(np.float32),
initial_params['MovSize_forGpu'],
initial_params['MovSpacing_forGpu'],
initial_params['SidePlanes'][0].astype(np.float32),
initial_params['SidePlanes'][1].astype(np.float32),
initial_params['SidePlanes'][2].astype(np.float32),
initial_params['DRRsize_forGpu'])

    grad_upstream = np.ones([128, 128])
    grad_upstream = np.ravel(grad_upstream.copy(), order='C').astype(np.float32)
    output = projector.backwardProp(initial_params['source'].astype(np.float32),
                                    initial_params['sourceDRR_array_1d'],
                                    grad_upstream)

projector.delete() # Destruct object, free GPU memory
output_reshaped = np.reshape(output, (128, 128, 128), order='C')
output_reshaped = output_reshaped[::-1, :, :]
output_reshaped = output_reshaped.copy()

[LiuLiluZJU]

不一样，你得去看看CBCT的反投影原理

[cong-code]

好的，谢谢！