Is there any way to look at the output of each layer of the Model and visualize it？

[Lookforworld]

I want to look at the output of each layer of the Model and visualize it，I can do that with Torchvision like this：

          import matplotlib.pyplot as plt
          import torch
          from torchvision import models, transforms
          from torch.utils.data import DataLoader, Dataset
          import torch.nn as nn
          from torch.autograd import Variable
          import torch.nn.functional as F
          import os
          import numpy as np
          from torchvision.datasets import ImageFolder
           
          torch.cuda.set_device(0)  
          is_cuda = True
          simple_transform = transforms.Compose([transforms.Resize((224, 224)),
                                                 transforms.ToTensor(),  
                                                 transforms.Normalize([0.485, 0.456, 0.406],  # std
                                                                      [0.229, 0.224, 0.225])])
          train = ImageFolder("./datas/dogs-vs-cats/train", simple_transform)
          valid = ImageFolder("./datas/dogs-vs-cats/valid", simple_transform)
          train_loader = DataLoader(train, batch_size=1,  shuffle=False, num_workers=5)
          val_loader = DataLoader(valid, batch_size=1, shuffle=False, num_workers=5)
           
          vgg = models.vgg16(pretrained=True).cuda()
          class LayerActivations:
              features = None
              def __init__(self, model, layer_num):
                  self.hook = model[layer_num].register_forward_hook(self.hook_fn)
           
              def hook_fn(self, module, input, output):
                  self.features = output.cpu()
           
              def remove(self):
                  self.hook.remove()
           
          print(vgg.features)
           
          conv_out = LayerActivations(vgg.features, 0)  
          img = next(iter(train_loader))[0]
           
          # imshow(img)
          o = vgg(Variable(img.cuda()))
          conv_out.remove()  
          act = conv_out.features  
          
          fig = plt.figure(figsize=(20, 50))
          fig.subplots_adjust(left=0, right=1, bottom=0, top=0.8, hspace=0, wspace=0.2)
          for i in range(30):
              ax = fig.add_subplot(12, 5, i+1, xticks=[], yticks=[])
              ax.imshow(act[0][i].detach().numpy(), cmap="gray")
           
          plt.show()

What should I do with Timm？Thanks！@rwightman

[rwightman]

@Lookforworld same thing works for timm models, you just need some familiarity with each model family so you know which module names to hook. There is a much greater diversity of models in timm so it's a bit more work.

There is a utility in https://github.com/rwightman/pytorch-image-models/blob/master/timm/utils/model.py#L35 that might make it a bit easier, it was built for extracting activation stats, but you could use it for grabbing/stashing the activations as well. In addition to being able to specify each module name to hook, you can also use wildcards to grab a bunch with similar names.

You can avoid using hooks and use features_only=True mode if you happen to only want the activations that would match typical feature pryamid outputs (the deepest feature at each stride level in the network).

[Lookforworld]

@Lookforworld same thing works for timm models, you just need some familiarity with each model family so you know which module names to hook. There is a much greater diversity of models in timm so it's a bit more work.

There is a utility in https://github.com/rwightman/pytorch-image-models/blob/master/timm/utils/model.py#L35 that might make it a bit easier, it was built for extracting activation stats, but you could use it for grabbing/stashing the activations as well. In addition to being able to specify each module name to hook, you can also use wildcards to grab a bunch with similar names.

You can avoid using hooks and use features_only=True mode if you happen to only want the activations that would match typical feature pryamid outputs (the deepest feature at each stride level in the network).

@rwightman Thanks for your advice. I'll try it！