Can TorchLens Be Used to Compare the Representational Spaces of Deep Nets and Human Brain Recordings?

[batiyilmaz]

Hello,

First of all, I would like to thank you so much for providing such an excellent toolbox and making it publicly available.

I am interested in learning how to implement TorchLens for the purpose of comparing different pre-trained deep neural networks' performance to brain activation data (e.g., M/EEG, fMRI). Is it possible and feasible to do using TorchLens, similar to Net2Brain (Bersch et al., 2022)? And if it is the case, where should I start learning about this more?

Thank you so much for your time and help in advance.

Best,
Bati

References
Net2Brain: A Toolbox to compare artificial vision models with human brain responses - https://arxiv.org/abs/2208.09677

[johnmarktaylor91]

Hello—sorry for the slow response on this, for some reason I don't get notifications for discussion posts! So, TorchLens is meant to be a total solution for extracting the activations from neural networks, but it doesn't have any functionality for analyzing the extracted activations. This is partially because different folks might have different use cases, and I'm not smart enough to predict all of these in advance, but also because there are already some great packages for analyzing DNN activations and comparing them to brain activity. For example, if you are doing representational similarity analysis, then rsatoolbox has pretty much all the functionality you'll ever need. So, your pipeline could involve extracting the activations with torchlens, and then passing these to rsatoolbox for subsequent analysis. Lemme know if you have any more specific questions, happy to help.