How to Overcome Curse-of-Dimensionality for OOD Detection

Preliminaries

It is tested under Ubuntu Linux 20.04 and Python 3.9 environment, and requries following packages to be installed:

• PyTorch
• scipy
• numpy
• sklearn
• faiss

Usage

1. Dataset Preparation for CIFAR Experiment

In-distribution dataset

The downloading process will start immediately upon running.

Out-of-distribution dataset

We provide links and instructions to download each dataset:

• Textures: download it and place it in the folder of ./ood_data/dtd.
• Places365: download it and place it in the folder of ./ood_data/Places365. We sample 10,000 images from the original test dataset. Download the sampled dataset from here.
• LSUN: download it and place it in the folder of ./ood_data/LSUN.
• iSUN: download it and place it in the folder of ./ood_data/iSUN.

Pre-trained model

Please download Pre-trained models and place in the ./checkpoints folder for respective ID dataset and model architecture. For example: DenseNet-101 model trained on CIFAR-10 should be placed in ./checkpoints/CIFAR-10/densenet folder.

Demo

1. Demo code for training SNN on CIFAR benchmark

To train DenseNet-101 on CIFAR-100 dataset with subspace learning, run the following command:

python train_densenet.py --id CIFAR-100 --bs 64 --r 0.25

Run the following command, to train ResNet-50 on CIFAR-100 dataset:

python train_resnet.py --id CIFAR-100 --bs 128 --r 0.05

2. Demo code for testing SNN on CIFAR benchmark

For inference, download the pre-trained models as mentioned above. To evaluate the OOD detection performance for a DenseNet model trained on CIFAR-100, run the following the command:

python test_cifar.py --in-dataset CIFAR-100 --model_arch densenet --bs 200

To run inference on a trained ResNet-50 model, run the following command:

python test_cifar.py --in-dataset CIFAR-100 --model_arch resnet50 --bs 200

2. Dataset Preparation for Large-scale Experiment (ImageNet-100)

In-distribution dataset

For generating the ImageNet-100 dataset, please follow the steps given here and save the dataset in ./IN-100 folder.

Out-of-distribution dataset

We have 4 OOD datasets from iNaturalist, SUN, Places, and Textures, and de-duplicated concepts overlapped with ImageNet-1k.

For iNaturalist, SUN, and Places, we have sampled 10,000 images from the selected concepts for each dataset, which can be download via the following links:

wget http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/iNaturalist.tar.gz
wget http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/SUN.tar.gz
wget http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/Places.tar.gz

For Textures, we use the entire dataset, which can be downloaded from their original website.

Please put all downloaded OOD datasets into ./ood_data.

1. Demo code for training SNN on Imagenet-100 benchmark

To train ResNet-101 on ImageNet-100 dataset with subspace learning, run the following command:

python train_imagenet.py --classes 100 --r 0.35

2. Demo code for testing SNN on Imagenet-100 benchmark

The pretrained model can be downloaded from here. After downloading, place the pre-trained model in the ./checkpoints folder. For inference on ImageNet-100, run the following command:

python feat_extract_largescale.py

References

The codebase is adapted from knn-ood.