Intriguing Properties of Contrastive Losses paper

Re-implementation of Intriguing Properties of Contrastive Losses paper.

Getting Started

• Download imagenette here (click Full Size download)

Environment

Anaconda

conda create -n re-intriguing-properties pip
conda activate re-intriguing-properties
conda install -c anaconda jupyter
pip install -r requirements.txt

Install the package

pip install -e .

Experiments

Linear Evaluation of SimCLR

simclr-lineval.ipynb performs the linear evaluation of SimCLR with specified number of epochs, batch size, and number of layers of the projection head.

SimCLR learns local features that exhibit hierarchical properties

• raw_pixels.ipynb visualizes the clustered pixels of raw input images.
• simclr.ipynb extracts features from block group 1, 2, 3, and 4 of SimCLR's base encoder. The extracted features can be then used in the following two Jupyter notebooks:
  • simclr_different_blocks.ipynb shows SimCLR's learned features on single input images and that the learning of local features is also achieved with other clustering methods.
  • simclr_different_blocks_batch.ipynb shows SimCLR's learned features on batches of input images.

Generating additional data

scripts/data_generation/ contains Jupyter notebooks that reproduce the construction of two out of three datasets of the original paper with explicit and controllable competing features that can be used to reproduce the other experiments of the original work that we did not replicate.