Question about methods Cosine and Capsule in the paper

[yypurpose]

Hi @KaihuaTang , thank you for doing such an inspiring job and opening the source code. Would you mind telling some details of the methods Cosine and Capsule in the Table 2 of the paper.

1. I got some confused because I don't know the cosine similarity is used in the training phase or just test phase. I got no performance improvement when using it in the training phase for both cifar100-LT and ImageNet-LT.

2. For Capsule, I also don't know how to do it.

Thanks a lot!

[KaihuaTang]

As far as I know, the Cosine Classifier is firstly used by https://github.com/zhmiao/OpenLongTailRecognition-OLTR in the Long-Tailed Classification field. If your cosine classifier didn't improve the performance, it probably because you didn't add a scale parameter to the logits (16.0 in OLTR and 16.0 / Num_Head in our project). The scale parameter will change the distribution after the softmax activation, and thus accelerating the gradient descent.

[KaihuaTang]

As to the Capsule Classifier, it just replaces the cosine norm with another normalization introduced in the https://arxiv.org/abs/1710.09829

[yypurpose]

As far as I know, the Cosine Classifier is firstly used by https://github.com/zhmiao/OpenLongTailRecognition-OLTR in the Long-Tailed Classification field. If your cosine classifier didn't improve the performance, it probably because you didn't add a scale parameter to the logits (16.0 in OLTR and 16.0 / Num_Head in our project). The scale parameter will change the distribution after the softmax activation, and thus accelerating the gradient descent.

Thank you for your prompt reply! I added a scale actually, but may not be the good one. Anyway, I'll try again.
Thanks a lot!