Testset in TripletMNIST and SiameseMNIST are not fixed

[luuyin]

Great work! Thanks for sharing such elegant codes.

I have a small question, in TripletMNIST(Dataset) class. your annotation about testsets is "Test: Creates fixed triplets for testing"

However, in the code it is
triplets = [[i,
random_state.choice(self.label_to_indices[self.test_labels[i].item()]),
random_state.choice(self.label_to_indices[
np.random.choice(
list(self.labels_set - set([self.test_labels[i].item()]))
)
])
]

I think it maybe should be the followings:

        triplets = [[i,
                     random_state.choice(self.label_to_indices[self.test_labels[i].item()]),
                     random_state.choice(self.label_to_indices[
                                            random_state.choice(
                                                 list(self.labels_set - set([self.test_labels[i].item()]))
                                             )
                                         ])
                     ]

I means use random_state.choice instead of np.random.choice of the original code, in order to create fixed testsets.

I am not sure If I am right, can we discuss it?

[adambielski]

Thanks @luuyin,
By fixed triplets in the test set I mean that it's fixed throughout the training, e.g. they are randomly chosen at the beginning of the training, and after each epoch we evaluate on the same triplets - this is done regardless of using random_state or np.random
But you're right that they should be properly conditioned on random_state, I'll fix that later. Thanks!