Moons dataset looks weird

[kennysong]

Nice article! The image of the moons dataset at the end of the article looks a bit strange since the points are heavily overlapping and not in a moon shape.

Is it the correct image?

[apoorvagnihotri]

Thanks a lot for the appreciation. :)

Yes, it is the correct image, though we have used a variance of 1 while creating the dataset. You can find the code used for replicating the above image here.

[kennysong]

Thanks for the reference! Was the reason for the high noise just to make the task harder and spread out the performance of different hyperparameter sets?

[apoorvagnihotri]

Hi @kennysong. To be honest, we didn't consider varying the noise. But I think your idea should be valid in this case.

I did some experiments with low noise and Bayesian Optimization (BO) is performing better. PFA, moon's data results on SVM.

New, simple moon’s dataset.

New accuracies on using an SVM.

Old accuracy on using an SVN.

---

New accuracies on using an RF.

Old accuracies on using an RF.

Our task is much difficult--because of using a very high noise in dataset--in the original article. Therefore, it was difficult for the BO to perform. As seen here, reducing the complexity helped BO achieve better results.

You can have a look at the ipynb used here.

Thanks a lot for pointing this out. I’ll keep this issue open as I think this is an important point. :)

[kennysong]

Oh cool! It does look like the original harder task spread out the performance of different algorithms more. With less noise, all approaches seem to do better and converge to the global optimum.

Thanks for following up!