Skip to content

Anomaly Detection Study

Hitansh Singla edited this page Jun 19, 2019 · 3 revisions

To find anomalies without any supervision is not exactly an easy task.

We need to ask our computer to do this without any single label:

image

Source: Stanford's own result with supervised learning

However, after carefully examining a lot of supervised algorithms and a lot of different segmentation techniques, we came up with one solution that seems to works (sometimes).

To detect the anomalies:

  1. We first loaded the image into grayscale format:

image

  1. Then we factorise the image as a product of weight and influence by using Non negative matrix factorisation:

image

This image above is reconstructed by using 50 features. Reducing the features reduces the sensitivity of algorithms.

10 features reconstruction:

image

  1. Now, we analyse the weight matrix:

image

We can see that there are some anomalies at the bottom. We find these anomalies(or outliers) by using the DBSCAN algorithm.

Similarly, we analysed the influence matrix.

After combining both weight and influence and detecting the anomalies, we came up with the following result:

image

On the right, there is the original image. On the left, you can see the portions cut by the algorithm where it felt that something is not right.