Topic recognition

README.md

@@ -1,15 +1,55 @@
-# Pattern Analysis
-Pattern Analysis of various datasets by COMP3710 students at the University of Queensland.
+# Description
+
+This contains methods to create embeddings via a Triplet Siamese Network, which can then be used to create a classifier for Alzheimer's disease using the ADNI dataset. The model attempts to identify Alzheimer's from MRIs, which would be very helpful for radiology.
+
+# Algorithm
+
+The triplet Siamese model works by taking in both a positive and negative input, as well as an anchor. An identical backbone (in this case, resnet) is used on all three inputs to produce an embedded space. The aim of the model is to make the distance between the positive and anchor less than the distance between the negative and the anchor, without knowing which camp the anchor belongs to. The lost function used to achieve this subtracts the negative distance from the positive to ensure a lower loss when closer to positive. This creates a good embedded space to then use to classify inputs. Using this, individual outputs from the embedded space can be inputted into a Random Forest Classifier, which can then be used to identify Alzheimers.
+
+# Pre-Processing
+
+Training Images are randomly rotated between -5 and 5 degrees, offset between -5% and 5% and scaled between 95% and 105%. They are then intensity normalized. After that, the mean and standard deviation is found for the entire training set, and both the training and testing sets are normalized using that mean and standard deviation. Finally, the training set is split into two, with 80% being used for the Triplet Siamese Model and the remaining 20% for the classifier.
+
+# Triplet Siamese Model
+
+Each epoch, each batch is randomly sorted to have a different triplet each time. The inputs then undergo the rotation, offset and scaling transform from pre-processing to keep the model on it's toes. The loss function is calculated using a tiny amount of L2 regularization, 0.0001. This helps with the generalization of the model.
+
+The learning rate begins very low at 0.0001, gradually increases to a normal level of low, 0.001, which is reached halfway through training. After the peak is reached, the learning rate decreased at the same rate back to its starting level.
+
+The optimizer used is a Scholastic Gradient Descent optimizer. When looking into the optimizer I discovered that weight decay and L2 regularization are the same thing, so actually 0.0005 is also used.
+
+# Classification Model
+
+The embedded space used for the three inputs in the Triplet Siamese Model is then used inside another model to classify individual inputs. The classification model is a Random Forest model, with 600 estimators, 250 min split and max depth of 12. This allows for more generalization and beats Neural Networks by quite a margin.
+
+# Dependencies
+
+Python version: 3.10.12
+
+Pytorch version: 2.1.0, Cuda 11.8
+
+Numpy version: 1.23.5
+
+Matplotlib version: 3.7.1
+
+Sklearn version: 1.2.2
+
+PIL version: 9.4.0
+
+# Inputs and Outputs
+
+![image](https://raw.githubusercontent.com/FinnRobertson15/PatternAnalysis-2023/topic-recognition/inputs.png)
+
+Top: 1 (Alzheimer's)
+
+Bottom: 0 (No Alzheimer's)
+
+# Results
+
+![image](https://raw.githubusercontent.com/FinnRobertson15/PatternAnalysis-2023/topic-recognition/PCA.png)
+
+Triplet loss graph
+
+![image](https://raw.githubusercontent.com/FinnRobertson15/PatternAnalysis-2023/topic-recognition/train.png)
 
-We create pattern recognition and image processing library for Tensorflow (TF), PyTorch or JAX.
-
-This library is created and maintained by The University of Queensland [COMP3710](https://my.uq.edu.au/programs-courses/course.html?course_code=comp3710) students.
-
-The library includes the following implemented in Tensorflow:
-* fractals 
-* recognition problems
-
-In the recognition folder, you will find many recognition problems solved including:
-* OASIS brain segmentation
-* Classification
-etc.
+Classification Accuracy of 65%

dataset.py

@@ -0,0 +1,120 @@
+import torch
+
+import torchvision.transforms as transforms
+
+import os
+from PIL import Image
+from torch.utils.data import Dataset, DataLoader
+import torch
+
+class CustomDataset(Dataset):
+    def __init__(self, root_dir, transform=None):
+        self.root_dir = root_dir
+        self.transform = transform
+        self.image_paths = os.listdir(root_dir)
+
+    def __len__(self):
+        return len(self.image_paths)
+
+    def __getitem__(self, idx):
+        img_name = os.path.join(self.root_dir, self.image_paths[idx])
+        image = Image.open(img_name)
+
+        if self.transform:
+            image = self.transform(image)
+
+        return image
+
+class TripletDataset(Dataset):
+    def __init__(self, AD, NC, transform=None):
+        self.X = AD + NC
+        self.AD = AD
+        self.NC = NC
+        self.Y = torch.cat((torch.ones(len(AD)), torch.zeros(len(NC))), dim=0)
+        self.anc_indices = torch.randperm(len(self.X))
+        self.pos_indices = torch.randperm(len(self.X)) % len(AD)
+        self.neg_indices = torch.randperm(len(self.X)) % len(NC)
+        self.transform = transform
+
+    def __len__(self):
+        return len(self.anc_indices)
+
+    def __getitem__(self, idx):
+        anc = self.anc_indices[idx]
+        pos = self.pos_indices[idx]
+        neg = self.neg_indices[idx]
+        img1 = self.X[anc]
+        img2 = self.AD[pos]
+        img3 = self.NC[neg]
+        label = self.Y[anc]
+
+        if self.transform:
+            img1 = self.transform(img1)
+            img2 = self.transform(img2)
+            img3 = self.transform(img3)
+
+        return img1, img2, img3, torch.tensor([1 - label, label])
+
+
+
+
+def intensity_normalization(img, mean = None, std = None):
+    mean = torch.mean(img)
+    std = torch.std(img)
+    return (img - mean) / std
+
+class CustomNormalize(object):
+    def __init__(self, mean, std):
+        self.mean = mean
+        self.std = std
+
+    def __call__(self, img):
+        return (img - self.mean) / self.std
+
+transform_train = transforms.Compose([
+    transforms.Resize((256, 256)),
+    transforms.ToTensor(),
+    transforms.RandomAffine(degrees=5, translate=(0.05, 0.05), scale=(0.95, 1.05)),  # Random affine transformations with smaller parameters
+    transforms.Lambda(intensity_normalization)
+])
+
+transform_test = transforms.Compose([
+    transforms.Resize((256, 256)),
+    transforms.ToTensor(),
+    transforms.Lambda(intensity_normalization)
+])
+
+class Normalize(object):
+    def __init__(self, mean, std):
+        self.mean = mean
+        self.std = std
+
+    def __call__(self, img):
+        return (img - self.mean) / self.std
+
+
+
+def gen_loaders(root_dir = '', batch_size = 96):
+    loaders = {}
+    AD_train = CustomDataset(root_dir=root_dir + os.path.join('train', 'AD'), transform=transform_train)
+    NC_train = CustomDataset(root_dir=root_dir + os.path.join('train', 'NC'), transform=transform_train)
+
+    X = torch.stack([img for img in AD_train + NC_train])
+    mean = X.mean()
+    std = X.std()
+    normalize = transforms.Compose([
+            Normalize(mean, std)
+        ])
+    train_dataset = TripletDataset(AD_train, NC_train, normalize)
+
+
+    # Create DataLoaders for the two parts
+    loaders['train'] = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)
+
+    AD_test = CustomDataset(root_dir=os.path.join('test', 'AD'), transform=transform_test)
+    NC_test = CustomDataset(root_dir=os.path.join('test', 'NC'), transform=transform_test)
+
+    test_dataset = TripletDataset(AD_test, NC_test, normalize)
+
+    loaders['test'] = DataLoader(test_dataset, batch_size=batch_size, shuffle=False)
+    return loaders

modules.py

@@ -0,0 +1,60 @@
+
+import torch
+import torch.nn as nn
+import torchvision.models as models
+import torch.nn.functional as F
+
+class TripletSiameseNetwork(nn.Module):
+    def __init__(self, pretrained=True):
+        super(TripletSiameseNetwork, self).__init__()
+        self.resnet = models.resnet18(pretrained=pretrained)
+        self.resnet.conv1 = nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3, bias=False)
+        self.bn = nn.BatchNorm1d(1000)
+        self.fc1 = nn.Linear(1000, 128)
+
+    def forward_once(self, x):
+        output = self.resnet(x)
+        output = self.bn(output)
+        output = self.fc1(output)
+        return output
+
+    def forward(self, anchor, positive, negative):
+        output_anchor = self.forward_once(anchor)
+        output_positive = self.forward_once(positive)
+        output_negative = self.forward_once(negative)
+        return output_anchor, output_positive, output_negative
+
+class TripletLoss(nn.Module):
+    def __init__(self, margin=1.0):
+        super(TripletLoss, self).__init__()
+        self.margin = margin
+
+    def forward(self, anchor, positive, negative):
+        distance_positive = F.pairwise_distance(anchor, positive)
+        distance_negative = F.pairwise_distance(anchor, negative)
+        losses = F.relu(distance_positive - distance_negative + self.margin)
+        return losses.mean()
+
+class TripletLossWithRegularization(nn.Module):
+    def __init__(self, model, margin=1.0, lambda_reg=0.0001):
+        super(TripletLossWithRegularization, self).__init__()
+        self.model = model
+        self.margin = margin
+        self.lambda_reg = lambda_reg  # Regularization parameter
+
+    def forward(self, anchor, positive, negative):
+        distance_positive = F.pairwise_distance(anchor, positive)
+        distance_negative = F.pairwise_distance(anchor, negative)
+        triplet_losses = F.relu(distance_positive - distance_negative + self.margin)
+        triplet_loss = triplet_losses.mean()
+
+        # Compute L2 regularization
+        l2_reg = None
+        for param in self.model.parameters():
+            if l2_reg is None:
+                l2_reg = param.norm(2)
+            else:
+                l2_reg = l2_reg + param.norm(2)
+
+        loss = triplet_loss + self.lambda_reg * l2_reg
+        return loss

predict.py

@@ -0,0 +1,26 @@
+from modules import TripletSiameseNetwork
+from dataset import gen_loaders
+import torch
+import pickle
+
+def predict(input=gen_loaders()['test'], triplet_path = 'trip_model.pth', classifier_path='classifier.pickle'):
+    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+    trip_model = TripletSiameseNetwork()
+    trip_model = torch.load(triplet_path)
+    with open(classifier_path, 'rb') as f:
+        clf = pickle.load(f)
+    outputs = None
+
+    for i, (img, _, _, _) in enumerate(input):
+        img = img.to(device)
+        with torch.no_grad():
+            features = trip_model.forward_once(img)
+
+        if outputs is None:
+            outputs = features.cpu()
+        else:
+            outputs = torch.cat((outputs, features.cpu()), dim=0)
+
+    pred = clf.predict(outputs)
+    return pred
+