notebooks/4-Unsupervised-2-Others.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4.2 Other Unsupervised Learning Techniques\n",
    "\n",
    "## Objectives:\n",
    "1. Understand unsupervised learning techniques beyond clustering.\n",
    "2. Learn about dimensionality reduction methods such as PCA.\n",
    "3. Explore anomaly detection techniques.\n",
    "4. Hands-on: Apply Principal Component Analysis (PCA) to a dataset.\n",
    "\n",
    "By the end of this session, you will have a deeper understanding of unsupervised learning methods and their applications.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 🌟 Dimensionality Reduction\n",
    "\n",
    "### What is Dimensionality Reduction?\n",
    "Dimensionality reduction reduces the number of input features while retaining the essential patterns in the data. It is useful for:\n",
    "- Simplifying datasets.\n",
    "- Visualizing high-dimensional data.\n",
    "- Speeding up computations.\n",
    "\n",
    "---\n",
    "\n",
    "### Principal Component Analysis (PCA)\n",
    "PCA is one of the most popular dimensionality reduction techniques.\n",
    "\n",
    "---\n",
    "\n",
    "### Key Steps:\n",
    "1. Standardize the dataset (mean = 0, variance = 1).\n",
    "2. Compute the covariance matrix.\n",
    "3. Identify the principal components (eigenvectors of the covariance matrix).\n",
    "4. Project the data onto the new subspace defined by the top principal components.\n",
    "\n",
    "---\n",
    "\n",
    "### Applications:\n",
    "1. Image Compression.\n",
    "2. Reducing features for faster model training.\n",
    "3. Visualizing high-dimensional data (e.g., reducing 100 features to 2).\n",
    "\n",
    "---\n",
    "\n",
    "### Hands-On Example: PCA\n",
    "We will implement PCA on a synthetic dataset later in this section.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hands-On: Dimensionality Reduction with PCA\n",
    "Let's apply PCA to a dataset to reduce its dimensionality on a synthetic dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import required libraries\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.decomposition import PCA\n",
    "from sklearn.datasets import make_classification\n",
    "from sklearn.preprocessing import StandardScaler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 1. Generate Synthetic Data\n",
    "X, y = make_classification(n_samples=200, n_features=10, random_state=42)\n",
    "\n",
    "# Standardize the data\n",
    "scaler = StandardScaler()\n",
    "X_scaled = scaler.fit_transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 2. Apply PCA\n",
    "pca = PCA(n_components=2)  # Reduce to 2 dimensions\n",
    "X_pca = pca.fit_transform(X_scaled)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 3. Visualize the Results\n",
    "plt.scatter(X_pca[:, 0], X_pca[:, 1], c=y, cmap='viridis', label='Data Points')\n",
    "plt.title('PCA Projection to 2D')\n",
    "plt.xlabel('Principal Component 1')\n",
    "plt.ylabel('Principal Component 2')\n",
    "plt.colorbar(label='Class')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 4. Explained Variance\n",
    "explained_variance = pca.explained_variance_ratio_\n",
    "print(f\"Explained Variance by Each Component: {explained_variance}\")\n",
    "print(f\"Total Explained Variance: {sum(explained_variance):.2f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NOTE**: The explained variance ratio tells us how much information is preserved by each principal component. We can use this information to decide how many principal components to keep. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# increase the number of components to 5\n",
    "pca = PCA(n_components=5)\n",
    "X_pca = pca.fit_transform(X_scaled)\n",
    "explained_variance = pca.explained_variance_ratio_\n",
    "print(f\"Explained Variance by Each Component: {explained_variance}\")\n",
    "print(f\"Total Explained Variance: {sum(explained_variance):.2f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# increase the number of components to 10\n",
    "pca = PCA(n_components=7)\n",
    "X_pca = pca.fit_transform(X_scaled)\n",
    "explained_variance = pca.explained_variance_ratio_\n",
    "print(f\"Explained Variance by Each Component: {explained_variance}\")\n",
    "print(f\"Total Explained Variance: {sum(explained_variance):.2f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 🌟 Anomaly Detection\n",
    "\n",
    "### What is Anomaly Detection?\n",
    "Anomaly detection identifies data points that deviate significantly from the normal patterns. It is often used for:\n",
    "- Fraud detection.\n",
    "- Monitoring network traffic for intrusions.\n",
    "- Detecting manufacturing defects.\n",
    "\n",
    "---\n",
    "\n",
    "### Techniques:\n",
    "1. **Density-Based Methods**:\n",
    "   - DBSCAN (clustering-based).\n",
    "   - Isolation Forest (tree-based).\n",
    "2. **Statistical Methods**:\n",
    "   - Z-scores: Data points with high Z-scores (e.g., > 3) are anomalies.\n",
    "\n",
    "---\n",
    "\n",
    "### Applications:\n",
    "1. Detecting fraudulent transactions in banking.\n",
    "2. Identifying faulty sensors in IoT systems.\n",
    "3. Finding outliers in customer behavior data.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hands-On: Anomaly Detection\n",
    "\n",
    "We will implement anomaly detection using the Isolation Forest algorithm on a synthetic dataset. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import required libraries\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.ensemble import IsolationForest\n",
    "from sklearn.datasets import make_blobs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 1. Generate Synthetic Data\n",
    "X, _ = make_blobs(n_samples=300, centers=1, cluster_std=1.0, random_state=42)\n",
    "outliers = np.random.uniform(low=-6, high=6, size=(20, 2))  # Add some outliers\n",
    "X_with_outliers = np.vstack([X, outliers])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Visualize the data with outliers\n",
    "plt.scatter(X_with_outliers[:, 0], X_with_outliers[:, 1], color='blue', label='Data Points')\n",
    "plt.title('Data with Outliers')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 2. Apply Isolation Forest for Anomaly Detection\n",
    "iso_forest = IsolationForest(contamination=0.05, random_state=42)  # Assume 5% contamination\n",
    "predictions = iso_forest.fit_predict(X_with_outliers)  # -1: Anomaly, 1: Normal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Separate anomalies and normal points\n",
    "normal = X_with_outliers[predictions == 1]\n",
    "anomalies = X_with_outliers[predictions == -1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Visualize Results\n",
    "plt.scatter(normal[:, 0], normal[:, 1], color='blue', label='Normal Points')\n",
    "plt.scatter(anomalies[:, 0], anomalies[:, 1], color='red', label='Anomalies', edgecolor='k')\n",
    "plt.title('Anomaly Detection with Isolation Forest')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 📝 Key Takeaways:\n",
    "1. Dimensionality reduction techniques like PCA help simplify data while preserving patterns.\n",
    "2. Anomaly detection is used to identify unusual data points in applications like fraud detection.\n",
    "3. Each of these techniques has distinct use cases in real-world scenarios."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}