L3/W4/lecture_note.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model Performance Analysis\n",
    "\n",
    "#### 1. **Model Performance Analysis**\n",
    "   - **Key Concepts**:\n",
    "     - **Black-box Evaluation**: Focuses on quantifying the model's performance via metrics and losses, without considering the internal structure.\n",
    "     - **Model Introspection**: Investigates the internal workings of the model, such as the data flow within each layer. Useful for understanding and improving model architecture.\n",
    "     - **TensorBoard**: A key tool for monitoring training, loss, and accuracy metrics in black-box evaluation. It also helps track optimization objectives like loss during training.\n",
    "\n",
    "   - **Analyzing Model Performance**:\n",
    "     - Analyze performance not just on the entire dataset, but also on **individual slices** (subsets of data). Slicing should be guided by domain knowledge.\n",
    "     - Consider factors like **data drift** and **concept drift**—changes in data distribution that occur over time.\n",
    "     - **Example**: For a shoe demand prediction model, evaluate performance on different types (e.g., color, style) to detect areas for improvement.\n",
    "\n",
    "   - **Optimization Landscape**:\n",
    "     - **Loss Surface**: Visual representation of the loss function as a function of model parameters.\n",
    "     - **Saddle Points**: Points in the landscape where one dimension curves up and another down—common in many optimization problems.\n",
    "     - Different optimization algorithms navigate the loss surface in unique ways, leading to varying convergence speeds.\n",
    "\n",
    "#### 2. **Model Debugging and Understanding Robustness**\n",
    "   - **Debugging**:\n",
    "     - Use model introspection to identify issues in layer-wise operations and data handling.\n",
    "     - Techniques like **class activation maps (CAM)** can help visualize which parts of the input data influence predictions.\n",
    "\n",
    "   - **Robustness and Sensitivity**:\n",
    "     - **Sensitivity Analysis**: Evaluates how changes in input affect model output. Crucial for understanding model stability.\n",
    "     - **Adversarial Attacks**: Small, often imperceptible, changes to input data that can lead to incorrect predictions. Analyzing robustness helps mitigate these vulnerabilities.\n",
    "\n",
    "#### 3. **Model Remediation**\n",
    "   - If performance degradation is detected, remediation strategies can include:\n",
    "     - Retraining with more relevant data.\n",
    "     - Fine-tuning model parameters.\n",
    "     - Adjusting hyperparameters.\n",
    "     - Employing advanced optimization techniques (e.g., SGD, Adam).\n",
    "     - **Transfer learning** to improve performance in data-limited scenarios.\n",
    "\n",
    "#### 4. **Fairness in ML**\n",
    "   - **Importance of Fairness**: Ensuring that the model performs equitably across different user groups, without bias.\n",
    "   - **Techniques for Fairness**:\n",
    "     - **Bias Detection**: Analyze model performance across different demographic slices.\n",
    "     - **Fairness Constraints**: Include fairness objectives during model training (e.g., equalized odds).\n",
    "     - Regular auditing and performance monitoring to detect any bias introduced over time.\n",
    "\n",
    "#### 5. **Continuous Evaluation and Monitoring**\n",
    "   - **Key Concepts**:\n",
    "     - After deployment, regularly evaluate model performance in production environments.\n",
    "     - **Performance Decay**: Monitor for decay due to changing data distributions and retrain when necessary.\n",
    "     - Use tools like **TFMA** for continuous monitoring of production models.\n",
    "  \n",
    "   - **When to Retrain**:\n",
    "     - Retrain when significant performance drops are detected.\n",
    "     - Consider retraining based on time (e.g., periodically) or triggers (e.g., performance metrics falling below a threshold).\n",
    "\n",
    "#### 6. **Key Tools and Technologies**\n",
    "   - **TensorFlow Model Analysis (TFMA)**: Tool for model evaluation in production, especially for data slices. Helps track performance on critical subpopulations.\n",
    "   - **TensorBoard**: For visualizing metrics during training.\n",
    "   - **Class Activation Maps (CAM)**: A visualization technique that shows the important regions in the input for predictions.\n",
    "\n",
    "### Short Notes:\n",
    "   - **Black-box evaluation**: Focuses on metrics without inspecting internal layers.\n",
    "   - **Model introspection**: Understand internal workings of layers; useful for improving architecture.\n",
    "   - **Slicing data**: Analyzing model performance on subsets of data.\n",
    "   - **Optimization landscape**: Loss surface; saddle points; different optimizers.\n",
    "   - **Debugging**: Introspection + tools like CAM.\n",
    "   - **Robustness & sensitivity**: Protect against adversarial attacks.\n",
    "   - **Remediation**: Retrain, tune, transfer learning, hyperparameters.\n",
    "   - **Fairness**: Bias detection, fairness constraints, ongoing audits.\n",
    "   - **Continuous evaluation**: Monitor performance decay; retrain as needed.\n",
    "   - **Tools**: TFMA, TensorBoard, CAM.\n",
    "\n",
    "This summary covers the key aspects of model performance analysis, debugging, robustness, fairness, and continuous evaluation for production ML."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to TensorFlow Model Analysis\n",
    "\n",
    "#### 1. **Introduction to TFMA**\n",
    "   - **Purpose**: TensorFlow Model Analysis (TFMA) is a versatile, open-source tool for conducting deep analysis of model performance, especially by evaluating performance across different data slices.\n",
    "   - **Use Cases**: Identifying performance issues in specific subsets of data that top-level metrics might overlook (e.g., specific customer groups, product types, time periods).\n",
    "\n",
    "#### 2. **High-Level Architecture of TFMA**\n",
    "   - **Pipeline Components**:\n",
    "     - **Read Inputs**: Converts raw input data (e.g., CSV, TF Records) into a dictionary format (`tfma.extracts`) for processing.\n",
    "     - **Extractors**: Uses Apache Beam for distributed processing. Includes `Input Extractor` for data slicing and `Predict Extractor` for running predictions on slices.\n",
    "     - **Evaluators**: Also utilizes Apache Beam for distributed evaluation. Includes `Metrics and Plots Evaluator` to compare model predictions against actual results. Custom evaluators can be added.\n",
    "     - **Write Results**: Stores evaluation results, making them available for inspection and further analysis.\n",
    "\n",
    "#### 3. **Comparison of TFMA and TensorBoard**\n",
    "   - **TensorBoard**:\n",
    "     - Focuses on monitoring the training process of a model in real-time, visualizing streaming metrics during training (often on a mini-batch basis).\n",
    "     - Useful for inspecting and comparing the training progress of multiple models.\n",
    "\n",
    "   - **TFMA**:\n",
    "     - Conducts deep post-training analysis of a trained model, often as part of a machine learning pipeline (e.g., TFX).\n",
    "     - Compares performance across different versions of trained models.\n",
    "     - Slices data to identify and analyze potential issues within specific subsets.\n",
    "\n",
    "#### 4. **Detailed Analysis and Slicing**\n",
    "   - **Slicing Metrics**: TFMA’s slicing capability allows for a more granular analysis of model performance by examining specific data segments (e.g., daytime vs. nighttime taxi rides).\n",
    "   - **Importance of Domain Knowledge**: Effective slicing requires understanding the domain and the data to determine the most meaningful segments to analyze.\n",
    "   - **Accuracy and Scalability**: TFMA uses Apache Beam to perform a full pass over the evaluation dataset, ensuring accurate metric computation, especially important for small slices of data.\n",
    "\n",
    "#### 5. **Practical Applications**\n",
    "   - **Model Performance Issues**: TFMA helps uncover issues that aggregate metrics might miss, such as a model underperforming in certain time periods or for specific customer segments.\n",
    "   - **Distributed Processing**: TFMA scales to large datasets using distributed processing backends with Apache Beam.\n",
    "\n",
    "### Short Notes:\n",
    "   - **TFMA Purpose**: Deep analysis of model performance, focusing on data slices.\n",
    "   - **Pipeline Components**: Read Inputs, Extractors, Evaluators, Write Results.\n",
    "   - **TFMA vs TensorBoard**: TFMA for post-training analysis; TensorBoard for real-time training monitoring.\n",
    "   - **Slicing Metrics**: Crucial for identifying hidden performance issues.\n",
    "   - **Scalability**: TFMA uses Apache Beam for large-scale, accurate evaluations.\n",
    "\n",
    "This summary covers the key aspects of TensorFlow Model Analysis (TFMA) and its role in analyzing and improving model performance, particularly by focusing on specific data slices and segments."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TFMA in Practice"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'tensforflow_transform'",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[2], line 3\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[38;5;66;03m# Step 1: Export EvalSavedModel for TFMA\u001b[39;00m\n\u001b[0;32m      2\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mtensorflow\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01mtf\u001b[39;00m\n\u001b[1;32m----> 3\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mtensforflow_transform\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01mtft\u001b[39;00m\n\u001b[0;32m      4\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mtensforflow_model_analysis\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01mtfma\u001b[39;00m\n",
      "\u001b[1;31mModuleNotFoundError\u001b[0m: No module named 'tensforflow_transform'"
     ]
    }
   ],
   "source": [
    "# Step 1: Export EvalSavedModel for TFMA\n",
    "import tensorflow as tf\n",
    "import tensforflow_transform as tft\n",
    "import tensforflow_model_analysis as tfma"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_serve_tf_examples_fn(model, tf_transform_output):\n",
    "    # Return a function that parses a serialized tf.Example and applies TFT\n",
    "    \n",
    "    tf_transform_output = tft.TFTransformOutput(tf_transform_output_dir)\n",
    "    signatures = {\n",
    "        \"serving_default\": _get_serve_tf_examples_fn(model, tf_transform_output).get_concrete_function(tf.TensorSpec(...)),\n",
    "    }\n",
    "    \n",
    "model.save(serving_model_dir_path, save_format=\"tf\", signatures=signatures)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Step 2: Create EvalConfig\n",
    "\n",
    "# Specify slicing spec\n",
    "slice_spec = [slicer.SingleSliceSpec(columns = [\"column_name\"]), ...]\n",
    "\n",
    "# Define metrics\n",
    "metrics = [tf.keras.metrics.Accuracy(name = \"Accuracy\"),\n",
    "           tfma.metrics.MeanPrediction(name=\"mean_prediction\"), ...]\n",
    "metrics_specs = tfma.metrics.specs_from_metrics(metrics)\n",
    "\n",
    "eval_config = tfma.EvalConfig(\n",
    "    model_specs = [tfma.ModelSpec(label_key = features.LABEL_KEY)],\n",
    "    slicing_specs = slice_spec,\n",
    "    metrics_specs = metrics_specs,\n",
    "    ...\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Step 3: Analyze model\n",
    "\n",
    "# Specify the path to the eval graph and to where the result should be written\n",
    "\n",
    "eval_model_dir = ...\n",
    "result_path = ...\n",
    "\n",
    "eval_shared_model = tfma.default_eval_shared_model(\n",
    "    eval_saved_model_path = eval_model_dir,\n",
    "    eval_config=eval_config\n",
    ")\n",
    "\n",
    "# Run TensorFlow Model Analysis\n",
    "eval_result = tfma.run_model_analysis(eval_shared_model = eval_shared_model,\n",
    "                                      output_path = result_path,\n",
    "                                      ...)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Step 4: Visualizing metrics\n",
    "\n",
    "# render results\n",
    "tfma.viewer.render_slicing_metrics(result)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model Debugging Overview\n",
    "\n",
    "#### 1. **Model Robustness**\n",
    "   - **Definition**: Robustness refers to a model’s ability to produce consistently accurate results, even when one or more features change significantly.\n",
    "   - **Key Characteristics**:\n",
    "     - **Gradual Changes**: A robust model’s performance changes gradually and predictably as data changes.\n",
    "     - **Sudden Changes**: An overly sensitive model might show wild fluctuations in performance with small data changes, indicating a lack of robustness.\n",
    "   - **Measuring Robustness**:\n",
    "     - **Avoid Training Data**: Do not measure robustness during training or using the training data. Use test data or unseen data.\n",
    "     - **New Data**: It is often beneficial to generate new types of data for robustness testing.\n",
    "     - **Metrics**: Use the same metrics as for model performance (e.g., RMSE for regression, AUC for classification) to evaluate robustness.\n",
    "\n",
    "#### 2. **Model Debugging**\n",
    "   - **Overview**: Model debugging focuses on detecting and fixing issues within a model, improving its robustness, and ensuring consistent performance across different data conditions. It borrows practices from model risk management, traditional diagnostics, and software testing.\n",
    "   - **Objectives**:\n",
    "     - **Transparency**: Make models less of a black box by clarifying how data flows internally.\n",
    "     - **Social Discrimination**: Ensure that the model performs fairly across different groups of people and does not exhibit bias.\n",
    "     - **Security**: Reduce model vulnerability to adversarial attacks, such as attempts to extract data from the model. Anonymizing data before training is critical, especially if sensitive data is involved.\n",
    "     - **Performance Decay**: Address issues related to performance decay over time as incoming data distributions change.\n",
    "\n",
    "#### 3. **Model Debugging Techniques**\n",
    "   - **Benchmarking Models**: Comparing the performance of a model against a baseline or another model to ensure consistency and robustness.\n",
    "   - **Sensitivity Analysis**: Testing how changes in input features affect model predictions. This can help identify features that the model is overly sensitive to, thus indicating areas where robustness can be improved.\n",
    "   - **Residual Analysis**: Analyzing the difference between the predicted and actual values to identify systematic errors or biases in the model.\n",
    "\n",
    "### Short Notes:\n",
    "   - **Robustness**: Consistent accuracy even with changing features; avoid using training data for robustness evaluation.\n",
    "   - **Model Debugging**: Improving model transparency, fairness, security, and addressing performance decay.\n",
    "   - **Debugging Techniques**: Benchmarking, sensitivity analysis, residual analysis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Benchmark Models\n",
    "\n",
    "#### 1. **Benchmarking Models**\n",
    "   - **Purpose**: Benchmarking models serve as simple, reliable baselines to compare against your developed model.\n",
    "   - **Characteristics**:\n",
    "     - **Simplicity**: Typically linear or straightforward models, not state-of-the-art.\n",
    "     - **Consistency**: They provide consistent performance, helping you establish a baseline for your problem.\n",
    "   - **Usage**:\n",
    "     - **Sanity Check**: Compare your model to the benchmark to ensure it performs better. If it doesn't, either your model has a problem or the simple model is sufficient for the task.\n",
    "     - **Debugging Tool**: Even if your model outperforms the benchmark, identifying instances where the benchmark succeeds, but your model fails, can help uncover issues in your model.\n",
    "   - **Role in Development**:\n",
    "     - **Before Development**: Benchmark models help set realistic performance expectations.\n",
    "     - **During Debugging**: They serve as a diagnostic tool to identify specific weaknesses in your model's predictions.\n",
    "\n",
    "### Short Notes:\n",
    "   - **Benchmarking Models**: Simple, consistent baselines for performance comparison.\n",
    "   - **Sanity Test**: Ensures your model outperforms a simple baseline.\n",
    "   - **Debugging**: Identifies weaknesses where your model underperforms relative to the benchmark."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sensitivity Analysis and Adversarial Attacks\n",
    "\n",
    "\n",
    "#### 1. **Sensitivity Analysis**\n",
    "   - **Purpose**: Sensitivity analysis examines how individual features affect a model's predictions. \n",
    "   - **Method**: By varying one feature while holding others constant, you can observe how sensitive the model is to changes in that feature.\n",
    "   - **Techniques**:\n",
    "     - **Random Attacks**: Generate random input data to test model outputs, which helps reveal unexpected bugs.\n",
    "     - **Partial Dependence Plots (PDPs)**: Visualize the relationship between a feature and the model's predictions. Tools like PDPbox and PyCEbox can create these plots.\n",
    "\n",
    "#### 2. **Vulnerability to Adversarial Attacks**\n",
    "   - **Adversarial Examples**: Slightly altered inputs designed to fool the model into making incorrect predictions with high confidence.\n",
    "   - **Real-World Implications**:\n",
    "     - **Autonomous Vehicles**: Misrecognizing altered traffic signs could lead to dangerous outcomes.\n",
    "     - **Security Applications**: Vulnerable models in security applications, such as suitcase scanners, pose serious threats.\n",
    "\n",
    "#### 3. **Types of Security Risks**\n",
    "   - **Informational Harms**:\n",
    "     - **Membership Inference Attacks**: Inferring if specific data was used in training.\n",
    "     - **Model Inversion Attacks**: Reconstructing training data from model outputs (e.g., recreating images of faces).\n",
    "     - **Model Extraction Attacks**: Reconstructing a model based on its outputs, compromising security and intellectual property.\n",
    "   - **Behavioral Harms**:\n",
    "     - **Model Poisoning Attacks**: Injecting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Adversarial Attack Demo\n",
    "\n",
    "\n",
    "#### 1. **Adversarial Examples**\n",
    "   - **Definition**: Adversarial examples are specially crafted inputs designed to deceive machine learning models, especially neural networks, into making incorrect predictions. These inputs are often indistinguishable from normal data to the human eye but cause significant errors in model predictions.\n",
    "\n",
    "#### 2. **Fast Gradient Sign Method (FGSM)**\n",
    "   - **Type**: White-box attack (attacker has full access to the model’s internals).\n",
    "   - **Goal**: To create adversarial examples that lead to misclassification by slightly perturbing the original input.\n",
    "   - **Mechanism**: FGSM perturbs the input data in the direction of the gradient of the loss with respect to the input features, scaled by a small factor $\\epsilon$.\n",
    "\n",
    "#### 3. **Example Attack**\n",
    "   - **Original Image**: A Labrador Retriever.\n",
    "   - **Process**:\n",
    "     1. **Pre-processing**: Load and prepare the image for analysis.\n",
    "     2. **Generate Adversarial Example**:\n",
    "        - **Code**: The code for FGSM is straightforward and involves computing gradients and adding perturbations.\n",
    "     3. **Different $\\epsilon$\n",
    " Values**:\n",
    "        - Small perturbations (small $\\epsilon$\n",
    ") cause the model to misclassify the image while still being visually similar to the original.\n",
    "        - As $\\epsilon$\n",
    " increases, the perturbations become more noticeable, leading to different misclassifications.\n",
    "   - **Visualization**:\n",
    "     - The model initially classifies the original image correctly.\n",
    "     - With small $\\epsilon$\n",
    ", the image is misclassified as a different breed (e.g., Saluki).\n",
    "     - As $\\epsilon$\n",
    " increases, the misclassification becomes more pronounced (e.g., Weimaraner).\n",
    "\n",
    "#### 4. **Conclusion**\n",
    "   - FGSM is a simple yet effective method to demonstrate how neural networks can be easily fooled by small, imperceptible changes in the input data.\n",
    "   - The ease of generating adversarial examples highlights the need for robust models that can resist such attacks.\n",
    "\n",
    "This example illustrates how adversarial attacks can exploit vulnerabilities in neural networks and emphasizes the importance of developing models that are resilient to such manipulations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Residual Analysis\n",
    "\n",
    "### Residual Analysis: Overview and Techniques\n",
    "\n",
    "#### 1. **Residual Analysis**\n",
    "   - **Purpose**: To evaluate the performance of regression models by examining the differences (residuals) between predicted values and the actual ground truth values.\n",
    "   - **Application**: Most useful for regression problems where you can measure prediction errors and have ground truth values for comparison.\n",
    "\n",
    "#### 2. **Key Concepts**\n",
    "   - **Residuals**: The difference between the predicted value and the actual value ($ \\text{residual} = \\text{actual} - \\text{predicted} $).\n",
    "   - **Ideal Residuals**: Should be randomly distributed without patterns. This indicates that the model has captured all predictive information.\n",
    "\n",
    "#### 3. **Techniques for Residual Analysis**\n",
    "   - **Residual Plots**:\n",
    "     - **Purpose**: To visually inspect the distribution of residuals.\n",
    "     - **Ideal Plot**: Residuals should appear as a random scatter with no discernible patterns.\n",
    "     - **Indicators of Issues**: Systematic patterns or clustering in residual plots suggest that the model is missing some predictive information.\n",
    "\n",
    "   - **Correlation with Features**:\n",
    "     - **Objective**: Residuals should not be correlated with any feature that was not included in the model.\n",
    "     - **Process**: Check if residuals can be predicted by any omitted features. If so, consider including those features in the model.\n",
    "\n",
    "   - **Autocorrelation**:\n",
    "     - **Definition**: When residuals are correlated with each other across observations.\n",
    "     - **Ideal Condition**: Residuals should not show autocorrelation.\n",
    "     - **Detection**: Use a Durbin-Watson test to detect autocorrelation. Visual inspection of residual plots can also reveal patterns.\n",
    "     - **Issue Indication**: If adjacent residuals show patterns (e.g., consecutive residuals having the same sign), it suggests that the model might not be capturing all relevant temporal or sequential information.\n",
    "\n",
    "#### 4. **Practical Steps**\n",
    "   - **Generate Residuals**: Calculate the residuals for your predictions.\n",
    "   - **Plot Residuals**: Create residual plots to visually inspect the distribution.\n",
    "   - **Check Correlation**: Assess whether residuals correlate with any omitted features.\n",
    "   - **Test for Autocorrelation**: Apply the Durbin-Watson test or similar methods to identify autocorrelation.\n",
    "\n",
    "#### 5. **Conclusion**\n",
    "   - **Objective of Residual Analysis**: To identify and address potential weaknesses in your model by understanding how well it captures the predictive information in the data.\n",
    "   - **Improvement Actions**: Based on the analysis, adjust the model to include relevant features or address any detected patterns or correlations in the residuals.\n",
    "\n",
    "Residual analysis provides valuable insights into model performance and can help in fine-tuning models to better capture the underlying patterns in the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model Remediation\n",
    "\n",
    "#### 1. **Data Augmentation**\n",
    "   - **Purpose**: Enhance the model’s ability to generalize and reduce sensitivity.\n",
    "   - **Techniques**:\n",
    "     - **Generative Methods**: Create synthetic examples to expand the training dataset.\n",
    "     - **Interpretive Techniques**: Apply transformations to existing data (e.g., rotations, cropping).\n",
    "     - **Adding Noise**: Introduce noise to training data to make the model more resilient to variations.\n",
    "   - **Benefits**: Helps correct for unbalanced data and improves generalization.\n",
    "\n",
    "#### 2. **Understanding Model Internals**\n",
    "   - **Importance**: Gaining insight into how a model makes decisions can help in improving robustness.\n",
    "   - **Tools and Techniques**:\n",
    "     - **Model Interpretability Tools**: Use tools that offer insights into model decisions and internal workings.\n",
    "     - **Interpretable Models**: Choose models designed for interpretability, such as tree-based models or certain neural network architectures.\n",
    "\n",
    "#### 3. **Model Editing**\n",
    "   - **Description**: Modify models to address performance issues or improve robustness.\n",
    "   - **Example**: Adjusting decision trees to better fit the data or correct errors in predictions.\n",
    "\n",
    "#### 4. **Model Assertions**\n",
    "   - **Purpose**: Apply business rules or sanity checks to ensure that model outputs are reasonable.\n",
    "   - **Examples**:\n",
    "     - **Age Predictions**: Ensure predicted ages are non-negative.\n",
    "     - **Credit Limits**: Ensure predicted credit limits do not exceed predefined maximums.\n",
    "\n",
    "#### 5. **Discrimination Remediation**\n",
    "   - **Objective**: Reduce or eliminate bias in model predictions.\n",
    "   - **Strategies**:\n",
    "     - **Diverse Data Sets**: Use data that accurately represents the population who will use the model.\n",
    "     - **Diverse Development Team**: Include team members with expertise in ethics, social sciences, and related fields.\n",
    "     - **Fairness Metrics**: Use tools like Fairness Indicators to assess and ensure model fairness.\n",
    "     - **Techniques**:\n",
    "       - **Fair Representations (LFR)**: Train models to learn fair representations of data.\n",
    "       - **Adversarial Debiasing**: Use tools such as AIF360 for adversarial debiasing.\n",
    "       - **Dual Objective Functions**: Optimize for both accuracy and fairness metrics.\n",
    "       - **Post-Processing**: Adjust model predictions after training to ensure fairness.\n",
    "\n",
    "#### 6. **Continuous Monitoring and Debugging**\n",
    "   - **Purpose**: Ensure ongoing model performance, fairness, and security.\n",
    "   - **Techniques**:\n",
    "     - **Monitoring Tools**: Implement tools to regularly check model accuracy, fairness, and security.\n",
    "     - **Anomaly Detection**: Use methods like autoencoders and isolation forests to detect unusual input and prediction values.\n",
    "     - **Data Integrity Constraints**: Apply constraints to ensure input data is valid.\n",
    "     - **Statistical Process Control**: Use statistical methods to monitor data and model predictions.\n",
    "     - **Benchmarking**: Compare model predictions to those of benchmark models to detect discrepancies.\n",
    "\n",
    "#### 7. **Resources and Tools**\n",
    "   - **Libraries and Tools**:\n",
    "     - **AIF360**: IBM’s toolkit for fairness in AI.\n",
    "     - **Google Model Remediation Library**: Tools for improving model fairness.\n",
    "     - **CleverHans**: Library for adversarial attacks and defenses.\n",
    "     - **Foolbox**: Library for evaluating the robustness of machine learning models.\n",
    "\n",
    "By incorporating these remediation techniques, you can enhance your model’s robustness, reduce bias, and ensure ongoing performance improvements throughout its lifecycle."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fairness\n",
    "\n",
    "#### **Overview of Fairness Indicators**\n",
    "\n",
    "**Fairness Indicators** is an open-source library developed by TensorFlow for assessing the fairness of machine learning models. It's designed to measure fairness metrics for both binary and multiclass classifiers. Key features of the library include:\n",
    "\n",
    "- **Measurement of Fairness Metrics**: Provides commonly used metrics to evaluate fairness, such as demographic parity, equal opportunity, and disparate impact.\n",
    "- **Comparison Across Subgroups**: Allows comparison of model performance across different subgroups and against baselines or other models.\n",
    "- **Confidence Intervals**: Uses confidence intervals to identify statistically significant disparities.\n",
    "- **Evaluation Over Multiple Thresholds**: Assesses how fairness metrics vary across different decision thresholds.\n",
    "\n",
    "#### **Using Fairness Indicators Effectively**\n",
    "\n",
    "1. **Slice Analysis**\n",
    "   - **Definition**: Evaluate model performance on different slices of data (subgroups) to uncover disparities.\n",
    "   - **Importance**: Helps in identifying fairness issues that might be masked when analyzing the entire dataset.\n",
    "   - **Approach**: Focus on slices sensitive to fairness, such as race, gender, or socioeconomic status.\n",
    "\n",
    "2. **Selecting Metrics**\n",
    "   - **Consideration**: Choose metrics relevant to your dataset and user base to accurately reflect fairness.\n",
    "   - **Examples**: \n",
    "     - **Demographic Parity**: Measures whether different groups are equally represented among positive predictions.\n",
    "     - **Equal Opportunity**: Assesses whether true positive rates are similar across groups.\n",
    "     - **Disparate Impact**: Evaluates whether the proportion of positive outcomes differs across groups.\n",
    "\n",
    "3. **Threshold Analysis**\n",
    "   - **Definition**: Evaluate fairness metrics across various decision thresholds.\n",
    "   - **Purpose**: Understand how changing thresholds impact fairness and model performance for different groups.\n",
    "\n",
    "4. **Margin of Separation**\n",
    "   - **Consideration**: For predictions near the decision boundary, report the rate at which the label is predicted.\n",
    "   - **Purpose**: Helps in understanding model performance and fairness for predictions that are less certain.\n",
    "\n",
    "#### **Guidelines for Avoiding Common Pitfalls**\n",
    "\n",
    "1. **Conduct Comprehensive Fairness Tests**\n",
    "   - Perform fairness evaluations on all available slices of data to get a holistic view of model performance.\n",
    "\n",
    "2. **Evaluate Across Multiple Thresholds**\n",
    "   - Assess how fairness metrics vary with different decision thresholds to understand their impact on various groups.\n",
    "\n",
    "3. **Report Predictions with Low Confidence**\n",
    "   - For predictions close to the decision boundary, report prediction rates to provide context on model reliability.\n",
    "\n",
    "#### **Best Practices for Implementing Fairness**\n",
    "\n",
    "- **Engage Domain Experts**: Consult with social scientists, cultural anthropologists, and members of affected communities to ensure comprehensive fairness assessment.\n",
    "- **Work with Labeled Data**: If possible, use labeled data to directly measure fairness. If labeled data is not available, apply statistical methods to estimate distributions and potential disparities.\n",
    "- **Consider User Experience**: Think about how different user groups experience your application and ensure that fairness metrics align with their needs and expectations.\n",
    "\n",
    "By following these guidelines and utilizing Fairness Indicators, you can better understand and mitigate biases in your models, leading to more equitable outcomes and improved model performance across diverse groups."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Measuring Fairness\n",
    "\n",
    "Fairness Indicators provide several key metrics to assess the fairness of a model. Here’s a detailed overview of each metric and its application:\n",
    "\n",
    "#### **1. Positive and Negative Rates**\n",
    "- **Positive Rate**: Percentage of data points classified as positive.\n",
    "- **Negative Rate**: Percentage of data points classified as negative.\n",
    "- **Purpose**: Helps evaluate demographic parity, which is crucial when having equal percentages of outcomes for different groups is important.\n",
    "\n",
    "#### **2. True Positive Rate (TPR) and False Negative Rate (FNR)**\n",
    "- **True Positive Rate (TPR)**: Percentage of positive data points correctly predicted as positive. Also known as recall.\n",
    "- **False Negative Rate (FNR)**: Percentage of positive data points incorrectly predicted as negative.\n",
    "- **Purpose**: Relates to equality of opportunity for the positive class. Essential for scenarios where the same percentage of qualified candidates should be positively rated across groups, such as in loan approvals or admissions.\n",
    "\n",
    "#### **3. True Negative Rate (TNR) and False Positive Rate (FPR)**\n",
    "- **True Negative Rate (TNR)**: Percentage of negative data points correctly predicted as negative.\n",
    "- **False Positive Rate (FPR)**: Percentage of negative data points incorrectly predicted as positive.\n",
    "- **Purpose**: Relates to quality of opportunity for the negative class. Important in cases where misclassifying negatives as positives is more concerning, such as in fraud detection or abusive content filtering.\n",
    "\n",
    "#### **4. Accuracy and Area Under the Curve (AUC)**\n",
    "- **Accuracy**: Percentage of data points correctly classified overall.\n",
    "- **Area Under the Curve (AUC)**: Measures the model’s ability to discriminate between classes, independent of class distribution.\n",
    "- **Purpose**: Evaluates predictive parity. Important for tasks where the overall precision of classification is critical, like face identification or clustering.\n",
    "\n",
    "### Practical Considerations\n",
    "\n",
    "1. **Interpreting Metrics**:\n",
    "   - Significant differences in metrics between groups may indicate fairness issues.\n",
    "   - Interpret results in the context of your specific use case and the impact of fairness on different groups.\n",
    "\n",
    "2. **Ongoing Fairness Evaluation**:\n",
    "   - Achieving equality in fairness metrics doesn’t guarantee overall fairness. Continuous evaluation throughout the development lifecycle and post-launch is crucial.\n",
    "   - Fairness metrics can change as training data, model inputs, or design evolve.\n",
    "\n",
    "3. **Adversarial Testing**:\n",
    "   - Fairness evaluations are not a replacement for adversarial testing. They should complement adversarial testing to identify and defend against rare, malicious examples.\n",
    "\n",
    "### Concrete Example: CelebA Dataset\n",
    "\n",
    "**Dataset Overview**:\n",
    "- **CelebA**: Contains 200,000 celebrity images with 40 attribute annotations (e.g., hair type, facial features) and facial landmark locations.\n",
    "\n",
    "**Example Task**:\n",
    "1. **Build a Classification Model**:\n",
    "   - Task: Identify whether a person in the image is smiling or not.\n",
    "\n",
    "2. **Evaluate Model Performance**:\n",
    "   - Use Fairness Indicators to assess the model’s performance across different age groups.\n",
    "\n",
    "3. **Plot Metrics**:\n",
    "   - Plot fairness metrics (e.g., TPR, FNR) to analyze how well the model performs across different demographic slices.\n",
    "\n",
    "**Steps**:\n",
    "1. **Data Preparation**: Load and preprocess the CelebA dataset.\n",
    "2. **Model Training**: Train a classifier on the smiling attribute.\n",
    "3. **Fairness Evaluation**: Use Fairness Indicators to analyze performance by age group.\n",
    "4. **Visualization**: Plot metrics to understand disparities and areas for improvement.\n",
    "\n",
    "By using Fairness Indicators in this way, you can systematically evaluate and improve the fairness of your model, ensuring that it performs equitably across different groups and minimizing potential biases."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Continuous Evaluation and Monitoring\n",
    "\n",
    "Continuous monitoring and evaluation are crucial for maintaining the performance of machine learning models in production. Here’s an overview of key concepts and techniques:\n",
    "\n",
    "#### **1. Understanding Data Drift and Shift**\n",
    "\n",
    "- **Concept Drift**: Refers to a change in the relationship between input data and labels. For instance, if fashion trends shift, a model trained on outdated data (e.g., predicting VHS sales) may become inaccurate.\n",
    "- **Emerging Concepts**: New patterns or labels that were not present in the training data but become relevant over time.\n",
    "- **Covariate Shift**: The distribution of input data changes while the conditional probability of the output given the input remains the same.\n",
    "- **Prior Probability Shift**: The distribution of the output labels changes while the distribution of the input data stays the same.\n",
    "\n",
    "#### **2. Monitoring Techniques**\n",
    "\n",
    "- **Monitoring Raw Data**: Compare incoming data to training data to detect covariate shift.\n",
    "- **Monitoring Predictions**: Evaluate predictions to detect prior probability shift.\n",
    "- **Labeling Process**: Track new data and labels to detect concept drift.\n",
    "\n",
    "#### **3. Supervised Techniques**\n",
    "\n",
    "- **Statistical Process Control**: Monitors error rates in the incoming data stream using statistical methods. Requires labeled data.\n",
    "- **Sequential Analysis**: Uses the contingency table from a classifier to calculate metrics like precision and recall. Checks for consistency over time.\n",
    "- **Error Distribution Monitoring**: Uses adaptive windowing to monitor mean error rates over time, comparing them against thresholds using the Hoeffding bound.\n",
    "\n",
    "#### **4. Unsupervised Techniques**\n",
    "\n",
    "- **Clustering or Novelty Detection**: Clusters incoming data to detect new, emerging patterns that were not present in the training data. Algorithms include OLINDDA, MINAS, ECSMiner, and GC3.\n",
    "- **Feature Distribution Monitoring**: Monitors changes in individual feature distributions using methods like Pearson correlation and Hellinger distance. Useful for detecting changes in feature distributions but not population drift.\n",
    "- **Model-Dependent Monitoring**: Monitors the space near the decision boundaries in the model’s latent feature space. Techniques like Margin Density Drift Detection (MD3) can detect drift by observing changes in margin density.\n",
    "\n",
    "#### **5. Continuous Evaluation Services**\n",
    "\n",
    "- **Google Cloud AI Continuous Evaluation**:\n",
    "  - Samples prediction inputs and outputs regularly.\n",
    "  - Compares model predictions to ground truth labels.\n",
    "  - Provides feedback on model performance.\n",
    "  - Integrates with AI Platform Prediction and BigQuery for data storage and evaluation.\n",
    "\n",
    "- **Microsoft Azure Machine Learning Datasets**: Focuses on data drift.\n",
    "- **Amazon SageMaker Model Monitor**: Focuses on concept drift.\n",
    "\n",
    "#### **6. Retraining Strategies**\n",
    "\n",
    "- **Trigger-Based Retraining**: Automatically retrain the model when performance metrics indicate a need for improvement.\n",
    "- **Scheduled Retraining**: Regularly retrain the model on a set schedule.\n",
    "- **Example**: In retail, if a model is predicting winter sports equipment sales and trends shift to summer sports, continuous monitoring helps catch the problem early and retrain the model with recent data.\n",
    "\n",
    "### Summary\n",
    "\n",
    "Monitoring and continuously evaluating your model helps you detect and address performance issues before they impact your application or users. By employing both supervised and unsupervised techniques, and leveraging services like Google Cloud AI Continuous Evaluation, you can ensure that your model adapts to changes in data and maintains its effectiveness over time. Regular retraining based on monitoring insights helps keep your model accurate and relevant."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}