PnPXAI: Plug-and-Play Explainable AI

PnPXAI is a Python package that provides a modular and easy-to-use framework for explainable artificial intelligence (XAI). It allows users to apply various XAI methods to their own models and datasets, and visualize the results in an interactive and intuitive way.

Features

• Detector: The detector module provides automatic detection of AI models implemented in PyTorch.
• Evaluator: The evaluator module provides various ways to evaluate and compare the performance and explainability of AI models with the categorized evaluation properties of correctness (fidelity, area between perturbation curves), continuity (sensitivity), and compactness (complexity).
• Explainers: The explainers module contains a collection of state-of-the-art XAI methods that can generate global or local explanations for any AI model, such as:
  • Perturbation-based (SHAP, LIME)
  • Relevance-based (IG, LRP, RAP, GuidedBackprop)
  • CAM-based (GradCAM, Guided GradCAM)
  • Gradient-based (SmoothGrad, VarGrad, FullGrad, Gradient × Input)
• Recommender: The recommender module offers a recommender system that can suggest the most suitable XAI methods for a given model and dataset, based on the user’s preferences and goals.
• Optimizer: The optimizer module is finds the best hyperparameter options, given a user-specified metric.

Installation

To install pnpxai from pip, run the following command:

pip install pnpxai

To install pnpxai from GitHub, run the following commands:

git clone [email protected]:OpenXAIProject/pnpxai.git
cd pnpxai
pip install -e .

Getting Started

This guide explains how to automatically explain your own models and datasets using the provided Python script. The complete code can be found here.

1. Setup: The setup involves setting a random seed for reproducibility and defining the device for computation (CPU or GPU).

   import torch
   from pnpxai.utils import set_seed
   
   # Set the seed for reproducibility
   set_seed(seed=0)
   
   # Determine the device based on the availability
   device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

2. Create Experiments: An experiment is an instance for explaining a specific model and dataset. Before creating an experiment, define the model and dataset to be explained.

   Automatic explainer selection: The AutoExplanationForImageClassification method automatically selects the most applicable explainers and metrics based on the model architecture using pnpxai.XaiRecommender.

   import torch
   from torch.utils.data import DataLoader
   
   from pnpxai import AutoExplanationForImageClassification
   
   # Bring your model
   model = ...
   
   # Prepare your data
   dataset = ...
   loader = DataLoader(dataset, batch_size=...)
   def input_extractor(x):
       ...
   def target_extractor(x):
       ...
   
   # Auto-explanation
   experiment = AutoExplanationForImageClassification(
       model,
       loader,
       input_extractor=input_extractor,
       label_extractor=label_extractor,
       target_extractor=target_extractor,
       target_labels=False,
   )
   optimized = experiment.optimize(
       data_ids=range(16),
       explainer_id=2,
       metric_id=1,
       direction='maximize', # less is better
       sampler='tpe', # Literal['tpe','random']
       n_trials=50, # by default, 50 for sampler in ['random', 'tpe'], None for ['grid']
       seed=42, # seed for sampler: by default, None
   )

   Manual explainer selection: Alternatively, you can manually specify the desired explanation method and evaluation metric using Experiment.

   from pnpxai.core.modality import ImageModality
   from pnpxai.explainers import LRPEpsilonPlus
   from pnpxai.evaluator.metrics import MuFidelity
   
   explainer = LRPEpsilonPlus(model)
   metric = MuFidelity(model, explainer)
   modality = ImageModality()
   
   experiment = Experiment(
       model,
       loader,
       modality,
       explainers=[explainer],
       metrics=[metric],
       input_extractor=input_extractor,
       label_extractor=label_extractor,
       target_extractor=target_extractor,
   )

Tutorials

• Image Classification
• Text Classification
• Time Series Classification
• Visual Question Answering
• Evaluator
• ImageNet Example All Explainers
• ImageNet Example All Metrics
• Free MCG [Reference]

Use Cases

Medical Domain Explainability

• Counterfactual Explanation (LEAR (Learn-Explain-Reinforce)) for Alzheimer’s Disease Diagnosis, a joint work with Research Task 2 (PI Bohyung Han, Seoul National University) [Reference]

• Attribution-based Explanation for Dysarthria Diagnosis, a joint work with Research Task 3 (PI Myoung-Wan Koo, Sogang University)

LLM Trsutworthiness

• Evaluating the Factuality of Korean Text Generated by LLMs (KorFactScore (Korean Factual precision in atomicity Score)), a joint work with Research Task 4 (PI Kyongman Bae, ETRI) [Reference]

Documentation

The Documentation contains the API reference for all of the functionality of the framework. Primarily, high-level modules of the framework include:

• Detector
• Explainer
• Recommender
• Evaluator
• Optimizer

Acknowledgements

This research was initiated by KAIST XAI Center and conducted in collaboration with multiple institutions, including Seoul National University, Korea University, Sogang University, and ETRI. We are grateful for the grant from the Institute of Information & communications Technology Planning & Evaluation (IITP) (No.RS-2022-II220984).

License

PnP XAI is released under Apache license 2.0. See LICENSE for additional details.