active_learning_multiannotator/active_learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Active Learning with Multiple Data Annotators via ActiveLab"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/cleanlab/examples/blob/master/active_learning_multiannotator/active_learning.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook demonstrates a practical approach to active learning for training classification models with cleanlab. You can run this same code with *any* classifier model for any data type (image, text, tabular, audio, etc).\n",
    "\n",
    "In active learning, we aim to construct a labeled dataset by collecting the fewest labels that still allow us to train an accurate classifier model. Here we assume data labeling is done in **batches**, and between these data labeling rounds, we retrain our classifier and combine it with an [ActiveLab algorithm](https://arxiv.org/abs/2301.11856) that decides which examples (i.e. datapoints) would be most informative to label next round. We assume there are **multiple** annotators to label the data, but since their labels are not necessarily perfect, we sometimes wish to ask a new annotator to provide an extra label for a previously labeled example. This active learning with multiple annotators setup is more widely applicable than academic studies which only consider labeling one example in each round or being limited to a single (noise-free) annotation per example. The code demonstrated here still works effectively in single annotator settings too!\n",
    "\n",
    "cleanlab provides two useful estimates for multi-annotator active learning: (1) an active learning score estimating how informative an additional label would be for each example, (2) accurate consensus labels for the previously labeled examples that can be used for classifier model training.\n",
    "\n",
    "This notebook demonstrates how to use these estimates for sequential active learning, showing how a classification model iteratively improves after (re)labeling examples for multiple rounds. Some examples in the initial dataset are allowed to have been labeled by multiple annotators, while other examples may have 0 labels (unlabeled pool). Cleanlab's active learning score indicates whether you should collect: another label for an already-labeled example (ie. having one more annotator review some potentially difficult example), or the first label for a new example that has not yet been labeled. The active learning scores properly weigh this trade-off based on cleanlab's estimates of label quality, in order to maximally improve your classifier with the fewest total labels.\n",
    "\n",
    "This notebook implements the following steps:\n",
    "\n",
    "1. Establish consensus labels (for the already labeled data). Use them to train a classifier model and then obtain out-of-sample predicted probabilities for each example.\n",
    "2. Compute active learning scores for every example, which estimate our current confidence in knowing its true label.\n",
    "3. Collect additional labels for the examples with the lowest active learning scores. These are the most potentially informative examples whose true label we are least certain of. \n",
    "4. Repeat the steps above to collect as many labels as your budget permits.\n",
    "\n",
    "The accuracy of the model trained on the resulting dataset will generally match that of the same model trained on a much larger set of randomly collected labels. I.e. this is the most cost-effective way to train an accurate classifier!\n",
    "\n",
    "In this example we use a modified version of the [Wall-Following Robot Navigation](https://www.openml.org/search?type=data&sort=runs&status=any&qualities.NumberOfClasses=gte_2&qualities.NumberOfInstances=between_1000_10000&id=1526) dataset, with multiple data annotators who each provide noisy labels."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import dependencies and get data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Please install the dependencies specified in this [requirements.txt](https://github.com/cleanlab/examples/blob/master/active_learning_multiannotator/requirements.txt) file before running the notebook.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.ensemble import ExtraTreesClassifier\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from cleanlab.multiannotator import get_majority_vote_label, get_label_quality_multiannotator, get_active_learning_scores\n",
    "\n",
    "from utils.model_training import fit_predict_proba\n",
    "from utils.active_learning import setup_next_iter_data, add_new_annotator\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "File ‘data.tar.gz’ already there; not retrieving.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget -nc 'https://cleanlab-public.s3.amazonaws.com/ActiveLearning/WallRobot/data.tar.gz'\n",
    "!tar -xf data.tar.gz data/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We load the following datafiles:\n",
    "\n",
    "- `multiannotator_labels` is a DataFrame that contains labels from multiple annotators for each example (different number of annotations per example)\n",
    "- `X_labeled` are the features for the examples that have been labeled by at least one annotator\n",
    "- `X_unlabeled` are the features for the examples that have not yet been labeled by any annotator\n",
    "\n",
    "Additionally, we load some extra files for demo purposes, which will probably not be available in your application:\n",
    "\n",
    "- `extra_labels_labeled` and `extra_labels_unlabeled` are sampled from when we get ask annotators to provide additional labels later in this notebook\n",
    "- `X_test` and `true_labels_test` are used to measure the current model's predictive accuracy in each round of multi-annotator active learning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "multiannotator_labels = pd.DataFrame(np.load(\"data/multiannotator_labels_labeled.npy\"))\n",
    "\n",
    "X_labeled = np.load(\"data/X_labeled.npy\")\n",
    "X_unlabeled = np.load(\"data/X_unlabeled.npy\")\n",
    "\n",
    "# The files imported below are only used for demonstration purposes,\n",
    "# they are not required for active learning and often won't be available in your application\n",
    "extra_labels_labeled = np.load(\"data/extra_labels_labeled.npy\")\n",
    "extra_labels_unlabeled = np.load(\"data/extra_labels_unlabeled.npy\")\n",
    "X_test = np.load(\"data/X_test.npy\")\n",
    "true_labels_test = np.load(\"data/true_labels_test.npy\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train model to obtain predicted probabilites\n",
    "\n",
    "First, we train our model on a set of consensus labels obtained using majority vote to get the out-of-sample predicted class probabilities for both the labeled and unlabeled data. Note that the `multiannotator_labels` DataFrame used to obtain consensus labels only includes the examples that have been labeled by at least one annotator, as we are not able to get consensus labels for examples that have not been labeled at all.\n",
    "\n",
    "The function will return two sets of predicted probabilites, `pred_probs_labeled` are the predicted probabilites for examples that have existing annotator labels (they correspond directly with the features `X_labeled`), whereas `active_learning_scores_unlabeled` are the predicted probabilites for examples that do not have any annotator labels (they correspond directly with the features `X_unlabeled`). These predicted probabilities will later be used to compute the active learning score.\n",
    "\n",
    "If working with your own dataset, you should consider modifying this `fit_predict_proba` function so that it is better fitted for training your specific dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get current consensus labels\n",
    "consensus_labels = get_majority_vote_label(multiannotator_labels)\n",
    "\n",
    "# Train model to get out-of-sample predicted probabilies for both labels and unlabeled data\n",
    "pred_probs_labeled, pred_probs_unlabeled = fit_predict_proba(\n",
    "    ExtraTreesClassifier(),\n",
    "    X_labeled,\n",
    "    consensus_labels,\n",
    "    cv_n_folds=5,\n",
    "    X_unlabeled=X_unlabeled,\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Obtain active learning scores"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we will get the active learning scores for each datapoint (both labeled and unlabeled) by using a combination of the annotators' agremeent and model confidence. These scores represent how confident we are about an example's true label based on the currently obtained annotations; examples with the lowest scores are those for which additional labels should be collected (i.e. likely the most informative). These scores are estimated via an **ActiveLab** algorithm developed by the Cleanlab team, and may sometimes prioritize an already-labeled example over an unlabeled example if the annotations for the labeled example are deemed unreliable (ActiveLab appropriately estimates the value of collecting new annotations for unlabeled data vs already-labeled data). \n",
    "\n",
    "Similar to above, the `multiannotator_labels` DataFrame here should only include examples that have received at least one annotation (`multiannotator_labels` should have the same number of rows as `pred_probs_labeled`). This method returns two arrays: `active_learning_scores` represents the scores for examples with existing annotations, and `active_learning_scores_unlabeled` represents the scores for examples with no annotations so far. Scores should be directly comparable between these two groups, and you should favor collecting an additional label for the examples with the lowest scores."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "active_learning_scores, active_learning_scores_unlabeled = get_active_learning_scores(\n",
    "    multiannotator_labels, pred_probs_labeled, pred_probs_unlabeled\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.60722785, 0.74885414, 0.43063227, 0.5846288 , 0.5857348 ])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Sample of active learning scores\n",
    "active_learning_scores[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get index to relabel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lastly, we can ranks the examples by their active learning scores, and obtain the index of the examples with the lowest scores; these are the least confident examples which we will want to collect more labels for.\n",
    "\n",
    "The code cell below shows how to combine the labeled and unlabeled examples before ranking them and obtaining their respective indices to collect more labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "batch_size_to_label = 100 # you can pick how many examples to collect more labels for at each round\n",
    "\n",
    "num_labeled = len(active_learning_scores)\n",
    "active_learning_scores_combined = np.concatenate((active_learning_scores, active_learning_scores_unlabeled))\n",
    "\n",
    "to_label_idx_combined = np.argsort(active_learning_scores_combined)[:batch_size_to_label]\n",
    "to_label_idx_labeled = to_label_idx_combined[to_label_idx_combined < num_labeled]\n",
    "to_label_idx_unlabeled = (to_label_idx_combined[to_label_idx_combined >= num_labeled] - num_labeled)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([857, 910,  78,  94, 689, 860, 716, 612, 850, 382, 264, 507, 685,\n",
       "       813, 148, 274, 575, 299,  87])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Sample of indices to collect more labels for\n",
    "to_label_idx_labeled"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ### Bringing Your Own Data (BYOD)?\n",
    "> \n",
    "> You can easily replace the above with your own dataset, and obtain the active learning scores and indices to collect more labels for using the code above.\n",
    "> \n",
    "> `multiannotator_labels` should be a numpy array of pandas DataFrame where each column represents an annotator and each row represents an example. The classes should be ingeters from 0 to num_classes - 1, where examples that were not labeled by a particular annotator are represented using `np.nan`. `multiannotator_labels` should also only contain the examples that have been labeled by at least one annotator (no row should consist of only `np.nan` values).\n",
    "> \n",
    "> `X_labeled` and `X_unlabeled` should be numpy arrays that have the same number of columns. However, if you do not have any unlabeled examples, you can also pass an empty array of `None` as the value of `X_unlabeled`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Improving model accuracy over 15 rounds of multi-annotator active learning (collecting new labels) \n",
    "\n",
    "The code below shows a full demonstration of how we can repeatedly use the functions demonstrated above for multiple rounds in order to select which examples to collect new labels for, ask annotators to provide these new labels (via a noisy simulation in this example), and use the newly collected labels to get improved consensus labels used to train an improved classification model.\n",
    "\n",
    "This demonstration runs this multi-annotator active learning loop for 15 rounds, choosing 100 examples to collect more labels for each round. Each round, we use the consensus labels of each labeled example to train a classifier (here we used an `ExtraTrees` classifier) and obtain out-of-sample predicted probabilites, which is then used to compute the active learning scores for every example. We then synthetically collect new labels (this process is meant to simulate getting a new annotator to annotated a selection of examples) and repeat the active learning loop. \n",
    "\n",
    "For the first round of training, we obtain consensus labels using majority vote as we do not have any predicted probabilites yet. For subsequent rounds, we employ the classifier-predicted probabilites for each example (obtained from the previous round) with cleanlab's [CROWDLAB algorithm](https://docs.cleanlab.ai/stable/tutorials/multiannotator.html), in order to produce higher quality consensus labels that the model can be trained with.\n",
    "\n",
    "[Optional step] We also measure the model performance on a test set each round to demonstrate the improvement of the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_rounds = 15\n",
    "batch_size_to_label = 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Function to get indices of examples with the lowest active learning score to collect more labels for\n",
    "def get_idx_to_label(\n",
    "    active_learning_scores,\n",
    "    batch_size_to_label,\n",
    "    active_learning_scores_unlabeled=None,\n",
    "):\n",
    "    if active_learning_scores_unlabeled is None:\n",
    "        active_learning_scores_unlabeled = np.array([])\n",
    "\n",
    "    num_labeled = len(active_learning_scores)\n",
    "    active_learning_scores_combined = np.concatenate((active_learning_scores, active_learning_scores_unlabeled))\n",
    "\n",
    "    if batch_size_to_label > len(active_learning_scores_combined):\n",
    "        raise ValueError(\"num_examples_to_relabel is larger than the total number of examples available\")\n",
    "\n",
    "    to_label_idx_combined = np.argsort(active_learning_scores_combined)[:batch_size_to_label]\n",
    "    to_label_idx = to_label_idx_combined[to_label_idx_combined < num_labeled]\n",
    "    to_label_idx_unlabeled = (to_label_idx_combined[to_label_idx_combined >= num_labeled] - num_labeled)\n",
    "\n",
    "    return to_label_idx, to_label_idx_unlabeled"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_accuracy_arr = np.full(num_rounds, np.nan)\n",
    "\n",
    "for i in range(num_rounds):\n",
    "    # Get consensus labels\n",
    "    if i == 0:\n",
    "        consensus_labels = get_majority_vote_label(multiannotator_labels)\n",
    "    else:\n",
    "        # We can use the pred_probs from last round as the best model estimate\n",
    "        results = get_label_quality_multiannotator(\n",
    "            multiannotator_labels, \n",
    "            pred_probs_labeled,\n",
    "            calibrate_probs=True,\n",
    "        )\n",
    "        consensus_labels = results[\"label_quality\"][\"consensus_label\"]\n",
    "\n",
    "    # Train model to get out-of-sample predicted probabilites \n",
    "    pred_probs, pred_probs_unlabeled = fit_predict_proba(\n",
    "        ExtraTreesClassifier(),\n",
    "        X_labeled,\n",
    "        consensus_labels,\n",
    "        cv_n_folds=5,\n",
    "        X_unlabeled=X_unlabeled,\n",
    "    )\n",
    "\n",
    "    # Train a model on the full set of labeled data, to evaluate model accuracy for the current round.\n",
    "    # This is optional step for demonstration, you may not have ground truth labels in applications.\n",
    "    model = ExtraTreesClassifier()\n",
    "    model.fit(X_labeled, consensus_labels)\n",
    "    pred_labels = model.predict(X_test)\n",
    "    model_accuracy_arr[i] = np.mean(pred_labels == true_labels_test)\n",
    "\n",
    "    # Compute active learning scores\n",
    "    active_learning_scores, active_learning_scores_unlabeled = get_active_learning_scores(\n",
    "        multiannotator_labels, pred_probs, pred_probs_unlabeled\n",
    "    )\n",
    "\n",
    "    # Get the indices of examples to collect more labels for\n",
    "    relabel_idx, relabel_idx_unlabeled = get_idx_to_label(\n",
    "        active_learning_scores=active_learning_scores,\n",
    "        active_learning_scores_unlabeled=active_learning_scores_unlabeled,\n",
    "        batch_size_to_label=batch_size_to_label,\n",
    "    )\n",
    "\n",
    "    # Format the data for the next round of active learning, moving some unlabeled examples to the labeled pool \n",
    "    # because we are collecting labels for them.\n",
    "    # Note: you need to replace this helper function with your own function to properly update the labeled \n",
    "    # and unlabeled pools in your application.\n",
    "    (\n",
    "        multiannotator_labels, relabel_idx_combined, X_labeled, X_unlabeled, pred_probs_labeled, \n",
    "        pred_probs_unlabeled, extra_labels_labeled, extra_labels_unlabeled,\n",
    "    ) = setup_next_iter_data(\n",
    "        multiannotator_labels, relabel_idx, relabel_idx_unlabeled, X_labeled, X_unlabeled, pred_probs, \n",
    "        pred_probs_unlabeled, extra_labels_labeled, extra_labels_unlabeled,\n",
    "    )\n",
    "\n",
    "    # Collect additional labels for the examples with lowest active learning scores (indices obtained above).\n",
    "    # Here we add one new annotator who labels all of the examples in the selected batch, \n",
    "    # but you could have the existing annotators label these examples instead. \n",
    "    # Note: you need to replace this helper function with your own function to collect additional labels in your application.\n",
    "    multiannotator_labels = add_new_annotator(\n",
    "        multiannotator_labels, extra_labels_labeled, relabel_idx_combined\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the final step of each active learning round above, we added one new annotator that annotates all the examples identified with the lowest active learning scores. It is not required that these examples be labeled by one new annotator (just done here for simplicity). These examples could instead be labeled by multiple new annotators, or multiple existing annotators, or any other combination. However you choose to collect additional labels, just be sure to properly update `multiannotator_labels` to reflect who provided these annotations.\n",
    "\n",
    "Our example above utilized helper functions (eg. `setup_next_iter_data`, `add_new_annotator`) to collect new labels after each round of active learning. You *must* replace these helper functions in real applications with your own code to obtain new labels from human annotators and properly reformat the labeled/unlabeled data with the new labels."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluate results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Initial model test accuracy: 0.902\n",
      "Final model test accuracy (after 15 rounds of active learning): 0.982\n"
     ]
    }
   ],
   "source": [
    "print(f\"Initial model test accuracy: {model_accuracy_arr[0]:.3}\")\n",
    "print(f\"Final model test accuracy (after 15 rounds of active learning): {model_accuracy_arr[-1]:.3}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAGwCAYAAABB4NqyAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAABkoklEQVR4nO3deVxU5f4H8M/MwDDDLrIjiiCKC4KBkJq2yI3UTM3M+lki2uJWGV1N09S8t8i6mabmlluaZTeVbt4bRlSWGyiIS7ijgsjqwjLIMMv5/YFMkagMDnNm4PN+vc7rFYdzzvM9lTMfn/Oc55EIgiCAiIiIqBWRil0AERERkbkxABEREVGrwwBERERErQ4DEBEREbU6DEBERETU6jAAERERUavDAEREREStjo3YBVgivV6Py5cvw8nJCRKJROxyiIiIqBEEQUBFRQV8fX0hld65j4cBqAGXL1+Gv7+/2GUQERFRE+Tl5aFdu3Z3PIYBqAFOTk4Aav8FOjs7i1wNERERNUZ5eTn8/f0N3+N3wgDUgLrHXs7OzgxAREREVqYxw1c4CJqIiIhaHQYgIiIianUYgIiIiKjVYQAiIiKiVocBiIiIiFodBiAiIiJqdRiAiIiIqNVhACIiIqJWhwGIiIiIWh0GICIiImp1GICIiIio1WEAIiIiolaHi6ESERG1IoIgoKhcDa1eL2odSlsZ2jraidY+AxAREVErUK3RIelwPtbvvYBTRRVil4MnwnzxybO9RGufAYiIiKgFKyyrxqYDF7AlLRfXqjQAAKkEsJWJOwrGRiYRt31RWyciIqJmkZl7Dev3XsD3xwqg1QsAAD9XJcb1DcDTvf3horQVuUJxMQARERG1EBqdHv87VoD1ey8gK++6YX9URzeM7xeAmK5esBG558dSiP5vYfny5QgICIBCoUB0dDTS09Nve6xGo8GCBQsQFBQEhUKBsLAwJCcn1ztGp9Ph7bffRseOHaFUKhEUFIR//OMfEAShuW+FiIhIFFdVNVj20xk8sPAnvPZVFrLyrkMuk+KpiHbY+coD+PrlPnishw/Dz5+I2gO0detWJCQkYOXKlYiOjsbixYsRGxuLU6dOwdPT85bj58yZg82bN2PNmjUICQnBrl27MGLECOzbtw+9etUOpFq4cCFWrFiBjRs3onv37jh06BDi4+Ph4uKCV1991dy3SERE1GxOFpZj/Z4LSMrKh1pb+1aXh5MdnovugP+Lbg8PJ/HesrJ0EkHErpHo6Gj07t0by5YtAwDo9Xr4+/vjlVdewcyZM2853tfXF7Nnz8aUKVMM+0aOHAmlUonNmzcDAB5//HF4eXlh7dq1tz3mbsrLy+Hi4oKysjI4Ozvfyy0SERGZlE4v4KeTxVi/9zz2nbti2B/q54LxDwRgSKgv5Dats6fHmO9v0XqAampqkJGRgVmzZhn2SaVSxMTEYP/+/Q2eo1aroVAo6u1TKpXYs2eP4ee+ffti9erVOH36NDp37owjR45gz549WLRo0W1rUavVUKvVhp/Ly8ubeltERETNoqJag68PXcLGfReQe7UKACCTSvBYd2/E9wtARIc2kEjEfbPKmogWgEpLS6HT6eDl5VVvv5eXF06ePNngObGxsVi0aBEGDBiAoKAgpKamYvv27dDpdIZjZs6cifLycoSEhEAmk0Gn0+Hdd9/FmDFjbltLYmIi3nnnHdPcGBERkQldKFVhw74L+CbjEirVWgCAi9IWz0T5Y2yfAPi5KkWu0DpZ1VtgS5YswYsvvoiQkBBIJBIEBQUhPj4e69atMxzz9ddf44svvsCWLVvQvXt3ZGVlYdq0afD19UVcXFyD1501axYSEhIMP5eXl8Pf37/Z74eIiKghgiBg37krWLfnPH46VYy6wSqdPB0R3y8AI3r5wV5uVV/hFke0f3vu7u6QyWQoKiqqt7+oqAje3t4NnuPh4YGkpCRUV1fjypUr8PX1xcyZMxEYGGg4Zvr06Zg5cyaeeeYZAEBoaCguXryIxMTE2wYgOzs72NlxoBgREYmrWqPDjsP52PCX2Zof7uKB+H4d0T/YnY+5TES0ACSXyxEREYHU1FQMHz4cQO0g6NTUVEydOvWO5yoUCvj5+UGj0WDbtm14+umnDb+rqqqCVFp/8JdMJoNe5DVPiIiIbqeg7AY27b+ILem5uH5ztmZ7uQxPRbRDXN8ABHk4ilxhyyNq/1lCQgLi4uIQGRmJqKgoLF68GCqVCvHx8QCAsWPHws/PD4mJiQCAtLQ05OfnIzw8HPn5+Zg/fz70ej1mzJhhuObQoUPx7rvvon379ujevTsOHz6MRYsWYfz48aLcIxER0e1k5l7Duj3n8f3xQuhuztbcrk3tbM2jIjlbc3MSNQCNHj0aJSUlmDt3LgoLCxEeHo7k5GTDwOjc3Nx6vTnV1dWYM2cOcnJy4OjoiMGDB2PTpk1wdXU1HLN06VK8/fbbmDx5MoqLi+Hr64uXX34Zc+fONfftERFRA1RqLWxl0lb7qrZaq0Py8UKs23sBR/40W3N0RzfE9+uIv3XzgkzKx1zNTdR5gCwV5wEiImoeR/Ku45nVB6ATBIT6uaCXvyvC27uiV/s28HVRtLjxLYIg4NK1Gzicdx2Hc6/hcO51ZF8uR42udliGXCbFE+G+iO8XgO6+LiJXa/2sYh4gIiJqXdRaHaZ/cwQ3NLVTl2RcvIaMi9cMv/d0skO4f20YCvd3Rc92LnCws66vqUq1Fkfzrt8MPNeRlXcNpZU1txzn5WyHMTdna3Z35Es4YrCu/7OIiMhqLf/pLE4XVcLdUY5143rjbHHlzZBwHScKylFcocYP2UX4Ibv27WCpBOji7XwzFLmil78rgjwcIbWQx0M6vXDzHq4h62bgOV1cgb8+V7GRStDd19kQ7nq1d0V7N/sW19tlbfgIrAF8BEZEZFq/Xy7DsGV7odUL+HTMfRgc6lPv9zdqdDh+uQxZuddxOK/2UVFBWfUt13FS2CDc39UQisL928DNQW6WeyitVNer7+ilMsPEhH/m56qsfax3s8buvi5Q2MrMUmNrx0dgRERkMTQ6Pab/+yi0egGDenjfEn4AQCmXoXeAG3oHuBn2FZZVIyvvmuFx0tFL11FRrcVvZ0rx25lSw3Ed2trXjiW62cPS1cf5ngdYq7U6ZF8uN/RQHc67hryrN245zl4uQ892Lgj3b2PopfJ0VjRwRbI07AFqAHuAiIhMZ9lPZ/CvH07D1d4WKa8/2OQVyrU6PU4VVeBw7h/ja86VqG45Tm4jRQ9fZ8NYol7tXeHnqrztI6e6gcqZf3qU9eeByn8W7OlY71FWsKcjbGSt8202S2TM9zcDUAMYgIiITON0UQUe/2QPanR6LB4djuG9/Ex6/bIqDbIuXTc8msrKu26YSPDPPAwDrGt7iiDA8GZWVt71BgcquznIa8+5+aZaz3aunJfHwjEA3SMGICKie6fV6TFy5X4cybuOgSGe+CwustkH/gqCgAtXquoNTD5RUA6t/s5fdbYyCbr51O814kBl68MxQEREJLp1e8/jSN51OCls8O6IULOECYlEgo7uDujo7oAn72sHoHZ9reP5ZYbxPFk3Jx/8Y6ByG3T3deZA5VaGAYiIiEwup6QSH/1wGgDw9pBu8HYRb2CwwlaGyAA3RP5pgDURR24REZFJ6fUC3tx2FGqtHv2D3TEqsp3YJRHdggGIiIhM6vP9F3DwwjU4yGVIfNI8j76IjMUAREREJpN3tQoLk08BAGYOCkG7NvYiV0TUMAYgIiIyCUGoffR1Q6NDdEc3jInuIHZJRLfFAERERCbxZXoe9p27AoWtFAtH9rSYNbuIGsIARERE9+zy9Rt4738nAAB/f7QLAtwdRK6I6M4YgIiI6J4IgoC3dhxDpVqL+9q7Ir5fR7FLIrorBiAiIron2zPz8cupEshtpPjgqTDI+OiLrAADEBERNVlxeTXe+e53AMC0mGB08nQUuSKixmEAIiKiJhEEAbOTjqO8WotQPxe81D9Q7JKIGo0BiIjIzKo1OrFLMImdRwuQkl0EW5kEH47qCRsZv1LIevD/ViIiM9HpBSRszUKPebvw6S9nIQh3XqHckl2pVGPef2offU15uBNCvO+88jaRpWEAIiIyA0EQMPfb49h+OB9avYAPkk/hrR3HodXpxS6tSeb953dcVdUgxNsJkx/qJHY5REZjACIiMoOPU07ji7RcSCTA05HtIJUAX6bnYsLGQ6hUa8UuzyjJxwux82gBZFIJPnwqDHIbfpWQ9eH/tUREzWz93vP45KezAIB/Du+BD54Kw6rnI6GwlWL36RI8vXI/CsuqRa6yca5X1eDtb48DAF4eEIjQdi4iV0TUNAxARETNKOlwPt75LhsA8PdHOxvWx/pbNy9sfakP3B3tkF1QjhGf7sWJgnIxS22UBTuzUVKhRpCHA14dGCx2OURNxgBERNRMfj5ZjL//+wgAIL5fAKY8XH+sTJi/K3ZM7otOno4oKKvGqJX78evpEjFKbZSfTxZje2Y+JBLgg6fCoLCViV0SUZMxABERNYNDF65i0hcZ0OoFjOjlh7eHdINEcusMyf5u9tg2sS/uD3RDpVqL8RsO4uuDeSJUfGfl1Rq8teMYAGBCv46I6NBG5IqI7g0DEBGRiZ0oKMf4DQdRrdHjkRBPfPDUnVdGd7G3xcbxURjRyw9avYAZ247iX7tOWdRr8on/O4mCsmp0aGuPNx7tInY5RPeMAYiIyIRyr1Rh7Lp0lFdrEdmhDZb/332wbcQEgXY2Mix6OgyvPlL7mGzZz2fx+tYsqLXiT5q492wpvkzPBQAsHNkTSjkffZH1YwAiIjKR4opqPL8uDSUVaoR4O2HtuN5GhQWJRIKER7vgg5E9YSOVICnrMuLWpaOsStOMVd+ZSq3Fm9uOAgCev78D7g9sK1otRKbEAEREZAJlNzSIW3cQF69Uob2bPT4fHwUXpW2TrvV0b3+sj+8NJzsbHMi5iidX7EXe1SoTV9w4H+46hUvXbsDPVYk3B4WIUgNRc7CIALR8+XIEBARAoVAgOjoa6enptz1Wo9FgwYIFCAoKgkKhQFhYGJKTk+sdExAQAIlEcss2ZcqU5r4VImqFqjU6vLjxEE4UlMPd0Q6bJkTB01lxT9fsH+yBf0/qAx8XBc6VqDDi0704knfdNAU3Uvr5q9iw7wIA4P2RoXC0szFr+0TNSfQAtHXrViQkJGDevHnIzMxEWFgYYmNjUVxc3ODxc+bMwapVq7B06VJkZ2dj4sSJGDFiBA4fPmw45uDBgygoKDBsKSkpAIBRo0aZ5Z6IqPXQ6vSYuiUT6Reuwklhg8/HR6FDWweTXDvE2xlJU/qhm48zSitrMHr1fvzwe6FJrn03N2p0mPFN7Sv8oyP90T/YwyztEpmLRBD5NYPo6Gj07t0by5YtAwDo9Xr4+/vjlVdewcyZM2853tfXF7Nnz67XmzNy5EgolUps3ry5wTamTZuGnTt34syZMw2+hvpX5eXlcHFxQVlZGZyducAfETVMrxcw/Zuj2JZ5CXY2UmyaEI2ojm4mb6dSrcXULZn45VQJJBJg3uPdMK5fR5O382fv/e8EVv+aA29nBX5IGABnRdMe5xGZkzHf36L2ANXU1CAjIwMxMTGGfVKpFDExMdi/f3+D56jVaigU9buWlUol9uzZc9s2Nm/ejPHjx982/KjVapSXl9fbiIjuRBAEvPu/E9iWeQkyqQSfjrmvWcIPADja2eCzsZH4v+j2EARg/nfZWPBdNnT65vn76+Hca/jstxwAwLsjejD8UIskagAqLS2FTqeDl5dXvf1eXl4oLGy4mzc2NhaLFi3CmTNnoNfrkZKSgu3bt6OgoKDB45OSknD9+nWMGzfutnUkJibCxcXFsPn7+zf5noiodfj0l3NYu+c8AOCDkT0xsKvXXc64NzYyKd4d3gMzbw5EXrf3PCZ/kYEbNaZ9TV6t1WHGN0ehF4ARvfya/b6IxCL6GCBjLVmyBMHBwQgJCYFcLsfUqVMRHx8PqbThW1m7di0GDRoEX1/f215z1qxZKCsrM2x5eZY3CysRWY4v03Px4a5TAIA5Q7piZEQ7s7QrkUgw8cEgLH22F+QyKXb9XoRn1xxAaaXaZG0sTT2LM8WVcHe0w9zHu5nsukSWRtQA5O7uDplMhqKionr7i4qK4O3t3eA5Hh4eSEpKgkqlwsWLF3Hy5Ek4OjoiMDDwlmMvXryIH3/8ES+88MId67Czs4Ozs3O9jYioId8fK8Dsm0tCTHk4CC/0v/Wzp7kNDfPFFy9Gw9XeFll51zHi0704V1J5z9c9nl+GFbvPAQD+Obw72jjI7/maRJZK1AAkl8sRERGB1NRUwz69Xo/U1FT06dPnjucqFAr4+flBq9Vi27ZtGDZs2C3HrF+/Hp6enhgyZIjJayei1mfv2VK89lUW9ALwbFR7/F3EJSF6B7hh+6S+aO9mj7yrN/Dkp/uQfv5qk6+n0ekx/Zuj0OkFDAn1wWM9fExYLZHlEf0RWEJCAtasWYONGzfixIkTmDRpElQqFeLj4wEAY8eOxaxZswzHp6WlYfv27cjJycFvv/2Gxx57DHq9HjNmzKh3Xb1ej/Xr1yMuLg42Npy7gojuzdFL1/HS54dQo9NjUA9v/HN4j0a9VdqcAj0csWNyX/Rq74qyGxo891kavs3Kb9K1VvxyDicKytHG3hbvDOtu4kqJLI/oyWD06NEoKSnB3LlzUVhYiPDwcCQnJxsGRufm5tYb31NdXY05c+YgJycHjo6OGDx4MDZt2gRXV9d61/3xxx+Rm5uL8ePHm/N2iKgFOltciXHrD0JVo0O/Tm2x+JlwyO6wuKk5tXW0w5cv3o/Xt2bh++OFeO2rLFy6dgOTHwpqdEA7VViBpT+dAQDMf6I73B3tmrNkIosg+jxAlojzABFRncvXb+CpFftwuawaYe1c8MWL91vkjMh6vYDE709gzW+1b6Y9G+WPfwzrAZu7LMSq1ekxcsU+HLlUhpiuXlgzNkL0ni2iprKaeYCIiCzZVVUNnl+bhstl1Qj0cMD6+CiLDD8AIJVKMHtINywY1h1SCfBleh4mbDyESrX2jud9tuc8jlwqg5PCBu+OEP+xHpG5MAARETWgUq1F/Pp0nCtRwcdFgU0TouFmBW9Fje0TgNXPR0JpK8Pu0yUYtXI/CsuqGzz2XEklFqWcBgC8/Xg3eN3j+mVE1oQBiIjoL9RaHSZuysCRS2VoY2+LTROi4OeqFLusRovp5oWtL98Pd0c7nCgox/Dle3GioP4M9zq9gBnfHEWNVo8BnT0wykxzGRFZCgYgIqI/0ekFJGw9gj1nS2Evl2F9fBQ6eTqJXZbRerZzxY7JfdHJ0xGF5dUYtXI/fj1dYvj95/svIOPiNTjIZUh8MpSPvqjVYQAiIrpJEAS8/e1x/PdYAWxlEqx+PhLh/q5il9Vk/m722DapL/oEtq19pLfhILYezMXFKyp8kFw7k/WswV2tqneLyFQYgIiIblqUchpb0nIhkQBLnumFB4LdxS7pnrkobbFxfBSe7OUHnV7Am9uOYfSqA7ih0eH+QDf8X1R7sUskEgUDEBERgLV7zmPpT2cBAP8c3gODQ1vOTMhyGyk+ejoMrw4MBgAUlldDaSvDwpE9IbWQ+YyIzM0y3+ckIjKj7ZmX8I+d2QCA6bFdMCa6g8gVmZ5EIkHC3zqjvZs9lv98Fq8O7IQObR3ELotINAxARNSq/XSyCNO/OQoAGN+vIyY/FCRyRc3rqYh2eIpvfBHxERgRtV4HL1zFpM2Z0OkFPNnLD3OGdOXbUEStBAMQEbVKJwrKMX7DQai1ejwS4omFT3E8DFFrwgBERK1O7pUqjF2XjopqLXoHtMHy/7sPtndZM4uIWhb+iSeiVqW4ohrPrU1DSYUaId5O+CyuN5RymdhlEZGZMQARUashCAImbc5E7tUqtHezx+fjo+CitBW7LCISAQMQEbUa+85dQcbFa1DYSrFpQhQ8ufgnUavFAERErcbK3ecAAKMj/TkHDlErxwBERK1C9uVy/HamFFIJ8EL/QLHLISKRMQARUauw+tfa3p8hPX3h72YvcjVEJDYGICJq8S5dq8J3RwsAAC8PYO8PETEAEVErsHbPeej0Avp1aosefi5il0NEFoABiIhatOtVNfgqPQ8A8PKAlr3OFxE1HgMQEbVomw9cxA2NDl19nNE/2F3scojIQjAAEVGLVa3RYcO+CwCAiQ8GcqFTIjJgACKiFmtb5iWUVtbAz1WJwaE+YpdDRBaEAYiIWiSdXsCaX3MAABMe6MjFTomoHn4iEFGLlJJdiAtXquCitMXo3v5il0NEFoYBiIhaHEEQsGJ3be/P2D4d4GBnI3JFRGRpGICIqMVJP38VR/KuQ24jRVzfALHLISILxABERC3Oqptjf0ZFtIO7o53I1RCRJWIAIqIW5XRRBX46WQwJFz0lojtgACKiFmX1zd6fx7p7o6O7g8jVEJGlEj0ALV++HAEBAVAoFIiOjkZ6evptj9VoNFiwYAGCgoKgUCgQFhaG5OTkW47Lz8/Hc889h7Zt20KpVCI0NBSHDh1qztsgIgtQUHYD32blAwBe4qKnRHQHogagrVu3IiEhAfPmzUNmZibCwsIQGxuL4uLiBo+fM2cOVq1ahaVLlyI7OxsTJ07EiBEjcPjwYcMx165dQ79+/WBra4vvv/8e2dnZ+Oijj9CmTRtz3RYRiWT93gvQ6AREd3RDr/b8M09EtycRBEEQq/Ho6Gj07t0by5YtAwDo9Xr4+/vjlVdewcyZM2853tfXF7Nnz8aUKVMM+0aOHAmlUonNmzcDAGbOnIm9e/fit99+a3Jd5eXlcHFxQVlZGZydnZt8HSIyn7IbGvR7/ydUqrVYP643Hg7xFLskIjIzY76/ResBqqmpQUZGBmJiYv4oRipFTEwM9u/f3+A5arUaCoWi3j6lUok9e/YYfv7Pf/6DyMhIjBo1Cp6enujVqxfWrFlzx1rUajXKy8vrbURkXbak5aJSrUVnL0c81MVD7HKIyMKJFoBKS0uh0+ng5eVVb7+XlxcKCwsbPCc2NhaLFi3CmTNnoNfrkZKSgu3bt6OgoMBwTE5ODlasWIHg4GDs2rULkyZNwquvvoqNGzfetpbExES4uLgYNn9/zhpLZE3UWh3W7z0PAHhpQBAXPSWiuxJ9ELQxlixZguDgYISEhEAul2Pq1KmIj4+HVPrHbej1etx3331477330KtXL7z00kt48cUXsXLlytted9asWSgrKzNseXl55rgdIjKRbw9fRnGFGt7OCjwR5it2OURkBUQLQO7u7pDJZCgqKqq3v6ioCN7e3g2e4+HhgaSkJKhUKly8eBEnT56Eo6MjAgP/eNvDx8cH3bp1q3de165dkZube9ta7Ozs4OzsXG8jIuug1wtY9es5ALWLnsptrOrvdUQkEtE+KeRyOSIiIpCammrYp9frkZqaij59+tzxXIVCAT8/P2i1Wmzbtg3Dhg0z/K5fv344depUveNPnz6NDh06mPYGiMgipJ4sxrkSFZzsbPBMFB9fE1HjiLpCYEJCAuLi4hAZGYmoqCgsXrwYKpUK8fHxAICxY8fCz88PiYmJAIC0tDTk5+cjPDwc+fn5mD9/PvR6PWbMmGG45uuvv46+ffvivffew9NPP4309HSsXr0aq1evFuUeiah5rdpd2/sz5v4OcFLYilwNEVkLUQPQ6NGjUVJSgrlz56KwsBDh4eFITk42DIzOzc2tN76nuroac+bMQU5ODhwdHTF48GBs2rQJrq6uhmN69+6NHTt2YNasWViwYAE6duyIxYsXY8yYMea+PSJqZhkXr+LQxWuQy6SI7xcgdjlEZEVEnQfIUnEeICLr8NLnh/BDdhFGR/pj4VM9xS6HiERmFfMAERHdi7PFlUg5UfsSxYtc9oKIjMQARERW6bPfciAIQExXL3TydBS7HCKyMgxARGR1isursT2zdtHTiQ+y94eIjMcARERWZ8O+C6jR6RHRoQ0iA9zELoeIrBADEBFZlUq1FpsOXAQAvMyxP0TURAxARK3MjRodrPnlz6/Sc1FRrUWghwNiunrd/QQiogYwABG1EtUaHf616xRC5+/CuPUHodbqxC7JaDVaPdbuubnoaf9ASKVc9JSImoYBiKgVyLh4DUM++Q3Lfj4LrV7A7tMleO3LLOj01tUT9N2Ryygoq4aHkx2G9/ITuxwismIMQEQtWFWNFu989zueWrkP50pUcHe0wxt/6wy5TIrk3wsxJ+mY1TwOEwQBq3/NAQDE9wuAwlYmckVEZM1EXQqDiJrP3rOlmLn9KPKu3gAAPBXRDnOGdIWrvRydPB0xZUsmvkzPQxt7OWY8FiJytXf3y+kSnCqqgINchjHRXNyYiO4NAxBRC1N2Q4PE/53AVwfzAAB+rkq892QoHuzsYThmUKgP3h0Rilnbj+HTX87BzUGOF/pb9htVdYue/l90e7gouegpEd0bBiCiFuTH7CLMTjqGonI1AGBsnw6Y8VgIHO1u/aP+bFR7XKuqwQfJp/DP/55AG3s5Rka0M3fJjXIk7zoO5FyFjVSC+H4dxS6HiFoABiCiFuBKpRrvfJeN/xy5DADo6O6AhSN7IqrjnScJnPRgEK5W1uCzPecxY9tRuChtEdPN8l4trxv780S4L3xdlSJXQ0QtAQdBE1kxQRDwbVY+/vbxr/jPkcuQSoCXHwzE96/1v2v4AQCJRIK3BnfFyPvaQacXMGVLJtJyrpih8sa7UKrC98cLAAAvceJDIjIR9gARWanCsmrMSTqGH08UAwBCvJ3wwVM90bOdq1HXkUolWDgyFGU3avDjiWK8sPEQvnr5fnT3dWmGqo332Z4c6AXg4S4eCPF2FrscImoh2ANEZGUEQcBX6bn428e78eOJYtjKJHg9pjP+M/UBo8NPHRuZFMv+7z5EBbihQq1F3LqDuFCqMm3hTVBaqca/D10CALz8YJDI1RBRS8IARGRF8q5W4bm1aZi5/RgqqrUI83fFzlf647WYYMht7u2Ps8JWhs/GRaKrjzNKK9V4fl0aisurTVR503y+7wLUWj3C2rkguhGP9IiIGsvoT0yVSvy/FRK1Njq9gHV7zuPRj3/F3rNXoLCVYvbgrtg+qS+6eDuZrB1nhS02ju+NDm3tkXf1BsauS0dZlcZk1zdGVY0Wn9ctevpgECQSLntBRKZjdADy8vLC+PHjsWfPnuaoh4j+4mxxBUat3IcFO7NxQ6PD/YFuSH5tAF4cEAhZM6yF5emkwOYJ0fB0ssPJwgqM33gQN2rMv27Y1wfzcL1Kgw5t7RHb3dvs7RNRy2Z0ANq8eTOuXr2KRx55BJ07d8b777+Py5cvN0dtRK2aRqfH8p/PYvCSPcjMvQ5HOxu8O6IHtrxwPwLcHZq1bX83e3w+IQrOChtkXLyGSV9kQKPTN2ubf6bV6bHmt9pFT1/s3zxBj4haN6MD0PDhw5GUlIT8/HxMnDgRW7ZsQYcOHfD4449j+/bt0Gq1zVEnUatyPL8Mw5btxYe7TqFGp8fDXTzww+sDMCa6g9lWQA/xdsa6cb2hsJXil1MlmP7vI9CbafHU/x4rQP71G2jrIMdTFjo5IxFZtyaPmvTw8EBCQgKOHj2KRYsW4ccff8RTTz0FX19fzJ07F1VVVaask6hVqNbo8OGukxi2fC+yC8rham+Lj0eHYd243qJMABgZ4IYVYyJgI5UgKesyFuzMbvbFUwVBwKrdtRMfxvXloqdE1DyaPA9QUVERNm7ciA0bNuDixYt46qmnMGHCBFy6dAkLFy7EgQMH8MMPP5iyVqIWLePiVcz45ijOldS+aDAk1Afzn+gODyc7Uet6OMQT/xoVhmlbs7Bh3wW0dZDjlYHBzdbe3rNXkF1QDqWtDM/fz0VPiah5GB2Atm/fjvXr12PXrl3o1q0bJk+ejOeeew6urq6GY/r27YuuXbuask6iFkul1uLDXaewcf8FCALg4WSHfwzrgcd6WM7A3+G9/HCtqgbvfJeNj1JOw9VB3mzhZNWvtYueju7tjzYO8mZpg4jI6AAUHx+PZ555Bnv37kXv3r0bPMbX1xezZ8++5+KIWro9Z0oxc/tRXLp2AwAwKqId5gzpBhd7y1vtPL5fR1xT1eCTn85i7rfH0cbeFo/39DVpG8fzy/DbmVLIpBJMeICLnhJR8zE6ABUUFMDe3v6OxyiVSsybN6/JRRG1dGU3NHjvvyew9VAeAMDPVYnEJ0MxoLOHyJXd2et/64yrVTXYfCAXr2/NgrPC1qQ11y16OiTUB/5ud/6cISK6F0YPgv7ll1+wa9euW/bv2rUL33//vUmKImrJUrKL8OjHuw3hJ65PB+x6fYDFhx+gdvHUd57ogcd7+kCjE/Dypgwczr1mkmvnXa3Cf49x0VMiMg+jA9DMmTOh0906KZogCJg5c6ZJiiJqiXR6Ae989zte/PwQisrVCHR3wL8n9sE7w3rA0c561iWWSSVY9HQ4+ge744ZGh/gNB3GmqOKer7t2z3no9AL6B7ujh59lLMRKRC2X0QHozJkz6Nat2y37Q0JCcPbsWZMURdTS3KjRYeLmDKzfewEA8PKDgfjfa/3RO8A617eS20ix8rkIhPu74nqVBs+vTcela02f+uKaqgZbD9b2iL08gIueElHzMzoAubi4ICcn55b9Z8+ehYND885OS2SNSirUeGb1fqRkF0FuI8Wy/+uFWYO6Wv38Ng52Nlg/rjc6eTqisLwaY9em40qluknX2nTgIm5odOju64x+ndqauFIiolsZHYCGDRuGadOm4dy5c4Z9Z8+exRtvvIEnnnjCpMURWbuzxRUY8eleHLlUhjb2ttjyQrTJ35wSUxsHOTZNiIKfqxI5pSqMW38QFdXGLZ5ardFhw74LAGrH/nDRUyIyB6MD0AcffAAHBweEhISgY8eO6NixI7p27Yq2bdviX//6V5OKWL58OQICAqBQKBAdHY309PTbHqvRaLBgwQIEBQVBoVAgLCwMycnJ9Y6ZP38+JBJJvS0kJKRJtRE11YGcK3jy0324dO0GOrS1x/bJ/RBppY+87sTHRYlNE6Lg5iDHsfwyvPR5Bqo1jV889d8Zl3BVVQM/VyWGhPo0Y6VERH8weuSli4sL9u3bh5SUFBw5cgRKpRI9e/bEgAEDmlTA1q1bkZCQgJUrVyI6OhqLFy9GbGwsTp06BU9Pz1uOnzNnDjZv3ow1a9YgJCQEu3btwogRI7Bv3z706tXLcFz37t3x448//nGjNtYzyJSsX9LhfEz/5gg0OgH3tXfFmrGRaOso7ozOzSnQwxEb46PwzOr92J9zBa99dRjL/+8+2Mju/HcsnV7AZ7/VPlJ/sX/Hux5PRGQqEqG5F/a5i+joaPTu3RvLli0DAOj1evj7++OVV15p8K2yukkWp0yZYtg3cuRIKJVKbN68GUBtD1BSUhKysrKaVFN5eTlcXFxQVlYGZ2fnJl2DWidBELD857P41w+nAQCDQ72x6Olwqx/v01j7zpVi3LqDqNHpMTrSH++PDL3jI63/HSvA5C8y4Wpvi30zH4G9nH9RIaKmM+b7u0mfNiqVCrt370Zubi5qamrq/e7VV19t9HVqamqQkZGBWbNmGfZJpVLExMRg//79DZ6jVquhUCjq7VMqldizZ0+9fWfOnIGvry8UCgX69OmDxMREtG/f/rbXVKv/GLxZXl7e6HsgqqPR6TFnx3HD/D4vDwjEm4+FmG31dkvQN8gdnzzbC5O/yMDWQ3lwc5Tjzccafvxcu+hp7VjCsX0CGH6IyKyM/sQ5fPgwBg8ejKqqKqhUKri5uaG0tBT29vbw9PQ0KgCVlpZCp9PBy8ur3n4vLy+cPHmywXNiY2OxaNEiDBgwAEFBQUhNTcX27dvrzU0UHR2NDRs2oEuXLigoKMA777yD/v374/jx43BycrrlmomJiXjnnXcaXTfRX1VUazD5i0z8dqYUUgnwzhPd8XyfALHLEsVjPbyR+GQo3tx2DCt+OYc29rZ4qYFX2w/kXMWRS2Wws5Eirg8XPSUi8zL6gfvrr7+OoUOH4tq1a1AqlThw4AAuXryIiIiIJg+CNsaSJUsQHByMkJAQyOVyTJ06FfHx8ZBK/7iVQYMGYdSoUejZsydiY2Pxv//9D9evX8fXX3/d4DVnzZqFsrIyw5aXl9fs90EtR0HZDYxauR+/nSmF0laGNWMjW234qTO6d3tDz897/zuJfx+69c9U3aKnoyLbtejxUURkmYwOQFlZWXjjjTcglUohk8mgVqvh7++PDz74AG+99ZZR13J3d4dMJkNRUVG9/UVFRfD2bnglbA8PDyQlJUGlUuHixYs4efIkHB0dERh4+6nzXV1d0blz59tO1GhnZwdnZ+d6G1Fj/H65DMOX78XJwgp4ONnh65f7YGBXr7uf2ApMfDDQsKTFzO3H8MPvhYbfnSwsxy+nSiCVAC88wGUviMj8jA5Atra2ht4WT09P5ObmAqh9O8zYnhO5XI6IiAikpqYa9un1eqSmpqJPnz53PFehUMDPzw9arRbbtm3DsGHDbntsZWUlzp07Bx8fvmJLpvPLqWI8vXI/isrV6OzliB2T+yK0HZdwqCORSDBrUAhGRbSDTi9g6peHcSDnCoA/Fj0d1MMHAe6cQJWIzM/oMUC9evXCwYMHERwcjAcffBBz585FaWkpNm3ahB49ehhdQEJCAuLi4hAZGYmoqCgsXrwYKpUK8fHxAICxY8fCz88PiYmJAIC0tDTk5+cjPDwc+fn5mD9/PvR6PWbMmGG45t///ncMHToUHTp0wOXLlzFv3jzIZDI8++yzRtdH1JAtabl4+9vj0OkF9A1qixXPRcBFaSt2WRZHIpEg8clQXL+hQUp2EV7ceAgfjw7Hf7IuA+Cip0QkHqMD0HvvvYeKitqFD999912MHTsWkyZNQnBwMNatW2d0AaNHj0ZJSQnmzp2LwsJChIeHIzk52TAwOjc3t974nurqasyZMwc5OTlwdHTE4MGDsWnTJri6uhqOuXTpEp599llcuXIFHh4eeOCBB3DgwAF4eFj+attk2fR6AR/+cAorfqkdvzLyvnZIfDIUchvOX3M7NjIplj7bC3Hr0pF2/ipe+PwQAOD+QDeE+buKWxwRtVpGzQMkCALy8vLg6el5y6voLQnnAaKGVGt0mP7NUXx3pLb3YlpMMF4bGMylGxqpvFqDZ1YdQHZB7TQT6+N74+Eut052SkTUVMZ8fxv111ZBENCpUye+JUWtzjVVDZ5fm4bvjlyGjVSCf40Kw7SYzgw/RnBW2GLj+Cj0au+KR7t54aHO7JElIvEY9QhMKpUiODgYV65cQXBwcHPVRGRRLl5RIX79QeSUquCksMGq5yLQt5O72GVZJQ8nO+yY3E/sMoiIjH8L7P3338f06dNx/Pjx5qiHyKJk5l7Dk5/uQ06pCn6uSmyb1Jfhh4ioBTB6LbA2bdqgqqoKWq0WcrkcSqWy3u+vXr1q0gLFwDFABADJxwvw2ldZUGv16OHnjHVxveHp3HLHvhERWbtmXQts8eLFTa2LyCoIgoC1e87j3f+dgCAAj4R4YumzveBgx7WqiIhaCqM/0ePi4pqjDiKLoNML+MfObGzYdwEA8Pz9HTBvaDfYyPiaOxFRS2J0AKqb+fl2brfiOpGlq6rR4tUvs/DjidqlWWYP7ooX+nfkm15ERC2Q0QEoICDgjl8If16VnchaFFdU44WNh3D0UhnkNlIsHh2OwaFcOoWIqKUyOgAdPny43s8ajQaHDx/GokWL8O6775qsMCJzOVtcgbh1B5F//QbcHORYMzYCER3cxC6LiIiakdEBKCws7JZ9kZGR8PX1xYcffognn3zSJIURmcO+c6WYuCkD5dVaBLS1x4b4KC7OSUTUCpjstZYuXbrg4MGDprocUbPbcfgSZnxzFBqdgIgObbBmbCTcHORil0VERGZgdAAqLy+v97MgCCgoKMD8+fM5OzRZBUEQsPSns1iUchoAMCTUBx89HQaFrUzkyoiIyFyMDkCurq63DIIWBAH+/v746quvTFYYUXPQ6PR4a/sx/DvjEgDg5QGBePOxEEilfNOLiKg1MToA/fTTT/UCkFQqhYeHBzp16gQbG04UR5ZtxjdHseNwPqQS4J1hPfD8/R3ELomIiERgdGJ56KGHmqEMoua36/dC7DicD5lUgtXPR2BgVy+xSyIiIpEYPb1tYmIi1q1bd8v+devWYeHChSYpisjUyqo0mJNUu4DvSwMCGX6IiFo5owPQqlWrEBIScsv+7t27Y+XKlSYpisjU/vHfbJRUqBHo4YDXBnKwPhFRa2d0ACosLISPz60z5Hp4eKCgoMAkRRGZ0s+nivFNxiVIJMCHT/Xk215ERGR8APL398fevXtv2b937174+vqapCgiU6mo1uCt7ccAAPF9O3KGZyIiAtCEQdAvvvgipk2bBo1Gg0ceeQQAkJqaihkzZuCNN94weYFE9yLx+5MoKKtGh7b2mB7bRexyiIjIQhgdgKZPn44rV65g8uTJqKmpAQAoFAq8+eabmDlzpskLJGqqfWdLsSUtFwDw/pM9oZTz0RcREdWSCIIgNOXEyspKnDhxAkqlEsHBwbCzszN1baIpLy+Hi4sLysrK4OzsLHY51AQqtRaPLfkVeVdv4Ln72+Ofw0PFLomIiJqZMd/fRvcAlZWVQafTwc3NDb179zbsv3r1KmxsbBgYyCJ8uOsU8q7egJ+rEjMHdRW7HCIisjBGD4J+5plnGlzy4uuvv8YzzzxjkqKI7sXBC1excf8FAEDik6FwtOMM5UREVJ/RASgtLQ0PP/zwLfsfeughpKWlmaQooqaq1ujw5jdHIQjA05HtMKCzh9glERGRBTI6AKnVami12lv2azQa3LhxwyRFETXVxymnkVOqgqeTHWYP6SZ2OUREZKGMDkBRUVFYvXr1LftXrlyJiIgIkxRF1BRZedex5rccAMB7I0LhorQVuSIiIrJURg+O+Oc//4mYmBgcOXIEAwcOBFA7D9DBgwfxww8/mLxAosZQa3WY8c0R6AVgeLgvYrpxrS8iIro9o3uA+vXrh/3798Pf3x9ff/01vvvuO3Tq1AlHjx5F//79m6NGorta9tNZnC6qhLujHPOGdhe7HCIisnBNej0mPDwcX3zxRb19er0eO3fuxOOPP26Swoga63h+GT795RwAYMGwHmjjIBe5IiIisnT3/H7w2bNnsW7dOmzYsAElJSXQaDSmqIuoUTQ6PWZ8cxQ6vYBBPbwxOPTWhXqJiIj+yuhHYABw48YNfP755xgwYAC6dOmCffv2Ye7cubh06VKTili+fDkCAgKgUCgQHR2N9PT02x6r0WiwYMECBAUFQaFQICwsDMnJybc9/v3334dEIsG0adOaVBtZtlW7zyG7oByu9rZYMKyH2OUQEZGVMCoAHTx4EC+//DK8vb2xePFiDBs2DBKJBJ9++ikmTpwILy/jB55u3boVCQkJmDdvHjIzMxEWFobY2FgUFxc3ePycOXOwatUqLF26FNnZ2Zg4cSJGjBiBw4cPN1jvqlWr0LNnT6PrIst3uqgCn6SeBQDMH9odHk4tZzkWIiJqXo0OQD179sSoUaPQtm1b7Nu3D5mZmXjjjTcgkUjuqYBFixbhxRdfRHx8PLp164aVK1fC3t4e69ata/D4TZs24a233sLgwYMRGBiISZMmYfDgwfjoo4/qHVdZWYkxY8ZgzZo1aNOmzT3VSJZHq9Nj+r+PoEanx8AQTwwL9xW7JCIisiKNDkCnTp3CgAED8PDDD6NbN9NMMFdTU4OMjAzExMT8UZBUipiYGOzfv7/Bc9RqNRQKRb19SqUSe/bsqbdvypQpGDJkSL1r345arUZ5eXm9jSzbur3nceRSGZwUNnh3ROg9B3EiImpdGh2AcnJy0KVLF0yaNAnt2rXD3//+dxw+fPievnhKS0uh0+lueXTm5eWFwsLCBs+JjY3FokWLcObMGej1eqSkpGD79u0oKCgwHPPVV18hMzMTiYmJjaojMTERLi4uhs3f37/J90TNL6ekEh/9cBoA8PaQbvB2UdzlDCIiovoaHYD8/Pwwe/ZsnD17Fps2bUJhYSH69esHrVaLDRs24PTp081Zp8GSJUsQHByMkJAQyOVyTJ06FfHx8ZBKa28lLy8Pr732Gr744otbeopuZ9asWSgrKzNseXl5zXkLdA/0egFvbjsKtVaP/sHuGBXZTuySiIjICjXpLbBHHnkEmzdvRkFBAZYtW4affvoJISEhRg82dnd3h0wmQ1FRUb39RUVF8Pb2bvAcDw8PJCUlQaVS4eLFizh58iQcHR0RGBgIAMjIyEBxcTHuu+8+2NjYwMbGBrt378Ynn3wCGxsb6HS6W65pZ2cHZ2fnehtZps/3X8DBC9fgIJch8Uk++iIioqZpUgCq4+LigsmTJ+PQoUPIzMzEQw89ZNT5crkcERERSE1NNezT6/VITU1Fnz597niuQqGAn58ftFottm3bhmHDhgEABg4ciGPHjiErK8uwRUZGYsyYMcjKyoJMJjP6Psky5F2twsLkUwCAmYO7ol0be5ErIiIia3XPEyHWCQ8PxyeffGL0eQkJCYiLi0NkZCSioqKwePFiqFQqxMfHAwDGjh0LPz8/w3ietLQ05OfnIzw8HPn5+Zg/fz70ej1mzJgBAHByckKPHvXng3FwcEDbtm1v2U/WQxBqH33d0OgQ3dENY6Lai10SERFZMZMFoKYaPXo0SkpKMHfuXBQWFiI8PBzJycmGgdG5ubmG8T0AUF1djTlz5iAnJweOjo4YPHgwNm3aBFdXV5HuwLpUa3SwlUkhk1rXo6Mv0/Ow79wVKGylWDiyJ6RWVj8REVkWiSAIgthFWJry8nK4uLigrKysRY0HqqjW4OF/7YZcJsG7T4bi4S6eYpfUKJev38CjH/+KSrUWc4Z0xQv9A8UuiYiILJAx39/3NAaIrEvGxWsorVTjclk14tcfRMLWLFxT1Yhd1h0JgoC3dhxDpVqL+9q7Ir5fR7FLIiKiFoABqBU5klcGAPB2VkAiAbYfzsffPt6N/x0ruMuZ4tmWmY9fTpVAbiPFB0+FWd2jOyIiskyNGgNkzODmV199tcnFUPM6euk6AODlBwMR5u+KGd8cxdniSkz+IhOPdffGguHd4elkOZMKFpdXY8F3vwMApsUEo5Ono8gVERFRS9GoMUAdOzbusYNEIkFOTs49FyW2ljgGSBAE9H73R5RW1mDbpL6I6NAGaq0Oy346ixW/nINWL8BFaYu3H++Gkff5iT6/jiAIeGlTBlKyixDq54Idk/vCRsYOSyIiuj1jvr8b1QN0/vx5kxRG4rlcVo3SyhrYSCXo7lv7P4WdjQxvPNoFg3r4YMa2IzieX46///sI/nPkMhKfDIWfq1K0enceLUBKdhFsZRJ8OKonww8REZlUk79VampqcOrUKWi1WlPWQ83kSN51AECIjxMUtvUng+zm64ykyf0w47EukNtI8evpEjy6aDc27b8Avd78LwleqVRj3n9qH31NebgTQrxbRi8cERFZDqMDUFVVFSZMmAB7e3t0794dubm5AIBXXnkF77//vskLJNM4cnP8T892rg3+3kYmxeSHOuH71/ojskMbqGp0ePvb3/HM6gPIKak0X6EA5v3nd1xV1SDE2wmTH+pk1raJiKh1MDoAzZo1C0eOHMEvv/xSb7HRmJgYbN261aTFkenU9QCF3yYA1QnycMTXL/fBO090h71chvQLVzFoyW9YtfsctDp9s9eZfLwQO48WQCaV4MOnwiC34aMvIiIyPaO/XZKSkrBs2TI88MAD9QbKdu/eHefOnTNpcWQaOr2AY5dqX4Hv6e9y1+OlUgni+gZg17QB6B/sDrVWj8TvT+LJFftwsrC82eq8XlWDt789DgB4eUAgQtvdvVYiIqKmMDoAlZSUwNPz1hmEVSqV6G8OUcNySiqhqtHBXi5DsKdTo8/zd7PH5+Oj8MFTPeGssMHRS2V4/JM9WJRyGjVa0/cGLdiZjZIKNYI8HPDqwGCTX5+IiKiO0QEoMjIS//3vfw0/14Wezz777K4ruJM4sm4+/urh62L0RIISiQRPR/rjx4QH8Wg3L2j1Aj5JPYPHl/5muK4p/HyyGNsz8yGRAB88FXbLQG0iIiJTMnox1Pfeew+DBg1CdnY2tFotlixZguzsbOzbtw+7d+9ujhrpHh29+fgrrBGPv27H01mBVc9H4L/HCjDv299xuqgST366FxMe6IiEv3WBUt70wFJercFbO44BAMb364iIDm2afC0iIqLGMLoH6IEHHkBWVha0Wi1CQ0Pxww8/wNPTE/v370dERERz1Ej36G5vgDWWRCLB4z19kZLwIEb08oNeANb8dh6DlvyKAzlXmnzdxP+dREFZNTq0tcffH+1yTzUSERE1BleDb0BLmglardWhx7xd0OgE/DbjYfi72Zvs2j+dLMJb24+jsLwaADAmuj1mDgqBk8K20dfYe7YUYz5LAwB89dL9uD+wrcnqIyKi1sXkM0GXlzf+zR9rDwwtzYmCCmh0AtrY26JdG9PO7PxIiBd+SHDD+9+fxJa0XHyRloufTxbj3SdD8XCXWwfK/5VKrcWb244CAJ6/vwPDDxERmU2jApCrq2uj3/DS6XT3VBCZVt0CqGH+jf9vaAxnhS3eGxGKx3v6YNb2Y7h4pQrx6w/iyV5+ePvxbmjjIL/tuR/uOoVL127Az1WJNweFmLw2IiKi22lUAPr5558N/3zhwgXMnDkT48aNM7z1tX//fmzcuBGJiYnNUyU1Wd2bWvc6/udu+ga5I/m1Afjoh1NYt/c8th/Ox69nSrBgWA8MDvW55fj081exYd8FAMD7I0PhaGf0eHwiIqImM3oM0MCBA/HCCy/g2Wefrbd/y5YtWL16NX755RdT1ieKljQGKGbRbpwtrsS6cZF4JMTLLG0ezr2GGd8cxZni2iU0HuvujQXDusPTuXbm8Bs1Ogz+5DecL1VhdKQ/Fj7V0yx1ERFRy2bM97fRb4Ht378fkZGRt+yPjIxEenq6sZejZlRRrcG5m+t4NXcP0J/1at8GO199AK8+0gk2UgmSfy9EzKLd+PehPAiCgI9/PI3zpSp4OdvhrSFdzVYXERFRHaMDkL+/P9asWXPL/s8++wz+/v4mKYpM41h+GQQB8HNVwt3Rzqxt29nIkPBoF/xn6gMI9XNBebUW0785itGrDuCz33IAAO+NCIWLsvFvjBEREZmK0QMvPv74Y4wcORLff/89oqOjAQDp6ek4c+YMtm3bZvICqemO5NVOgBju7ypaDd18nbFjcl98tuc8FqWcRvqFqwCAEb38MLCreR7JERER/ZXRPUCDBw/GmTNnMHToUFy9ehVXr17F0KFDcfr0aQwePLg5aqQmOmIYAC3uoqI2MikmPhiE5Nf6Y0BnD4T7u2Lu491ErYmIiFq3Jr16065dO7z33numroVM7M+vwFuCQA9HfD4+SuwyiIiImhaArl+/jrVr1+LEiRMAgO7du2P8+PFwcRG3p4H+UFxRjctl1ZBIgB5+/O9CRET0Z0Y/Ajt06BCCgoLw8ccfGx6BLVq0CEFBQcjMzGyOGqkJjt4c/xPs6cg5doiIiP7C6G/G119/HU888QTWrFkDG5va07VaLV544QVMmzYNv/76q8mLJOOZagFUIiKilsjoAHTo0KF64QcAbGxsMGPGjAbnByJxHLlU2wNkKeN/iIiILInRj8CcnZ2Rm5t7y/68vDw4OTmZpCi6N4Ig/DEAWuQ3wIiIiCyR0QFo9OjRmDBhArZu3Yq8vDzk5eXhq6++anB5DBJH7tUqXK/SQC6TIsTbupfyICIiag5GPwL717/+BYlEgrFjx0Kr1QIAbG1tMWnSJLz//vsmL5CMV7cAajdfZ8htjM64RERELZ7RAUgul2PJkiVITEzEuXPnAABBQUGwt7c3eXHUNEfrxv/w8RcREVGDmtw9YG9vj9DQUISGht5z+Fm+fDkCAgKgUCgQHR19x0VVNRoNFixYgKCgICgUCoSFhSE5ObneMStWrEDPnj3h7OwMZ2dn9OnTB99///091WhN6maA5gBoIiKihjW6B2j8+PGNOm7dunVGFbB161YkJCRg5cqViI6OxuLFixEbG4tTp07B09PzluPnzJmDzZs3Y82aNQgJCcGuXbswYsQI7Nu3D7169QJQO1P1+++/j+DgYAiCgI0bN2LYsGE4fPgwunfvblR91kar0+P45doeIL4CT0RE1DCJIAhCYw6USqXo0KEDevXqhTudsmPHDqMKiI6ORu/evbFs2TIAgF6vh7+/P1555RXMnDnzluN9fX0xe/ZsTJkyxbBv5MiRUCqV2Lx5823bcXNzw4cffogJEybctaby8nK4uLigrKwMzs7WNYg4+3I5Bn/yG5zsbHBk3qOQSiVil0RERGQWxnx/N7oHaNKkSfjyyy9x/vx5xMfH47nnnoObm9s9FVpTU4OMjAzMmjXLsE8qlSImJgb79+9v8By1Wg2FQlFvn1KpxJ49exo8XqfT4d///jdUKhX69Olz22uq1WrDz+Xl5cbeisWomwAxtJ0Lww8REdFtNHoM0PLly1FQUIAZM2bgu+++g7+/P55++mns2rXrjj1Cd1JaWgqdTgcvL696+728vFBYWNjgObGxsVi0aBHOnDkDvV6PlJQUbN++HQUFBfWOO3bsGBwdHWFnZ4eJEydix44d6Nat4RXIExMT4eLiYtj8/f2bdD+WwNIWQCUiIrJERg2CtrOzw7PPPouUlBRkZ2eje/fumDx5MgICAlBZWdlcNdazZMkSBAcHIyQkBHK5HFOnTkV8fDyk0vq30qVLF2RlZSEtLQ2TJk1CXFwcsrOzG7zmrFmzUFZWZtjy8vLMcSvNIiuPb4ARERHdTZPfApNKpZBIJBAEATqdrknXcHd3h0wmQ1FRUb39RUVF8Pb2bvAcDw8PJCUlQaVS4eLFizh58iQcHR0RGBhY7zi5XI5OnTohIiICiYmJCAsLw5IlSxq8pp2dneGNsbrNGt2o0eF0UQUA9gARERHdiVEBSK1W48svv8Tf/vY3dO7cGceOHcOyZcuQm5sLR0dHoxuXy+WIiIhAamqqYZ9er0dqauptx+vUUSgU8PPzg1arxbZt2zBs2LA7Hq/X6+uN82mJfr9cBp1egIeTHbydFXc/gYiIqJVq9CDoyZMn46uvvoK/vz/Gjx+PL7/8Eu7u7vdcQEJCAuLi4hAZGYmoqCgsXrwYKpUK8fHxAICxY8fCz88PiYmJAIC0tDTk5+cjPDwc+fn5mD9/PvR6PWbMmGG45qxZszBo0CC0b98eFRUV2LJlC3755Rfs2rXrnuu1ZIYFUNu5QiLhAGgiIqLbaXQAWrlyJdq3b4/AwEDs3r0bu3fvbvC47du3G1XA6NGjUVJSgrlz56KwsBDh4eFITk42DIzOzc2tN76nuroac+bMQU5ODhwdHTF48GBs2rQJrq6uhmOKi4sxduxYFBQUwMXFBT179sSuXbvwt7/9zajarI1hAkSO/yEiIrqjRs8DNG7cuEb1Kqxfv/6eixKbtc4D9NCHP+PClSp8Pj4KAzp7iF0OERGRWTXLPEAbNmy417qoGV2vqsGFK1UAgJ7sASIiIrojLhXeQtQtgBrQ1h6u9nKRqyEiIrJsDEAtBBdAJSIiajwGoBai7g0wLoBKRER0dwxALYAgCMi62QMU7s/xP0RERHfDANQCFJRVo7RSDZlUgm4+DEBERER3wwDUAtQtgNrFywlKuUzcYoiIiKwAA1ALYFgAlY+/iIiIGoUBqAWo6wEK4wBoIiKiRmEAsnJ6vYBjfAOMiIjIKAxAVi6nVIUKtRYKWyk6ezmKXQ4REZFVYACycnUTIPbwdYGNjP85iYiIGoPfmFbOMP6HM0ATERE1GgOQlcu6VPcGmKu4hRAREVkRBiArVqPV48TlcgBAGFeAJyIiajQGICt2srAcNTo9XO1t0d7NXuxyiIiIrAYDkBWrGwDds50rJBKJuMUQERFZEQYgK1a3Anw4H38REREZhQHIiv25B4iIiIgajwHISlWqtThbUgkA6Mk1wIiIiIzCAGSljl0qgyAAvi4KeDopxC6HiIjIqjAAWSlOgEhERNR0DEBW6sjNAMTxP0RERMZjALJSR/LqZoDm+B8iIiJjMQBZodJKNfKv34BEAoT6MQAREREZiwHICtWN/wnycISTwlbcYoiIiKwQA5AVyqp7/MXxP0RERE3CAGSF/ngDjI+/iIiImoIByMoIgmCYAZo9QERERE3DAGRl8q7ewLUqDWxlEoT4OIldDhERkVViALIydfP/dPNxhp2NTNxiiIiIrJRFBKDly5cjICAACoUC0dHRSE9Pv+2xGo0GCxYsQFBQEBQKBcLCwpCcnFzvmMTERPTu3RtOTk7w9PTE8OHDcerUqea+DbPgAqhERET3TvQAtHXrViQkJGDevHnIzMxEWFgYYmNjUVxc3ODxc+bMwapVq7B06VJkZ2dj4sSJGDFiBA4fPmw4Zvfu3ZgyZQoOHDiAlJQUaDQaPProo1CpVOa6rWZz9FLdBIiu4hZCRERkxSSCIAhiFhAdHY3evXtj2bJlAAC9Xg9/f3+88sormDlz5i3H+/r6Yvbs2ZgyZYph38iRI6FUKrF58+YG2ygpKYGnpyd2796NAQMG3LWm8vJyuLi4oKysDM7Ozk28M9PT6vQInf8Dbmh0SHl9AIK9OAaIiIiojjHf36L2ANXU1CAjIwMxMTGGfVKpFDExMdi/f3+D56jVaigU9Vc/VyqV2LNnz23bKSur7TVxc3O77TXLy8vrbZbobEklbmh0cLSzQaCHo9jlEBERWS1RA1BpaSl0Oh28vLzq7ffy8kJhYWGD58TGxmLRokU4c+YM9Ho9UlJSsH37dhQUFDR4vF6vx7Rp09CvXz/06NGjwWMSExPh4uJi2Pz9/e/txppJ3fifHn7OkEkl4hZDRERkxUQfA2SsJUuWIDg4GCEhIZDL5Zg6dSri4+MhlTZ8K1OmTMHx48fx1Vdf3faas2bNQllZmWHLy8trrvLvyRGO/yEiIjIJUQOQu7s7ZDIZioqK6u0vKiqCt7d3g+d4eHggKSkJKpUKFy9exMmTJ+Ho6IjAwMBbjp06dSp27tyJn3/+Ge3atbttHXZ2dnB2dq63WaK6HqBwvgFGRER0T0QNQHK5HBEREUhNTTXs0+v1SE1NRZ8+fe54rkKhgJ+fH7RaLbZt24Zhw4YZficIAqZOnYodO3bgp59+QseOHZvtHsylWqPDqcIKAEBP9gARERHdExuxC0hISEBcXBwiIyMRFRWFxYsXQ6VSIT4+HgAwduxY+Pn5ITExEQCQlpaG/Px8hIeHIz8/H/Pnz4der8eMGTMM15wyZQq2bNmCb7/9Fk5OTobxRC4uLlAqlea/SRP4/XI5tHoB7o528HVR3P0EIiIiui3RA9Do0aNRUlKCuXPnorCwEOHh4UhOTjYMjM7Nza03vqe6uhpz5sxBTk4OHB0dMXjwYGzatAmurq6GY1asWAEAeOihh+q1tX79eowbN665b6lZGBZAbecCiYQDoImIiO6F6PMAWSJLnAdo2leHkZR1GQl/64xXBwaLXQ4REZHFsZp5gKjx6t4A69nOReRKiIiIrB8DkBUoq9LgfGntMh5hfAOMiIjonjEAWYGj+dcBAO3d7NHGQS5uMURERC0AA5AV4AKoREREpsUAZAWybk6AGMbxP0RERCbBAGQFDK/AsweIiIjIJBiALFxhWTWKytWQSSXo7msZr+QTERFZOwYgC3fkZu9PsKcj7OWiz1tJRETUIjAAWTjDAqh8/EVERGQyDEAW7qhhAkRXcQshIiJqQRiALJheLxgegYX58w0wIiIiU2EAsmDnr6hQUa2FnY0Unb2cxC6HiIioxWAAsmB1r7/38HOBrYz/qYiIiEyF36oW7EgeF0AlIiJqDgxAFqxu/A/fACMiIjItBiALVaPV4/fL5QD4BhgREZGpMQBZqNNFFajR6uGssEFAW3uxyyEiImpRGIAslGEBVH9XSCQScYshIiJqYRiALJRhAVQ+/iIiIjI5BiALVfcGGFeAJyIiMj0GIAukUmtxprgCABDGV+CJiIhMjgHIAh3PL4NeAHxcFPB0VohdDhERUYvDAGSB/lgAlb0/REREzYEByAJlGRZAdRW1DiIiopaKAcgCHal7BZ5vgBERETULBiALc6VSjUvXbgAAQvkIjIiIqFkwAFmYuvE/gR4OcFbYilwNERFRy8QAZGEMC6Dy8RcREVGzYQCyMHXjf/gGGBERUfNhALIggiAYHoHxDTAiIqLmwwBkQS5du4ErqhrYSCXo6uMsdjlEREQtlugBaPny5QgICIBCoUB0dDTS09Nve6xGo8GCBQsQFBQEhUKBsLAwJCcn1zvm119/xdChQ+Hr6wuJRIKkpKRmvgPTqev96erjDIWtTORqiIiIWi5RA9DWrVuRkJCAefPmITMzE2FhYYiNjUVxcXGDx8+ZMwerVq3C0qVLkZ2djYkTJ2LEiBE4fPiw4RiVSoWwsDAsX77cXLdhMkcMEyBy/A8REVFzkgiCIIjVeHR0NHr37o1ly5YBAPR6Pfz9/fHKK69g5syZtxzv6+uL2bNnY8qUKYZ9I0eOhFKpxObNm285XiKRYMeOHRg+fLhRdZWXl8PFxQVlZWVwdjbfo6jRq/Yj7fxVfPBUTzwd6W+2domIiFoCY76/ResBqqmpQUZGBmJiYv4oRipFTEwM9u/f3+A5arUaCkX9xUGVSiX27NlzT7Wo1WqUl5fX28xNpxdwLL/2EVg4B0ATERE1K9ECUGlpKXQ6Hby8vOrt9/LyQmFhYYPnxMbGYtGiRThz5gz0ej1SUlKwfft2FBQU3FMtiYmJcHFxMWz+/ubvfTlXUomqGh3s5TIEeTiavX0iIqLWRPRB0MZYsmQJgoODERISArlcjqlTpyI+Ph5S6b3dxqxZs1BWVmbY8vLyTFRx42XdnP8n1M8FMqnE7O0TERG1JqIFIHd3d8hkMhQVFdXbX1RUBG9v7wbP8fDwQFJSElQqFS5evIiTJ0/C0dERgYGB91SLnZ0dnJ2d623mZlgAlY+/iIiImp1oAUgulyMiIgKpqamGfXq9HqmpqejTp88dz1UoFPDz84NWq8W2bdswbNiw5i632RkmQOQSGERERM3ORszGExISEBcXh8jISERFRWHx4sVQqVSIj48HAIwdOxZ+fn5ITEwEAKSlpSE/Px/h4eHIz8/H/PnzodfrMWPGDMM1KysrcfbsWcPP58+fR1ZWFtzc3NC+fXvz3mAjVWt0OFFQO/CaS2AQERE1P1ED0OjRo1FSUoK5c+eisLAQ4eHhSE5ONgyMzs3NrTe+p7q6GnPmzEFOTg4cHR0xePBgbNq0Ca6uroZjDh06hIcfftjwc0JCAgAgLi4OGzZsMMt9GetEQTm0egFtHeRo10YpdjlEREQtnqjzAFkqc88DtGHvecz/LhsPd/HA+vioZm+PiIioJbKKeYDoD1wAlYiIyLwYgCxAlmEJDFdR6yAiImotGIBEVl6tQU6JCgDfACMiIjIXBiCRHbv5+MvfTQk3B7nI1RAREbUODEAiq1sBvid7f4iIiMyGAUhkdTNAhzMAERERmQ0DkMiO5NU+AuMEiERERObDACSiovJqFJZXQyoBevgxABEREZkLA5CI6h5/BXs6wcFO1Em5iYiIWhUGIBH9MQEie3+IiIjMiQFIRHwDjIiISBwMQCIRBOGPN8A4AzQREZFZMQCJ5MKVKpRXayG3kaKLt5PY5RAREbUqDEAiOXrz8Vd3X2fYyvifgYiIyJz4zSuSrJuPv7j+FxERkfkxAImEb4ARERGJhwFIBBqdHsfzbwYg9gARERGZHQOQCE4XVUCt1cNJYYOAtg5il0NERNTqMACJoG79r7B2rpBKJSJXQ0RE1PowAImgbv4fLoBKREQkDgYgEdTNAB3GCRCJiIhEwQBkZlU1WpwuqgDAAdBERERiYQAys98vl0MvAF7OdvB2UYhdDhERUavEAGRmf4z/cRW1DiIiotaMAcjMjtycAJELoBIREYmHAcjMjnAJDCIiItExAJnRNVUNcq9WAQBC+Qo8ERGRaBiAzKju9fdAdwe4KG3FLYaIiKgVYwAyoyuVNXCys+EEiERERCKzEbuA1mRkRDuM6OUHVY1W7FKIiIhaNfYAmZlUKoGTgo+/iIiIxGQRAWj58uUICAiAQqFAdHQ00tPTb3usRqPBggULEBQUBIVCgbCwMCQnJ9/TNYmIiKh1ET0Abd26FQkJCZg3bx4yMzMRFhaG2NhYFBcXN3j8nDlzsGrVKixduhTZ2dmYOHEiRowYgcOHDzf5mkRERNS6SARBEMQsIDo6Gr1798ayZcsAAHq9Hv7+/njllVcwc+bMW4739fXF7NmzMWXKFMO+kSNHQqlUYvPmzU265l+Vl5fDxcUFZWVlcHZ2NsVtEhERUTMz5vtb1B6gmpoaZGRkICYmxrBPKpUiJiYG+/fvb/ActVoNhaL+GlpKpRJ79uy5p2uWl5fX24iIiKjlEjUAlZaWQqfTwcvLq95+Ly8vFBYWNnhObGwsFi1ahDNnzkCv1yMlJQXbt29HQUFBk6+ZmJgIFxcXw+bv72+CuyMiIiJLJfoYIGMtWbIEwcHBCAkJgVwux9SpUxEfHw+ptOm3MmvWLJSVlRm2vLw8E1ZMRERElkbUAOTu7g6ZTIaioqJ6+4uKiuDt7d3gOR4eHkhKSoJKpcLFixdx8uRJODo6IjAwsMnXtLOzg7Ozc72NiIiIWi5RA5BcLkdERARSU1MN+/R6PVJTU9GnT587nqtQKODn5wetVott27Zh2LBh93xNIiIiah1Enwk6ISEBcXFxiIyMRFRUFBYvXgyVSoX4+HgAwNixY+Hn54fExEQAQFpaGvLz8xEeHo78/HzMnz8fer0eM2bMaPQ1iYiIqHUTPQCNHj0aJSUlmDt3LgoLCxEeHo7k5GTDIObc3Nx643uqq6sxZ84c5OTkwNHREYMHD8amTZvg6ura6GsSERFR6yb6PECWiPMAERERWR+rmQeIiIiISAwMQERERNTqiD4GyBLVPRXkjNBERETWo+57uzGjexiAGlBRUQEAnBGaiIjIClVUVMDFxeWOx3AQdAP0ej0uX74MJycnSCQSk167vLwc/v7+yMvLE2WANdsXt31LqIHtt+72LaEGtt+622/OGgRBQEVFBXx9fe+6QgR7gBoglUrRrl27Zm1D7Bmn2b74M36LXQPbb93tW0INbL91t99cNdyt56cOB0ETERFRq8MARERERK0OA5CZ2dnZYd68ebCzs2P7rbB9S6iB7bfu9i2hBrbfutu3lBo4CJqIiIhaHfYAERERUavDAEREREStDgMQERERtToMQERERNTqMACZ0fLlyxEQEACFQoHo6Gikp6ebre1ff/0VQ4cOha+vLyQSCZKSkszWNgAkJiaid+/ecHJygqenJ4YPH45Tp06Zrf0VK1agZ8+ehkm3+vTpg++//95s7f/V+++/D4lEgmnTppmlvfnz50MikdTbQkJCzNL2n+Xn5+O5555D27ZtoVQqERoaikOHDpml7YCAgFv+HUgkEkyZMsUs7et0Orz99tvo2LEjlEolgoKC8I9//KNRaxaZSkVFBaZNm4YOHTpAqVSib9++OHjwYLO1d7fPHUEQMHfuXPj4+ECpVCImJgZnzpwxW/vbt2/Ho48+irZt20IikSArK8tkbd+tfY1GgzfffBOhoaFwcHCAr68vxo4di8uXL5ulfaD2cyEkJAQODg5o06YNYmJikJaWZrb2/2zixImQSCRYvHixydq/GwYgM9m6dSsSEhIwb948ZGZmIiwsDLGxsSguLjZL+yqVCmFhYVi+fLlZ2vur3bt3Y8qUKThw4ABSUlKg0Wjw6KOPQqVSmaX9du3a4f3330dGRgYOHTqERx55BMOGDcPvv/9ulvb/7ODBg1i1ahV69uxp1na7d++OgoICw7Znzx6ztn/t2jX069cPtra2+P7775GdnY2PPvoIbdq0MUv7Bw8erHf/KSkpAIBRo0aZpf2FCxdixYoVWLZsGU6cOIGFCxfigw8+wNKlS83SPgC88MILSElJwaZNm3Ds2DE8+uijiImJQX5+frO0d7fPnQ8++ACffPIJVq5cibS0NDg4OCA2NhbV1dVmaV+lUuGBBx7AwoULTdKeMe1XVVUhMzMTb7/9NjIzM7F9+3acOnUKTzzxhFnaB4DOnTtj2bJlOHbsGPbs2YOAgAA8+uijKCkpMUv7dXbs2IEDBw7A19fXJO02mkBmERUVJUyZMsXws06nE3x9fYXExESz1wJA2LFjh9nb/bPi4mIBgLB7927RamjTpo3w2WefmbXNiooKITg4WEhJSREefPBB4bXXXjNLu/PmzRPCwsLM0tbtvPnmm8IDDzwgag1/9tprrwlBQUGCXq83S3tDhgwRxo8fX2/fk08+KYwZM8Ys7VdVVQkymUzYuXNnvf333XefMHv27GZv/6+fO3q9XvD29hY+/PBDw77r168LdnZ2wpdfftns7f/Z+fPnBQDC4cOHTd5uY9qvk56eLgAQLl68KEr7ZWVlAgDhxx9/NFv7ly5dEvz8/ITjx48LHTp0ED7++GOTt3077AEyg5qaGmRkZCAmJsawTyqVIiYmBvv37xexMvGUlZUBANzc3Mzetk6nw1dffQWVSoU+ffqYte0pU6ZgyJAh9f5fMJczZ87A19cXgYGBGDNmDHJzc83a/n/+8x9ERkZi1KhR8PT0RK9evbBmzRqz1lCnpqYGmzdvxvjx402+4PHt9O3bF6mpqTh9+jQA4MiRI9izZw8GDRpklva1Wi10Oh0UCkW9/Uql0uy9gQBw/vx5FBYW1vuz4OLigujo6Fb9uSiRSODq6mr2tmtqarB69Wq4uLggLCzMLG3q9Xo8//zzmD59Orp3726WNv+Mi6GaQWlpKXQ6Hby8vOrt9/LywsmTJ0WqSjx6vR7Tpk1Dv3790KNHD7O1e+zYMfTp0wfV1dVwdHTEjh070K1bN7O1/9VXXyEzM7NZx1zcTnR0NDZs2IAuXbqgoKAA77zzDvr374/jx4/DycnJLDXk5ORgxYoVSEhIwFtvvYWDBw/i1VdfhVwuR1xcnFlqqJOUlITr169j3LhxZmtz5syZKC8vR0hICGQyGXQ6Hd59912MGTPGLO07OTmhT58++Mc//oGuXbvCy8sLX375Jfbv349OnTqZpYY/KywsBIAGPxfrfteaVFdX480338Szzz5r1gVKd+7ciWeeeQZVVVXw8fFBSkoK3N3dzdL2woULYWNjg1dffdUs7f0VAxCZ3ZQpU3D8+HGz/62zS5cuyMrKQllZGb755hvExcVh9+7dZglBeXl5eO2115CSknLL38DN4c+9DD179kR0dDQ6dOiAr7/+GhMmTDBLDXq9HpGRkXjvvfcAAL169cLx48excuVKswegtWvXYtCgQWYdc/D111/jiy++wJYtW9C9e3dkZWVh2rRp8PX1Ndv9b9q0CePHj4efnx9kMhnuu+8+PPvss8jIyDBL+9QwjUaDp59+GoIgYMWKFWZt++GHH0ZWVhZKS0uxZs0aPP3000hLS4Onp2eztpuRkYElS5YgMzPTbL2wf8VHYGbg7u4OmUyGoqKievuLiorg7e0tUlXimDp1Knbu3Imff/4Z7dq1M2vbcrkcnTp1QkREBBITExEWFoYlS5aYpe2MjAwUFxfjvvvug42NDWxsbLB792588sknsLGxgU6nM0sddVxdXdG5c2ecPXvWbG36+PjcEja7du1q9kdxFy9exI8//ogXXnjBrO1Onz4dM2fOxDPPPIPQ0FA8//zzeP3115GYmGi2GoKCgrB7925UVlYiLy8P6enp0Gg0CAwMNFsNdeo++1r752Jd+Ll48SJSUlLM2vsDAA4ODujUqRPuv/9+rF27FjY2Nli7dm2zt/vbb7+huLgY7du3N3wmXrx4EW+88QYCAgKavX2AAcgs5HI5IiIikJqaatin1+uRmppq9jEoYhEEAVOnTsWOHTvw008/oWPHjmKXBL1eD7VabZa2Bg4ciGPHjiErK8uwRUZGYsyYMcjKyoJMJjNLHXUqKytx7tw5+Pj4mK3Nfv363TL1wenTp9GhQwez1QAA69evh6enJ4YMGWLWdquqqiCV1v/Ilclk0Ov1Zq0DqP3S8/HxwbVr17Br1y4MGzbM7DV07NgR3t7e9T4Xy8vLkZaW1mo+F+vCz5kzZ/Djjz+ibdu2Ypdkts/F559/HkePHq33mejr64vp06dj165dzd4+wEdgZpOQkIC4uDhERkYiKioKixcvhkqlQnx8vFnar6ysrPe3/fPnzyMrKwtubm5o3759s7c/ZcoUbNmyBd9++y2cnJwMz/hdXFygVCqbvf1Zs2Zh0KBBaN++PSoqKrBlyxb88ssvZvuD5uTkdMt4JwcHB7Rt29Ys46D+/ve/Y+jQoejQoQMuX76MefPmQSaT4dlnn232tuu8/vrr6Nu3L9577z08/fTTSE9Px+rVq7F69Wqz1aDX67F+/XrExcXBxsa8H39Dhw7Fu+++i/bt26N79+44fPgwFi1ahPHjx5uthl27dkEQBHTp0gVnz57F9OnTERIS0myfQ3f73Jk2bRr++c9/Ijg4GB07dsTbb78NX19fDB8+3CztX716Fbm5uYa5d+oCure3t0l6oe7Uvo+PD5566ilkZmZi586d0Ol0hs9FNzc3yOXyZm2/bdu2ePfdd/HEE0/Ax8cHpaWlWL58OfLz8002NcTd/v3/NfDZ2trC29sbXbp0MUn7d2W2981IWLp0qdC+fXtBLpcLUVFRwoEDB8zW9s8//ywAuGWLi4szS/sNtQ1AWL9+vVnaHz9+vNChQwdBLpcLHh4ewsCBA4UffvjBLG3fjjlfgx89erTg4+MjyOVywc/PTxg9erRw9uxZs7T9Z999953Qo0cPwc7OTggJCRFWr15t1vZ37dolABBOnTpl1nYFQRDKy8uF1157TWjfvr2gUCiEwMBAYfbs2YJarTZbDVu3bhUCAwMFuVwueHt7C1OmTBGuX7/ebO3d7XNHr9cLb7/9tuDl5SXY2dkJAwcONOl/m7u1v379+gZ/P2/evGZvv+7V+4a2n3/+udnbv3HjhjBixAjB19dXkMvlgo+Pj/DEE08I6enpJmn7bu03xNyvwUsEwYzTkBIRERFZAI4BIiIiolaHAYiIiIhaHQYgIiIianUYgIiIiKjVYQAiIiKiVocBiIiIiFodBiAiIiJqdRiAiIiIqNVhACIiugcPPfQQpk2bJnYZRGQkBiAisnjjxo2DRCKBRCKBra0tOnbsiBkzZqC6ulrs0ojISnExVCKyCo899hjWr18PjUaDjIwMxMXFQSKRYOHChWKXRkRWiD1ARGQV7Ozs4O3tDX9/fwwfPhwxMTFISUkBAKjVarz66qvw9PSEQqHAAw88gIMHDxrO3bBhA1xdXetdLykpCRKJxPDz/PnzER4ejk2bNiEgIAAuLi545plnUFFRYThGpVJh7NixcHR0hI+PDz766KPmvWkiajYMQERkdY4fP459+/ZBLpcDAGbMmIFt27Zh48aNyMzMRKdOnRAbG4urV68add1z584hKSkJO3fuxM6dO7F79268//77ht9Pnz4du3fvxrfffosffvgBv/zyCzIzM016b0RkHgxARGQVdu7cCUdHRygUCoSGhqK4uBjTp0+HSqXCihUr8OGHH2LQoEHo1q0b1qxZA6VSibVr1xrVhl6vx4YNG9CjRw/0798fzz//PFJTUwEAlZWVWLt2Lf71r39h4MCBCA0NxcaNG6HVapvjdomomXEMEBFZhYcffhgrVqyASqXCxx9/DBsbG4wcORJHjx6FRqNBv379DMfa2toiKioKJ06cMKqNgIAAODk5GX728fFBcXExgNreoZqaGkRHRxt+7+bmhi5dutzjnRGRGBiAiMgqODg4oFOnTgCAdevWISwsDGvXrkXv3r3veq5UKoUgCPX2aTSaW46ztbWt97NEIoFer7+HqonIUvERGBFZHalUirfeegtz5sxBUFAQ5HI59u7da/i9RqPBwYMH0a1bNwCAh4cHKioqoFKpDMdkZWUZ1WZQUBBsbW2RlpZm2Hft2jWcPn363m6GiETBAEREVmnUqFGQyWRYsWIFJk2ahOnTpyM5ORnZ2dl48cUXUVVVhQkTJgAAoqOjYW9vj7feegvnzp3Dli1bsGHDBqPac3R0xIQJEzB9+nT89NNPOH78OMaNGweplB+jRNaIj8CIyCrZ2Nhg6tSp+OCDD3D+/Hno9Xo8//zzqKioQGRkJHbt2oU2bdoAqB2rs3nzZkyfPh1r1qzBwIEDMX/+fLz00ktGtfnhhx+isrISQ4cOhZOTE9544w2UlZU1x+0RUTOTCH99ME5ERETUwrHvloiIiFodBiAiIiJqdRiAiIiIqNVhACIiIqJWhwGIiIiIWh0GICIiImp1GICIiIio1WEAIiIiolaHAYiIiIhaHQYgIiIianUYgIiIiKjV+X8hSe/QcEWMpwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(model_accuracy_arr)\n",
    "plt.xticks(range(num_rounds))\n",
    "plt.xlabel(\"Round\")\n",
    "plt.ylabel(\"Model Accuracy\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the plot above, we can see that the model accuracy increases steadily with each additional round of collecting more labels, getting improved consensus labels, and model training.\n",
    "To learn about the ActiveLab algorithm and benchmarks of its effectiveness, check out our paper:\n",
    "\n",
    "**[ActiveLab: Active Learning with Re-Labeling by Multiple Annotators](https://arxiv.org/abs/2301.11856)** \n",
    "\n",
    "Beyond the setting demonstrated here (with multiple annotators and unlabeled examples), ActiveLab is also effective for: standard active learning where you just collect at most one label per example (no re-labeling), as well as *active label cleaning* (with no unlabeled pool) where you only want to re-label examples to ensure 100% correct consensus labels (with the least amount of re-labeling)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Other useful multiannotator statistics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After conducting multiple rounds of multi-annotator active learning, you can also estimate various statistics from the labeled data such as the higher quality [CROWDLAB consensus label](https://arxiv.org/abs/2210.06812) for each example, consensus label quality scores that quantify how likely each consensus label is correct, and annotator quality scores that quantify how noisy each annotator's labels are overall. \n",
    "\n",
    "Below we show how to obtain this information using the annotators' labels and the model's predicted class probabilites. Check out cleanlab's [multiannotator module](https://docs.cleanlab.ai/stable/tutorials/multiannotator.html) for details!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the annotator labels and predicted probabilites to get the consensus labels and other statistics:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "multiannotator_results = get_label_quality_multiannotator(multiannotator_labels, pred_probs_labeled)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each row of the `label_quality` dataframe corresponds to an example, here you can get the CROWDLAB consensus label and its quality score:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>consensus_label</th>\n",
       "      <th>consensus_quality_score</th>\n",
       "      <th>annotator_agreement</th>\n",
       "      <th>num_annotations</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0.891332</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>0.914493</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0.349226</td>\n",
       "      <td>0.25</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0.919139</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0.837486</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   consensus_label  consensus_quality_score  annotator_agreement  \\\n",
       "0                0                 0.891332                 1.00   \n",
       "1                3                 0.914493                 1.00   \n",
       "2                1                 0.349226                 0.25   \n",
       "3                1                 0.919139                 1.00   \n",
       "4                1                 0.837486                 1.00   \n",
       "\n",
       "   num_annotations  \n",
       "0                1  \n",
       "1                1  \n",
       "2                8  \n",
       "3                1  \n",
       "4                1  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "multiannotator_results[\"label_quality\"].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each row of the `annotator_stats` dataframe corresponds to an annotator, here you can get their annotator quality and other information about the annotators:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>annotator_quality</th>\n",
       "      <th>agreement_with_consensus</th>\n",
       "      <th>worst_class</th>\n",
       "      <th>num_examples_labeled</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>0.339440</td>\n",
       "      <td>0.49</td>\n",
       "      <td>3</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>0.401138</td>\n",
       "      <td>0.58</td>\n",
       "      <td>3</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>0.404506</td>\n",
       "      <td>0.47</td>\n",
       "      <td>3</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>0.433437</td>\n",
       "      <td>0.54</td>\n",
       "      <td>3</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>0.434918</td>\n",
       "      <td>0.62</td>\n",
       "      <td>3</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    annotator_quality  agreement_with_consensus  worst_class  \\\n",
       "44           0.339440                      0.49            3   \n",
       "42           0.401138                      0.58            3   \n",
       "32           0.404506                      0.47            3   \n",
       "33           0.433437                      0.54            3   \n",
       "41           0.434918                      0.62            3   \n",
       "\n",
       "    num_examples_labeled  \n",
       "44                   100  \n",
       "42                   100  \n",
       "32                   100  \n",
       "33                   100  \n",
       "41                   100  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "multiannotator_results[\"annotator_stats\"].head()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  },
  "vscode": {
   "interpreter": {
    "hash": "00885e89789f58e60dbba52a405dc834aaf92411914fde0d391f9b48289a0610"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}