Python58-XGBoost学习与实战之初识XGBoost（四）early sop.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# XGBoost快速入门——与scikit-learn一起使用-early stop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 运行xgboost安装包中的实例程序\n",
    "import xgboost as xgb\n",
    "from xgboost import XGBClassifier\n",
    "\n",
    "# 加载LibSVM格式数据模块\n",
    "from sklearn.datasets import load_svmlight_file\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据读取\n",
    "scikit-learn支持多种格式的数据，包括LibSVM格式数据\n",
    "XGBoost可以加载libsvm格式的文本数据，libsvm的文件格式（稀疏特征）如下：\n",
    "\n",
    "1 101:1.2 102:0.03\n",
    "\n",
    "0 1:2.1 10001:300 10002:400\n",
    "\n",
    "...\n",
    "\n",
    "每一行表示一个样本，第一行的开头的“1”是样本的标签。“101”和“102”为特征索引，'1.2'和'0.03' 为特征的值。\n",
    "在两类分类中，用“1”表示正样本，用“0” 表示负样本。也支持[0,1]表示概率用来做标签，表示为正样本的概率。\n",
    "\n",
    "下面的示例数据需要我们通过一些蘑菇的若干属性判断这个品种是否有毒。\n",
    "\n",
    "UCI数据描述：http://archive.ics.uci.edu/ml/machine-learning-databases/mushroom/ ，\n",
    "\n",
    "每个样本描述了蘑菇的22个属性，比如形状、气味等等(加工成libsvm格式后变成了126维特征)，\n",
    "然后给出了这个蘑菇是否可食用。其中6513个样本做训练，1611个样本做测试。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(6513, 126)\n",
      "(1611, 126)\n"
     ]
    }
   ],
   "source": [
    "# read in data，数据在xgboost安装的路径下的demo目录,\n",
    "\n",
    "my_workpath = 'files/data/python58-data/'\n",
    "X_train,y_train = load_svmlight_file(my_workpath + 'agaricus.txt.train')\n",
    "X_test,y_test = load_svmlight_file(my_workpath + 'agaricus.txt.test')\n",
    "\n",
    "print(X_train.shape)\n",
    "print (X_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练集-校验集分离"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2150, 126)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# split data into train and test sets, 1/3的训练数据作为校验数据\n",
    "seed=7\n",
    "test_size=0.33\n",
    "X_train_part,X_validate,y_train_part,y_validate=train_test_split(X_train,y_train,test_size=test_size,\n",
    "                                                                random_state=seed)\n",
    "X_train_part.shape\n",
    "X_validate.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练参数设置\n",
    "- max_depth： 树的最大深度。缺省值为6，取值范围为：[1,∞]\n",
    "- eta：为了防止过拟合，更新过程中用到的收缩步长。在每次提升计算之后，算法会直接获得新特征的权重。 \n",
    "- eta通过缩减特征的权重使提升计算过程更加保守。缺省值为0.3，取值范围为：[0,1]\n",
    "- silent：取0时表示打印出运行时信息，取1时表示以缄默方式运行，不打印运行时信息。缺省值为0\n",
    "- objective： 定义学习任务及相应的学习目标，“binary:logistic” 表示二分类的逻辑回归问题，输出为概率。\n",
    "\n",
    "- 其他参数取默认值。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练模型\n",
    "有了参数列表和数据就可以训练模型了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0]\tvalidation_0-error:0.045611\tvalidation_1-error:0.048372\n",
      "Multiple eval metrics have been passed: 'validation_1-error' will be used for early stopping.\n",
      "\n",
      "Will train until validation_1-error hasn't improved in 10 rounds.\n",
      "[1]\tvalidation_0-error:0.041256\tvalidation_1-error:0.042326\n",
      "[2]\tvalidation_0-error:0.045611\tvalidation_1-error:0.048372\n",
      "[3]\tvalidation_0-error:0.041256\tvalidation_1-error:0.042326\n",
      "[4]\tvalidation_0-error:0.041256\tvalidation_1-error:0.042326\n",
      "[5]\tvalidation_0-error:0.041256\tvalidation_1-error:0.042326\n",
      "[6]\tvalidation_0-error:0.023378\tvalidation_1-error:0.023256\n",
      "[7]\tvalidation_0-error:0.041256\tvalidation_1-error:0.042326\n",
      "[8]\tvalidation_0-error:0.041256\tvalidation_1-error:0.042326\n",
      "[9]\tvalidation_0-error:0.023378\tvalidation_1-error:0.023256\n",
      "[10]\tvalidation_0-error:0.00573\tvalidation_1-error:0.006512\n",
      "[11]\tvalidation_0-error:0.01719\tvalidation_1-error:0.017674\n",
      "[12]\tvalidation_0-error:0.01719\tvalidation_1-error:0.017674\n",
      "[13]\tvalidation_0-error:0.01719\tvalidation_1-error:0.017674\n",
      "[14]\tvalidation_0-error:0.01719\tvalidation_1-error:0.017674\n",
      "[15]\tvalidation_0-error:0.01719\tvalidation_1-error:0.017674\n",
      "[16]\tvalidation_0-error:0.01719\tvalidation_1-error:0.017674\n",
      "[17]\tvalidation_0-error:0.01719\tvalidation_1-error:0.017674\n",
      "[18]\tvalidation_0-error:0.0259\tvalidation_1-error:0.024651\n",
      "[19]\tvalidation_0-error:0.022003\tvalidation_1-error:0.020465\n",
      "[20]\tvalidation_0-error:0.022003\tvalidation_1-error:0.020465\n",
      "Stopping. Best iteration:\n",
      "[10]\tvalidation_0-error:0.00573\tvalidation_1-error:0.006512\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=1,\n",
       "       colsample_bytree=1, gamma=0, learning_rate=0.1, max_delta_step=0,\n",
       "       max_depth=2, min_child_weight=1, missing=None, n_estimators=100,\n",
       "       n_jobs=1, nthread=None, object='binary.logistic',\n",
       "       objective='binary:logistic', random_state=0, reg_alpha=0,\n",
       "       reg_lambda=1, scale_pos_weight=1, seed=None, silent=True,\n",
       "       subsample=1)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 设置boosting迭代计算次数\n",
    "num_round=100\n",
    "\n",
    "# bst=XGBClassifier(**params)\n",
    "# bst=XGBClassifier()\n",
    "bst=XGBClassifier(max_depth=2,learning_rate=0.1,n_estimators=num_round,\n",
    "                  silent=True,object='binary.logistic')\n",
    "eval_set=[(X_train_part,y_train_part),(X_validate,y_validate)]\n",
    "bst.fit(X_train_part,y_train_part,early_stopping_rounds=10,eval_metric='error',\n",
    "        eval_set=eval_set,verbose=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "模型在每次校验集上的性能存在模型中，可用来进一步进行分析 model.evals result() 返回一个字典：评估数据集和分数\n",
    "\n",
    "显示学习曲线"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEWCAYAAABMoxE0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xt8XFW5+P/Pk3uazKRtkjbTG2lpaSZQKCVWBBEQRFChCEWLiKgc661f8YIKnp8I6NGDHuWI5aAoeBAVihS0SBFQpBwult4pbdKSlpamTdo0vSRpm/vz+2PvKcN0kkwys2cmzfN+vebVmb3XXvuZaTJP9lprryWqijHGGDNYGakOwBhjzNBmicQYY0xcLJEYY4yJiyUSY4wxcbFEYowxJi6WSIwxxsTFEokxQ5SIfFpEXkx1HMZYIjFpTUQKRWSbiHwibJtPRN4Skblh26pE5K8isl9EDojIRhH5DxEZ5e7/tIh0i0ir+9gqIl9M8nuJjCH0GJfMONxY5ojIWhFpFpG9IvIPESl3990qIr9Pdkxm6LJEYtKaqrYC84Gfi0ipu/nHwEpVfRRARM4CngdeAipUdSRwMdAFnBZW3SuqWqiqhcBc4Mcicnpy3smxMYQ9dg20EhHJGmwAIjIV+B3wDaAImAz8D9Az2DrN8GaJxKQ9VX0GeBK4S0TOAz4GfDmsyI+B36rqj1R1t3vMW6r6PVV9vpc6VwPVQDC0TUQuE5EN7hXN8yISvi/objvglrksbN+H3CugFhHZKSI3DuZ9ishNIrLFrWejiHw0bN+nReQlEblTRPYBt0Yce7eI/DRi2xMi8tUop5oJvKmq/1BHi6ouVtW3RORi4DvAx92rpXVuXeNEZImI7BORWhH5XNh5bhWRR0VkkRv7ahE5Lcp5zXHKEokZKr4GnAc8CtyoqvUAIlIAvAdYPJDKRORdwEnASvf1ScBDwFeBUmAp8ISI5IhINvAE8AwwBvh/wB9EZLpb3X3A51XVB5wCPDfI97gFOAfnKuE24PciEgjb/25gqxvDf0Qc+wBwtYhkuO+nBLjAfU+RVgMVblI6X0QKQztU9W/AD4FF7tVSKCE8BNQB43Cu5n4oIheE1TkH+BMwGvgj8Gf3czPDgCUSMySo6n5gAzACeCxs1yicn+OG0AYR+bF75XBIRP6/sLJnuttbgVeBB4E33H0fB55U1WdVtRP4LyAfOAs4EygE/lNVO1T1OeCvwNXusZ1ApYj4VXW/e7XTm1AMoceWsPf4J1Xdpao9qrrIjW122LG7VPUXqtqlqkciPp9XgYM4yQNgHvB86AotouxWnKQ8HngE2Csi/xueUMKJyETgvcC3VbVNVdcCvwGuDSu2SlUfdT+7nwF57udmhgFLJGZIEJFPAuXA34E7wnbtx2nbP/qXu6p+y+0neRwI70v4l6qOdPtIyoCTcf76Bucv7e1hdfQAO3C+bMcBO9xtIdvdfQBXAh8CtovIMhF5Tx9vJRRD6HFi2Hv8lNsBfkBEDuBc3ZSEHbujj3rBuSr5pPv8kziJMipV/ZeqfkxVS3Gugt4H/HsvxccB+1S1JWxb+Pt/R2zu5xS6ejHDgCUSk/ZEZAxwJ/A54PPAx0TkfQCqeghYDlwxkDrdv9QXA5e6m3YBJ4SdU4CJwE5338RQs5FrkrsPVV2hqnNwmpz+jPNX/oCIyAnAr4EFQLGbCF8HJDzsfqr5PTDH7Z8IurH0S1VX4FzlndLLeXYBo0XEF7bt6Pt3TQx7LxnABPc4MwxYIjFDwULgz6r6T7dv5FvAr0Uk193/LeCzbmf1GAARmYAzGikqESkGPorTXAbOl/+HReQCt23/G0A78DJOojoEfEtEst0O/0uBh90+lGtEpMht1mkGugfxHgtwvsAb3fg+w9tf7DFR1TpgBc6VyOLI5q8QEXmviHwu7LOqAC4D/uUW2Q2UhxKnqu7A+Rx+JCJ5InIqcD3wh7BqzxCRK9zRZF/F+ez+hRkWLJGYtCYil+O0z38ztE1Vf4PTdHKL+/pF4P04zTOb3Wahv+EMCf5FWHXvcUciteKM2GrE6ThHVTfhNAf9AtiLkygudftEOnC+aC9x9/0P8ClVrXHrvRbYJiLNwBd4u3kpmqMxhD3epaobgZ8Cr+B8kc/AGc48UA+4x/barAUccN/Pevez+BtOM+CP3f1/cv9tEpFQf8/VOE2Lu9yy31PVZ8Pq/AtOP9N+nM/jCjexmmFAbGErY44fbpPf74HyiD4dL895KzBVVftKoOY4Zlckxhwn3Ca5G4DfJCuJGAOWSIw5Lrg3Tx7AGb323ykOxwwz1rRljDEmLnZFYowxJi6DnvhtKCkpKdHy8vJUh2GMMUPKqlWr9ro3rfZpWCSS8vJyVq5cmeowjDFmSBGR7f2XsqYtY4wxcbJEYowxJi6WSIwxxsRlWPSRGGPMYHV2dlJXV0dbW1uqQ/FMXl4eEyZMIDt7cEvIWCIxxpg+1NXV4fP5KC8vx5kU+viiqjQ1NVFXV8fkyb3Oc9ona9oyxpg+tLW1UVxcfFwmEQARobi4OK4rLkskxhjTj+M1iYTE+/4skfRCVXlkxQ7+UX3MSqVJ8/KWvazdcSBl5zfGmFh4mkhE5GIR2SQitSJyU5T9uSKyyN2/XETKI/ZPctdruDFs2zYRWe8uSerZXYZdPcoDr2zj64+sY+eBqOsDeWp93UE+ff8KPvmb5WxvOpT08xtj0kNTUxMzZ85k5syZlJWVMX78+KOvOzo6Yq7n/vvvp6GhwZMYPUskIpIJ3I2zGFAlcLWIVEYUux7Yr6pTcZZSvSNi/53AU1GqP19VZ6pqVYLDPio7M4OFn5hFV3cPX3loDZ3dyZuVu6WtkwUPrWZ0QQ6ZGcKCP66hvWswi+4ZY4a64uJi1q5dy9q1a/nCF77A1772taOvc3JyYq5nSCYSYDZQq6pb3RXmHgbmRJSZg7OiG8CjwAXuWtmhlfG28vZSqEk3uaSAH14xg1Xb93Pns5uTck5V5ebH1lO3/wi/+MTp/GTuqazfeZD/fKqm/4ONMcPKAw88wOzZs5k5cyZf+tKX6Onpoauri2uvvZYZM2ZwyimncNddd7Fo0SLWrl3Lxz/+8QFfycTCy+G/44EdYa/rgHf3VkZVu0TkIFAsIkeAbwMfAG6MOEaBZ0REgV+p6r3RTi4i84H5AJMmTRr0m5gzczyvbGnif57fwrunFHPuSf3OXxaXh1fs4K+v1fPND07nXeWjAfjM2eX89qVtnDmlmA+eXObp+Y0xvbvtiQ1s3NWc0Dorx/n53qUnD/i4119/nccff5yXX36ZrKws5s+fz8MPP8yJJ57I3r17Wb9+PQAHDhxg5MiR/OIXv2DhwoXMnDkzofGDt1ck0YYBRC5+0luZ24A7VbU1yv6zVXUWTpPZl92lRY+tRPVeVa1S1arS0vi+/L936clMH+vj64vWsqfZu5uSahqauXXJBs6ZVsIXzz3x6PabLqlgxvgivvmnddTtP+zZ+Y0xQ8ff//53VqxYQVVVFTNnzmTZsmVs2bKFqVOnsmnTJm644QaefvppioqKPI/FyyuSOmBi2OsJwK5eytSJSBZQBOzDuXKZKyI/BkYCPSLSpqoLVXUXgKruEZHHcZrQXvDwfZCfk8nCT5zOZQtf4oaH1/L7f3s3mRmJHQ54uKOLL/9hNf78bH72sZlkhNWfm+Wc/8N3vchXHlrDos+/h+xMG3BnTLIN5srBK6rKZz/7Wb7//e8fs++1117jqaee4q677mLx4sXce2/UhpuE8fLbaAUwTUQmi0gOMA9YElFmCXCd+3wu8Jw6zlHVclUtx1k29IequlBECkTEByAiBcBFwOsevoejpo31cfuck3llaxMLn6tNeP23/GUDW/ce4ucfn0mpL/eY/ScUF/CfV85g9VsH+OkzyemvMcakrwsvvJBHHnmEvXv3As7orrfeeovGxkZUlauuuorbbruN1atXA+Dz+WhpafEkFs+uSNw+jwXA00AmcL+qbhCR24GVqroEuA94UERqca5E5vVT7Vjgcbc/Pgv4o6r+zav3EGnuGRN4ZUsTP//HZmZPHs17TixOSL2LV9Xx6Ko6vnLBNM6aWtJruY+cOo6XtzTxy2VbOHPKaM6bPiYh5zfGDD0zZszge9/7HhdeeCE9PT1kZ2fzy1/+kszMTK6//npUFRHhjjucwbCf+cxn+Ld/+zfy8/N59dVXBzTiqz/DYs32qqoqTdTCVofau7j0Fy/S2t7FUzecQ3HhsVcPA1G7p5XLFr7IjPFF/PFzZ/bbZNbW2c3ld7/EnpZ2nrrhHMb68+I6vzGmb9XV1QSDwVSH4blo71NEVsVym4U1tA9QQW4WCz8xiwNHOvn6I+vo6Rl8Im7r7GbBH1eTl53Jz+edHlO/S152Jgs/MYsjHd185aE1dMdxfmOMSQRLJINQOc7PLR+pZNnmRu79v62Druf7f91ITUMLP/3YaZQVxX5lMXVMIT+4/BSWv7mPu/7xxqDPb4wxiWCJZJCuefckPjwjwE+e3sSq7fsGfPxfX9vFH5a/xefPncL5g+jruPKMCVw5awJ3PfcGL9fuHfDxxpjYHe9dAPG+P0skgyQi/OjKGYwbmcdXHlrLgcOx3ym6vekQNy9ez+mTRnLjRdMHHcPtc05mSkkBNyxay97W9kHXY4zpXV5eHk1NTcdtMgmtR5KXN/j+Vutsj9O6HQeY+8uXOW/6GO699ox+p2Nu7+pm7j2vsL3pEEtvOIcJo0bEdf7q+mYuv/slZk8ezQOfmf2O+0+MMfEbziskxtrZbiskxum0iSO56ZIg3//rRv735W185uy+Vxi746lNrN95kF9de0bcSQQgGHCmV/jO4+v55Qtb+NJ5U+Ou0xjztuzs7EGvHDhcWNNWAnz27HIuDI7lh0urea2u9/VDnt24m/tfepNPn1We0Dmzrp49kY+cGuCnz2xm5baB99cYY0w8LJEkgIjwX1edSmlhLgv+uIbmts5jyuw8cIQb/7SOU8b7uflDFQk//4+umMGEUfl85aE17D+U2Jk9jTGmL5ZIEmTkiBzuuvp0dh44ws2PrX9Hx1ynu6ZJd4+y8OpZ5GZlJvz8vrxsFl49i8bWdr756LrjtmPQGJN+LJEkUFX5aL5x0Uk8+Vo9D7369gz6P3t2M6u27+eHV8ygvKTAs/PPmFDEdz4U5O/Ve7j/pW2enccYY8JZIkmwL7zvRM6ZVsJtT2ygur6ZZZsbuef5LVw9exKXnTbO8/N/+qxyPlA5lv98qpp1tt67MSYJbPivB/a2tnPJz/8PX24WB490UlKYy18WnE1eduKbtKI5cLiDD9/1IhkZ8ORXzsGfl93/QcYYEyHW4b+WSDzy8pa9XPOb5eRlZfLE/zubqWN8ST3/qu37+divXiEvK4OcrNRdeH709AnccmllSs5957Ob+d0r2+Kq46LKMu6Ye2pC4jFmqLFEEiYViQTgqfX1FI3I5qwTe58a3kt/37ibF95oTMm5AZZv3ceBIx0s/86FKTn/hT9bRnePcs60wX3+a946wNbGVtbf+kG70dMMS3ZDYhq4ZEYgpee/sHIsF1aOTdn5f/3CVv5jaTX7DnUwuiBxax/Eoq2zm62NrXz5/Kl8Y5DT0Dz86lvc9Nh66vYfYVJx/DePGnO8ss5245mKgNOcV9PQnPRz1+5ppUehosw/6DoqAs6x1SmI35ihxNNEIiIXi8gmEakVkZui7M8VkUXu/uUiUh6xf5KItIrIjbHWadJHMPRFXO/N8p592Vjf7MYw+L6p6WN9iDjzmRljeudZIhGRTOBu4BKgErhaRCJ7Xa8H9qvqVOBO4I6I/XcCTw2wTpMmSgpzKSnMpSYFX8Q19S3kZWdwQvHg79vJz8lkcnEBNSlIhMYMJV5ekcwGalV1q6p2AA8DcyLKzAEecJ8/Clwg7vS5InI5sBXYMMA6TRoJBnzUNCT/i7imoZnpY30xrTrZl4qALyVNc8YMJV4mkvHAjrDXde62qGVUtQs4CBSLSAHwbeC2QdQJgIjMF5GVIrKysTF1I5eGu4oyH5t2t9DV3ZO0c6oq1fXNcfWPhFSU+dm+7zCH2rsSEJkxxycvE0m0PwUjxxr3VuY24E5VbR1Enc5G1XtVtUpVq0pLS/sN1ngjGPDT0dXDtqZDSTvnnpZ29h/ujKt/JCQY8KMKm3Zb85YxvfFy+G8dMDHs9QRgVy9l6kQkCygC9gHvBuaKyI+BkUCPiLQBq2Ko06SR0FVBdX1L0m7KDHWOh0ZdxaOizB15Vt/CrEmj4q7PmOORl1ckK4BpIjJZRHKAecCSiDJLgOvc53OB59RxjqqWq2o58N/AD1V1YYx1mjRy4pgCsjIkqf0MoT6ZYAKatiaMyqcwN8v6SYzpg2dXJKraJSILgKeBTOB+Vd0gIrcDK1V1CXAf8KCI1OJcicwbTJ1evQcTv9ysTKaOKUzqEODq+mbGFeVRNCL+OcZEhIoynw0BNqYPnt7ZrqpLgaUR224Je94GXNVPHbf2V6dJbxVlPl59M3krN9bUtySkWSskGPDz5zU7UVXcQYXGmDB2Z7vxXEXAz66DbRw8fOzKkYnW3tXNlsbWo30biVAR8NHS3sXOA0cSVqcxxxNLJMZzoS/1ZEw1Urunla4eTegVSfiAAWPMsSyRGM9Vul/qybjDPXQXemUChv6GvD1yy/pJjInGEonxXKkvl9EFOUm5w72moZmcrAzK45gaJVJBbhYnFI9IyR36xgwFlkiM546OfEpKImnhpLGFZGUm9kfbid+uSIyJxhKJSYpgwM+mhma6e7xdSK26vjkh949Eqijzs23vIY50dCe8bmOGOkskJikqyny0dfaw3cOpUhpb2tnb2pHQjvaQYMBPj8JmmyrFmGNYIjFJEVqbxMt+htDd58EEDv0NCaZwkS5j0p0lEpMUU8cUkpkhno58Co3Y8uKKZOKoEYzIybQhwMZEYYnEJEVediZTSgrY6OEXcXV9M2P9uZ6sD5+RIUy3qVKMicoSiUmaioDf06ah6oaWhKxB0ptgwE9NQwuq3g4YMGaosURikqaizEfd/iM0tyV+qpTO7h5q97RQkcAbESMFy3wcPNJJQ3ObZ+cwZiiyRGKSJnSH+yYPOty3NLbS2a1Hz+GFUN+LNW8Z806WSEzShK4WvOhwP9rR7mHT1vTQnGHW4W7MO1giMUlT5s+jKD/bkzvcqxuayc4UppQmbmqUSP68bCaMyrepUoyJYInEJE1oqhSvrkimjvGRneCpUSJVlPlt8kZjInj6WyciF4vIJhGpFZGbouzPFZFF7v7lIlLubp8tImvdxzoR+WjYMdtEZL27b6WX8ZvEC4186knwVCnV9c1Hbxr0UjDgY+veQ7R12lQpxoR4lkhEJBO4G7gEqASuFpHKiGLXA/tVdSpwJ3CHu/11oEpVZwIXA78SkfDVHM9X1ZmqWuVV/MYbwYCPwx3d7Nh/OGF1NrW2s6el3ZM5tiIFA366e5TaPa2en8uYocLLK5LZQK2qblXVDuBhYE5EmTnAA+7zR4ELRERU9bCqdrnb8wAbuH+c8GKRqNAoMC+H/oYcXaTLmreMOcrLRDIe2BH2us7dFrWMmzgOAsUAIvJuEdkArAe+EJZYFHhGRFaJyPzeTi4i80VkpYisbGxsTMgbMvE7aayPDEnsF/FGt66gh0N/Q04oLiAvO8NGbhkTxstEIlG2RV5Z9FpGVZer6snAu4CbRSTP3X+2qs7CaTL7soi8L9rJVfVeVa1S1arS0tLBvQOTcPk5mZSXFCT0DveahhZKCnMpKcxNWJ29ycwQpo/12eSNxoTxMpHUARPDXk8AdvVWxu0DKQL2hRdQ1WrgEHCK+3qX++8e4HGcJjQzhATL/AkdQlvTkJyO9pBgwE91fbNNlWKMy8tEsgKYJiKTRSQHmAcsiSizBLjOfT4XeE5V1T0mC0BETgCmA9tEpEBEfO72AuAinI55M4RUlPnY3nSYQ+1d/RfuR1d3D5t3tx7tu0iGijIf+w930tjSnrRzGpPOPEskbp/GAuBpoBp4RFU3iMjtInKZW+w+oFhEaoGvA6Ehwu8F1onIWpyrji+p6l5gLPCiiKwDXgWeVNW/efUejDcSuTbJm3sP0dHVk5T+kZDQVCkbrcPdGACy+i8yeKq6FFgase2WsOdtwFVRjnsQeDDK9q3AaYmP1CRTRdgiUWecMCquukJ3yXs5NUqk0DDjmoYWzps+JmnnNSZd2Z3tJunGj8zHl5d1dH6seNTUN5OVIZw4xrupUSIVjchmXFGe3eFujMsSiUk6EXE73OP/Iq5paGHqmEJyszITEFnsKgJ+GwJsjMsSiUmJioCPmvr4F4mqrm9Oakd7SEWZjy2NrbR32VQpxlgiMSlRUeanpb2Luv1HBl3HgcMd1B9s82SN9v4EA366epQtew4l/dzGpBtLJCYl3u5wH3zzUM3RjvbkX5EEwwYMGDPcWSIxKTF9rA+Jc6qU0LFerorYm/LiAnKyMmzOLWOwRGJSpCA3ixNGj4jrL/qa+hZGF+RQ6vN+apRIWZkZnDS20Ba5MgZLJCaFnEWi4mnacjraRaJN2ea9YJmN3DIGLJGYFAoG/LzZdIgjHQMf+dTdo2za3ZLUO9ojVQT87G1tt6lSzLBnicSkTEXAhyps2j3wv+q3NR2irbMnJR3tIcEy63A3BiyRmBQ6OtXIIDqsQ01iqb4iCY/FmOHKEolJmQmj8inIyRxUh3VNQzOZGcLUMYUeRBab0QU5jPXnUm1XJGaYs0RiUiYjQ6gI+Ac1i251fTNTSgrIy07u1CiRKqzD3RhLJCa1Ksp81Axikajq+paU3NEeqSLgo3ZPC53dPakOxZiUsURiUqoi4Ke5rYv6g20xH9Pc1snOA0dS2tEeUhnw09mtbG20qVLM8OVpIhGRi0Vkk4jUishNUfbnisgid/9yESl3t88WkbXuY52IfDTWOs3QUjmIqUY2uX0qqbijPVLF0bVJrJ/EDF+eJRIRyQTuBi4BKoGrRaQyotj1wH5VnQrcCdzhbn8dqFLVmcDFwK9EJCvGOs0QctJYJ5EMpJ8hNC1JRRLXae/NlNICsjPFVks0w5qXVySzgVpV3aqqHcDDwJyIMnOAB9znjwIXiIio6mF3qV6APCDUgB5LnWYI8eVlM3F0/oDmrKqub6EoP5syf56HkcUmOzODqWN8NgTYDGteJpLxwI6w13Xutqhl3MRxECgGEJF3i8gGYD3wBXd/LHWaIaaizD+gIcCpnholUjDgs6YtM6x5mUii/ZZHDs3ptYyqLlfVk4F3ATeLSF6MdToVi8wXkZUisrKxsXEAYZtkCwb8bG1spa2z/6lSenqUTQ2pnRolUrDMz+7mdvYd6kh1KMakhJeJpA6YGPZ6ArCrtzIikgUUAfvCC6hqNXAIOCXGOkPH3auqVapaVVpaGsfbMF4LlvnoUXhjd2u/Zd/ad5jDHd1H1wNJB0fXVrF+EjNMeZlIVgDTRGSyiOQA84AlEWWWANe5z+cCz6mqusdkAYjICcB0YFuMdZohJnQ/SCx3iIeakEKjpdJB8Gj81k9ihqcsrypW1S4RWQA8DWQC96vqBhG5HVipqkuA+4AHRaQW50pknnv4e4GbRKQT6AG+pKp7AaLV6dV7MMlxwugR5GdnxtRhXV3fQoa8PdorHZQU5lJSmGtXJGbY8iyRAKjqUmBpxLZbwp63AVdFOe5B4MFY6zRDW0aGML3MF9PIrer6ZspLCsjPSe3UKJGCAZ/NuWWGLbuz3aSF0Min/qZKqWloOTprcDqpKPOxeXcrXTZVihmGLJGYtFBR5mf/4U729LFIVGt7F2/tO5wWU6NECgb8dHT1sK3Jpkoxw48lEpMWjnZY99G8FZoaJZ2G/oaEOv832o2JZhiyRGLSwvSy/qdKSaepUSKdOKaArAyxDnczLFkiMWmhKD+b8SPz+7xDvKahGV9uFuNH5icxstjkZmVyYmnhoBbpMmaos0Ri0oazNknvX8Q19S1UBNJnapRIwYDPrkjMsNRvIhGRTBH5STKCMcNbMOBnS2Mr7V3HTpXS06POiK007B8JqQj42XWwjQOHbaoUM7z0m0hUtRs4Q9L1z0Bz3KgI+OjqUWr3HDtVys4DR2ht70qrO9ojhUaTWfOWGW5ibdpaA/xFRK4VkStCDy8DM8PP0UWiojRvpXNHe0hooS1r3jLDTax3to8GmoD3h21T4LGER2SGrcklBeRmZUTtcK9paEEEpqfR1CiRSn25jC7IsSsSM+zElEhU9TNeB2JM5tGpUqJfkZwwegQFuZ7O6hMXEaEixqlejDmexNS0JSITRORxEdkjIrtFZLGITPA6ODP8VJRFXySqpqElrftHQirK/Gza3UJ3T99TvRhzPIm1j+S3ONO1j8NZkfAJd5sxCVVR5mdvaweNYVOlHO7oYlvTobTuHwkJBny0dfaw3aZKMcNIrImkVFV/q6pd7uN/AVstyiRctKlSNjW0oJqeU6NEejt+6ycxw0esiWSviHzSvackU0Q+idP5bkxCvT2E9u1EEuq8TsdZfyNNHVNIhmBruJu4Ld/axFcfXkNre1eqQ+lXrInks8DHgAagHmc1w896FZQZvkYV5FDmz3vHEOCa+mYKcjKZMCr9pkaJlJedyZTSQrsiMXH72bOb+fPaXXz14bX0pHmfW0x3tgNXquplqlqqqmNU9XJV3R7DsReLyCYRqRWRm6LszxWRRe7+5SJS7m7/gIisEpH17r/vDzvmebfOte5jzIDesUl7ziJRb38RVze0UBHwk5ExNO6JDQb8dkVi4rJj32GWv7mPU8b7+Xv1bn7yzKZUh9SnWO9snzPQit0EdDdwCVAJXC0ilRHFrgf2q+pU4E7gDnf7XuBSVZ2Bs6Z75GqJ16jqTPexZ6CxmfRWEfBTu6eFjq4eVJXq+ua0XIOkNxVlPur2H6G5rTPVoZgh6rHVOwG455oz+MS7J3HP81t4fE1diqPqXaxNWy+JyEIROUdEZoUe/RwzG6hV1a2q2gE8zLEJaQ7wgPv8UeACERFVXaOqu9ztG4A8EcmNMVYzxFWU+ejsVrbubWXXwTZa2rqoGAId7SFBd3TZJrsx0QyCqvLYmjrOnDKaiaNHcNtoHvIlAAAaUUlEQVRlJ3PmlNF8e/F6Vr+1P9XhRRVrIjkLOBm4Hfip+/ivfo4ZD+wIe13nbotaRlW7gINAcUSZK4E1qhq+dN5v3Wat7/Y2B5iIzBeRlSKysrGxsZ9QTToJBt6eKiU03UhwCF2RBG2qFBOHldv3s73pMFfOcm7Vy87M4J5rzqDMn8f8361i14EjKY7wWLH0kWQA96jq+RGP9/d3aJRtkT1GfZYRkZNxmrs+H7b/GrfJ6xz3cW20k6vqvapapapVpaU2UnkomVJSQE5mBtUNzUdHbE0fQomkzJ9HUX62rZZoBmXxqjryszO5ZEbg6LZRBTncd10V7Z3dfO53KznckV4juWLpI+kBFgyi7jpgYtjrCcCu3sqISBZQBOxzX08AHgc+papbwuLZ6f7bAvwRpwnNHEeyMjOYNtYZ+bSxvpmJo/Px5WWnOqyYhaZKsQ53M1Btnd08+Vo9l5xSRmHEdEDTxvq46xOnU13fzDceWZdWI7libdp6VkRuFJGJIjI69OjnmBXANBGZLCI5wDycu+PDLcHpTAdnSPFzqqoiMhJ4ErhZVV8KFRaRLBEpcZ9nAx8BXo/xPZghpKLMT019MzX1zUNiapRIwYCfTQ0tafXLbtLf0xsaaGnv4sozos9Adf70MXznQ0Geer2B//7HG0mOrncDuY/ky8ALwCr3sbKvA9w+jwXA00A18IiqbhCR20XkMrfYfUCxiNQCXwdCQ4QXAFOB70YM880FnhaR14C1wE7g1zG+BzOEBAM+9rS0s3XvoSFxR3ukYMDH4Y5uduw/nOpQzBCyePVOxhXl8Z4pkV3Fb7v+vZO56owJ3PWPN3hiXWQjT2rEOvvv5MFUrqpLgaUR224Je94GXBXluB8AP+il2jMGE4sZWkLJQ3VodbSHhK6iquubOaG4IMXRmKFgd3MbL77RyJfOm9rnPVMiwg8+egrbmg5x45/WcULxCE6dMDKJkR6rzysSEflW2POrIvb90KugjAm/b2QoDf0NOWmsDxGbc8vE7vE1O+lRuGJW5ODWY+VmZXLPJ8+gpDCXz/1uJbub25IQYe/6a9qaF/b85oh9Fyc4FmOOKi7MpdSXS352JpNGj0h1OAOWn5PJ5OIC63A3MVFVFq+qY9akkUwpLYzpmJLCXH5zXRUtbV3M/91K2jq7PY6yd/0lEunlebTXxiTUu8pHMeuEkWQOkalRIgUDfrsiMTFZv/Mgb+xp7bWTvTfBgJ+fzzud13Ye5JuPvoZqagZ39JdItJfn0V4bk1A/vWom915bleowBq2izMdb+w7TYlOlmH4sXlVHTlYGHzl13ICP/UDlWL75wek8sW4Xd/+z1oPo+tdfZ/tpItKMc/WR7z7HfZ3naWRm2MvPyUx1CHEJDRjY1NBCVXl/o+XNcNXR1cOSdbv4QOVYivIHd7/UF889kTd2t/Jfz2xm6phCLj4l0P9BCdTnFYmqZqqqX1V9qprlPg+9Hjp3iBmTAsFxxy7SZUyk52r2sP9wJ3NnDX71chHhR1fMYObEkXxt0To27DqYwAj7F+t9JMaYARpXlIc/L+sdU+IbE2nx6jpKfbmcM60krnrysjO591NnMHJENp97YOU7lqv2miUSYzwiIm6Hu12RmOiaWtv5Z80eLp85jqzM+L+Ox/jy+PWnqth3uIPPP5i8kVyWSIzxkE2VYvqyZN0uunp0wKO1+nLK+CJ+9rGZrH7rAN95fH1SRnJZIjHGQ5UBP4c7utm+z6ZKMcd6bPVOTh7nT/h8ch+aEeBrF57EP2v2sOug9zcrWiIxxkMV7iJX1rxlIm3e3cL6nQePrjuSaF+5YCpPf/V9jB+Z70n94SyRGOOhk8b6yBBLJOZYi1fVkZUhzJk58HtHYiEijPEn5y4NSyTGeCgvO5MppYWWSMw7dHX38PianZw3fQzFhUN/FXFLJMZ4zKZKMZFerN3LnpZ25p7R/wSNQ4ElEmM8Fgz42HngCAcP21QpxrF49U5Gjsjm/IoxqQ4lISyRGOOx0FQpNhOwAWhu6+SZDQ1cdto4crOG9jRAIZ4mEhG5WEQ2iUitiNwUZX+uiCxy9y8XkXJ3+wdEZJWIrHf/fX/YMWe422tF5C4RGZpTw5phozJgU6WYtz35Wj3tXT2ejdZKBc8SiYhkAncDlwCVwNUiUhlR7Hpgv6pOBe4E7nC37wUuVdUZOGu6Pxh2zD3AfGCa+7B1UUxaG+PLZXRBjvWTGMAZrTV1TCGnTihKdSgJ4+UVyWygVlW3qmoH8DAwJ6LMHOAB9/mjwAUiIqq6RlVDixFvAPLcq5cA4FfVV9S5XfN3wOUevgdj4uZMleKj2pq2hr1tew+xcvt+rpw1geOpMcXLRDIe2BH2us7dFrWMqnYBB4HIVe+vBNaoartbvq6fOgEQkfkislJEVjY2Ng76TRiTCBVlzlQpXd09qQ7FpNBjq+vIEPjo6cfHaK0QLxNJtHQbOelLn2VE5GSc5q7PD6BOZ6PqvapapapVpaWlMYRrjHeCAT/tXT1sazqU6lBMivT0KItX7+TsqSWUFR1fyzl5mUjqgIlhrycAu3orIyJZQBGwz309AXgc+JSqbgkrH95DFa1OY9JO0J0qZaP1kwxby9/cx84DR5ibwAka04WXiWQFME1EJotIDjAPWBJRZglOZzrAXOA5VVURGQk8Cdysqi+FCqtqPdAiIme6o7U+BfzFw/dgTEJMHVNIVobYyK1hbPHqOgpzs7iosizVoSScZ4nE7fNYADwNVAOPqOoGEbldRC5zi90HFItILfB1IDREeAEwFfiuiKx1H6E7d74I/AaoBbYAT3n1HoxJlNysTKaOKaTGEsmwdLiji6fW1/PhGYEhv4R0NP2t2R4XVV0KLI3YdkvY8zbgqijH/QD4QS91rgROSWykxngvGPDzypamVIdhUuBvrzdwqKM7oeuOpBO7s92YJAkGfDQ0t7H/UEeqQzFJtnh1HZNGj+Bd5aNSHYonLJEYkyRBu8N9WNp14Agvb2niilnjj6t7R8JZIjEmSUKr4G20RDKsPL5mJ6ocV1OiRLJEYkySlPpyKSnMtalShhFVZfGqOmZPHs3E0SNSHY5nLJEYk0TBgM+atoaRNTsOsHXvIeYex1cjYInEmKSqDPip3dNKp02VMiwsXlVHXnYGl8w4/u4dCWeJxJgkCgb8dHT3sKWxNdWhGI+1dXbzxLpdXHxyGb687FSH4ylP7yMxxrzT0UWu6luOdr575eUte/nTyrr+CxpP7DvUQXNb13F770g4SyTGJNGU0gJyMjOorm/mco9ngL3z2c28vrOZUl+up+cxvTtnWglnnViS6jA8Z4nEmCTKzsxg2thCz4cAHzzSyeq3DvDFc0/kxg9O9/RcxlgfiTFJFgz4PR8C/FLtXrp7lPOm2xIKxnuWSIxJsooyH3tb22lsaffsHMs2NeLLy2LmxJGencOYEEskxiRZpcdTpagqyzY3cs60ErIy7VfceM9+yoxJMq/n3Nq8u5WG5jbOPcmatUxyWCIxJslGFeRQ5s/zLJEs27wHgPdZIjFJYonEmBRwpkrxpsP9+U2NTB/rI1CU70n9xkTyNJGIyMUisklEakXkpij7c0Vkkbt/uYiUu9uLReSfItIqIgsjjnnerTNy5URjhoxgwM+Wxlbau7oTWu+h9i5WbNtno7VMUnmWSEQkE7gbuASoBK4WkcqIYtcD+1V1KnAncIe7vQ34LnBjL9Vfo6oz3ceexEdvjLeCAT9dPUrtnsROlfLKliY6u9X6R0xSeXlFMhuoVdWtqtoBPAzMiSgzB3jAff4ocIGIiKoeUtUXcRKKMcedtzvcE9u8tWxzIyNyMjnjOF2Jz6QnLxPJeGBH2Os6d1vUMqraBRwEimOo+7dus9Z3pZclx0RkvoisFJGVjY2NA4/eGA+VF48gNysjoR3uqsrzm/dw1onF5GZlJqxeY/rjZSKJ9gWvgygT6RpVnQGc4z6ujVZIVe9V1SpVrSottct8k16yMjOYXpbYtUm2NR1mx74j1qxlks7LRFIHTAx7PQHY1VsZEckCioB9fVWqqjvdf1uAP+I0oRkz5ATL/FTXN6Pa399OsXl+k9NdeO5JNv7EJJeXiWQFME1EJotIDjAPWBJRZglwnft8LvCc9vFbJSJZIlLiPs8GPgK8nvDIjUmCYMDH/sOd7G5OzFQpyzY3MqWkgEnFx++SriY9eTb7r6p2icgC4GkgE7hfVTeIyO3ASlVdAtwHPCgitThXIvNCx4vINsAP5IjI5cBFwHbgaTeJZAJ/B37t1Xswxkvhd7iXFeXFVVdbZzf/2trEvHdNSkRoxgyIp9PIq+pSYGnEtlvCnrcBV/VybHkv1Z6RqPiMSaWKUCJpaOb8iviao159cx9tnT2ca/ePmBSwO9uNSZGi/GzGj8xPyBDgZZsbycnK4MzJsQx6NCaxLJEYk0LO2iTxj9x6ftMezpxSTH6ODfs1yWeJxJgUqgz42NrYSlvn4KdK2bHvMFsaD9mwX5MylkiMSaGKgJ8ehc27B9+89cIbzg23lkhMqlgiMSaFErE2ybJNjYwfmc+JpQWJCsuYAbFEYkwKnTB6BCNyMgfd4d7R1cPLW5o4d3opvcwWZIznLJEYk0IZGcL0Mh8bB3lFsmr7flrbu6xZy6SUJRJjUiw0cmswU6Us29xIVoZw9tQSDyIzJjaWSIxJsWDAT0tbF7sODnzVhGWbG6kqH0Vhrqf3FhvTJ0skxqRYZcAHQPWugTVv7W5uo7q+2SZpNClnicSYFJteNriRWy9stmG/Jj1YIjEmxQpzsziheATVDQNLJM9vbmSML5ege0VjTKpYIjEmDVSU+QY0BLiru4cX39jLuSfZsF+TepZIjEkDwYCfbU2HONzRFVP5dXUHOXik02b7NWnBEokxaSAY8KMKNQ2xXZUs29xIhsB7bdivSQOWSIxJA5UDnCpl2eZGZk4cycgROV6GZUxMPE0kInKxiGwSkVoRuSnK/lwRWeTuXy4i5e72YhH5p4i0isjCiGPOEJH17jF3iTUQm+PAhFH5+HKzqImhn2TfoQ5eqztgw35N2vAskYhIJnA3cAlQCVwtIpURxa4H9qvqVOBO4A53exvwXeDGKFXfA8wHprmPixMfvTHJJSJUBHwxXZH83xuNqGL9IyZteHlFMhuoVdWtqtoBPAzMiSgzB3jAff4ocIGIiKoeUtUXcRLKUSISAPyq+oo680n8Drjcw/dgTNIEA35qGlro6el7qpRlmxoZXZDDqeOLkhSZMX3zMpGMB3aEva5zt0Uto6pdwEGgr7VCx7v19FUnACIyX0RWisjKxsbGAYZuTPIFA35a27uo23+k1zI9PcoLbzRyzrQSMjKsVdekBy8TSbSf8sg/tWIpM6jyqnqvqlapalVpqTUBmPRXUebcWNjXTMAb65vZ29phd7ObtOJlIqkDJoa9ngDs6q2MiGQBRcC+fuqc0E+dxgxJ08t8iPQ9cmuZOy3KOdMskZj04WUiWQFME5HJIpIDzAOWRJRZAlznPp8LPKd9zKWtqvVAi4ic6Y7W+hTwl8SHbkzyjcjJYnJxQZ+J5PlNezhlvJ9SX24SIzOmb54lErfPYwHwNFANPKKqG0TkdhG5zC12H1AsIrXA14GjQ4RFZBvwM+DTIlIXNuLri8BvgFpgC/CUV+/BmGQLBvy9zrl18Egnq986wHk27NekGU8XMVDVpcDSiG23hD1vA67q5djyXravBE5JXJTGpI9gwMeT6+tpaevEl5f9jn0v1+6lu0dt2K9JO3ZnuzFpJOje4b4pylQpyzY34svL4vSJI5MdljF9skRiTBoJ9jJViqqybHMj751aQlam/dqa9GI/kcakkUBRHkX52WyMmCpl8+5W6g+22bBfk5YskRiTRkSEYJSpUpZt3gPYtCgmPVkiMSbNVJT52dTQQnfYVCnLNjcyfayPQFF+CiMzJjpLJMakmcqAnyOd3WxvOgTAofYuVry5365GTNqyRGJMmnm7w93pJ/nX1iY6unusf8SkLUskxqSZaWMLycyQo/0kz29qJD87k6ryUSmOzJjoLJEYk2bysjOZUuJMlaKqPL95D2edWExuVmaqQzMmKkskxqSh0Nok25oOs2PfEc6z/hGTxiyRGJOGggE/Ow8cYclaZ3JrW1bXpDNLJMakoWDAWZvkgVe2MbmkgEnFI1IbkDF9sERiTBoKjdzad8gWsTLpzxKJMWlojC+X0QU5AJZITNqzRGJMGgpNlZKTlcGZU4pTHY4xffI0kYjIxSKySURqReSmKPtzRWSRu3+5iJSH7bvZ3b5JRD4Ytn2biKwXkbUistLL+I1JpS+eO5VbLz2Z/Bwb9mvSm2cLW4lIJnA38AGctdZXiMgSVd0YVux6YL+qThWRecAdwMfd1RDnAScD44C/i8hJqtrtHne+qu71KnZj0sF7p5WkOgRjYuLlFclsoFZVt6pqB/AwMCeizBzgAff5o8AF7lrsc4CHVbVdVd/EWVZ3toexGmOMGSQvE8l4YEfY6zp3W9Qy7hrvB4Hifo5V4BkRWSUi8z2I2xhjzAB4uWa7RNmmMZbp69izVXWXiIwBnhWRGlV94ZiTO0lmPsCkSZNij9oYY8yAeHlFUgdMDHs9AdjVWxkRyQKKgH19HauqoX/3AI/TS5OXqt6rqlWqWlVaasMnjTHGK14mkhXANBGZLCI5OJ3nSyLKLAGuc5/PBZ5TVXW3z3NHdU0GpgGvikiBiPgARKQAuAh43cP3YIwxph+eNW2papeILACeBjKB+1V1g4jcDqxU1SXAfcCDIlKLcyUyzz12g4g8AmwEuoAvq2q3iIwFHnf648kC/qiqf/PqPRhjjOmfOBcAx7eqqipdudJuOTHGmIEQkVWqWtVfObuz3RhjTFyGxRWJiDQC2wd5eAmQzjc/WnzxsfjiY/HFJ93jO0FV+x2tNCwSSTxEZGUsl3apYvHFx+KLj8UXn3SPL1bWtGWMMSYulkiMMcbExRJJ/+5NdQD9sPjiY/HFx+KLT7rHFxPrIzHGGBMXuyIxxhgTF0skxhhj4mKJxBXPao5JiG2iiPxTRKpFZIOI3BClzHkictBdOXKtiNySrPjc8/e5cqU47nI/v9dEZFYSY5se9rmsFZFmEflqRJmkfn4icr+I7BGR18O2jRaRZ0XkDfffUb0ce51b5g0RuS5aGY/i+4mI1Lj/f4+LyMhejvV8FdNe4rtVRHaG/R9+qJdj+/xd9zC+RWGxbRORtb0cO/RWgVXVYf/AmQtsCzAFyAHWAZURZb4E/NJ9Pg9YlMT4AsAs97kP2BwlvvOAv6bwM9wGlPSx/0PAUzhLBJwJLE/h/3UDzo1WKfv8gPcBs4DXw7b9GLjJfX4TcEeU40YDW91/R7nPRyUpvouALPf5HdHii+VnwcP4bgVujOH/v8/fda/ii9j/U+CWVH1+iX7YFYkjntUcPaeq9aq62n3eAlRz7CJh6W4O8Dt1/AsYKSKBFMRxAbBFVQc700FCqLOGzr6IzeE/Yw8Al0c59IPAs6q6T1X3A88CFycjPlV9Rp0F6AD+hbO8Q0r08vnFIpbf9bj1FZ/7vfEx4KFEnzdVLJE44lnNMancJrXTgeVRdr9HRNaJyFMicnJSA+t/5cpYPuNkmEfvv8Cp/PwAxqpqPTh/PABjopRJl8/xszhXmNGkchXTBW7T2/29NA2mw+d3DrBbVd/oZf+QWwXWEokjntUck0ZECoHFwFdVtTli92qc5prTgF8Af05mbDgrV84CLgG+LCLvi9ifDp9fDnAZ8Kcou1P9+cUqHT7Hf8dZ3uEPvRTp72fBK/cAJwIzgXqc5qNIKf/8gKvp+2okVZ/foFkiccSzmmNSiEg2ThL5g6o+FrlfVZtVtdV9vhTIFpGSZMWn/a9cGctn7LVLgNWqujtyR6o/P9fuUHOf+++eKGVS+jm6nfsfAa5Rt0E/Ugw/C55Q1d2q2q2qPcCvezlvqj+/LOAKYFFvZVL1+cXDEokjntUcPee2qd4HVKvqz3opUxbqsxGR2Tj/t01Jii+WlSuXAJ9yR2+dCRwMNeMkUa9/Caby8wsT/jN2HfCXKGWeBi4SkVFu081F7jbPicjFwLeBy1T1cC9lUraKaUSf20d7OW8sv+teuhCoUdW6aDtT+fnFJdW9/enywBlVtBlnRMe/u9tux/mlAcjDaRKpBV4FpiQxtvfiXH6/Bqx1Hx8CvgB8wS2zANiAMwrlX8BZSYxvinvedW4Moc8vPD4B7nY/3/VAVZL/f0fgJIaisG0p+/xwElo90InzV/L1OH1u/wDecP8d7ZatAn4Tduxn3Z/DWuAzSYyvFqd/IfQzGBrFOA5Y2tfPQpLie9D92XoNJzkEIuNzXx/zu56M+Nzt/xv6mQsrm/TPL9EPmyLFGGNMXKxpyxhjTFwskRhjjImLJRJjjDFxsURijDEmLpZIjDHGxMUSiTFxEpFud6bW10Xkid5mxU3wObel4IZJY6KyRGJM/I6o6kxVPQVntoMvpzogY5LJEokxifUK7iSA7l38P3GvVNaLyMfd7eeJyF9DB4jIQhH5tPt8m4jcJiKr3WMq3O3FIvKMiKwRkV8Rfc4oY1LCEokxCSIimTjT1Iem3LgCZwLB03CmxvhJjFPn71Vn0r57gBvdbd8DXlTV0936JyUydmPiYYnEmPjlu6vdNeEsOPWsu/29wEPqTCS4G1gGvCuG+kKTcq4Cyt3n7wN+D6CqTwL7ExO6MfGzRGJM/I6o6kzgBJxV90J9JL01P3Xxzt+9vIj97e6/3UBW2Habz8ikJUskxiSIqh4EvgLc6E77/wLwcRHJFJFSnKuKV4HtQKWI5IpIEU5zWH9eAK4BEJFLcJbZNSYtZPVfxBgTK1VdIyLrcKYn/z3wHpyZXBX4lqo2AIjIIziz1L4BrImh6tuAh0RkNU4T2VsehG/MoNjsv8YYY+JiTVvGGGPiYonEGGNMXCyRGGOMiYslEmOMMXGxRGKMMSYulkiMMcbExRKJMcaYuPz/rv6HN4Hiu+wAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# retrieve performance metrics\n",
    "result=bst.evals_result()\n",
    "\n",
    "epochs=len(result['validation_0']['error'])\n",
    "x_axis=range(0,epochs)\n",
    "\n",
    "# plot classification error\n",
    "fig,ax=plt.subplots()\n",
    "ax.plot(x_axis,result['validation_0']['error'],label='Test')\n",
    "ax.legend()\n",
    "plt.ylabel('Error')\n",
    "plt.xlabel(\"Round\")\n",
    "plt.title('XGBoos Early Stop')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 测试\n",
    "模型训练好后，可以用训练好的模型对测试数据进行预测\n",
    "\n",
    "XGBoost预测的输出是概率，输出值是样本为第一类的概率。我们需要将概率值转换为0或1。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test Accuracy: 97.27%\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "G:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\preprocessing\\label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.\n",
      "  if diff:\n"
     ]
    }
   ],
   "source": [
    "# make prediction\n",
    "preds = bst.predict(X_test)\n",
    "predictions = [round(value) for value in preds]\n",
    "\n",
    "test_accuracy = accuracy_score(y_test, predictions)\n",
    "print(\"Test Accuracy: %.2f%%\" % (test_accuracy * 100.0))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}