shfl/rfout.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Backdoor Attacks-Resilient Aggregation based on Robust Filtering of Outliers in Federated Learning for image classification\n",
    "\n",
    "In this notebook we provide the implemetation of the averaging mechanism RFOut, presented in [this paper](). We also provide a simple example of how to use it in a federated environment. We use [EMNIST Digits](https://www.nist.gov/itl/products-and-services/emnist-dataset) dataset, a popular dataset for the experiments. We federate the data following an I.I.D. distribution over 20 nodes (clients).\n",
    "\n",
    "### Data reading"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import shfl\n",
    "\n",
    "database = shfl.data_base.Emnist()\n",
    "train_data, train_labels, test_data, test_labels = database.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [],
   "source": [
    "iid_distribution = shfl.data_distribution.IidDataDistribution(database)\n",
    "federated_data, test_data, test_labels = iid_distribution.get_federated_data(num_nodes=20, percent=100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We reshape training and test data in order to fit the required shape. For reshaping the federated data, we use the class FederatedTransformation. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "class Reshape(shfl.private.FederatedTransformation):\n",
    "    \n",
    "    def apply(self, labeled_data):\n",
    "        labeled_data.data = np.reshape(labeled_data.data, (labeled_data.data.shape[0], labeled_data.data.shape[1], labeled_data.data.shape[2],1))\n",
    "        \n",
    "shfl.private.federated_operation.apply_federated_transformation(federated_data, Reshape())\n",
    "\n",
    "test_data = np.reshape(test_data, (test_data.shape[0], test_data.shape[1], test_data.shape[2],1))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Deep Learning model\n",
    "\n",
    "We use a simple deep learning model based on two layers of CNN implemented in Keras."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "def model_builder():\n",
    "    model = tf.keras.models.Sequential()\n",
    "    model.add(tf.keras.layers.Conv2D(32, kernel_size=(3, 3), padding='same', activation='relu', strides=1, input_shape=(28, 28, 1)))\n",
    "    model.add(tf.keras.layers.MaxPooling2D(pool_size=2, strides=2, padding='valid'))\n",
    "    model.add(tf.keras.layers.Dropout(0.4))\n",
    "    model.add(tf.keras.layers.Conv2D(32, kernel_size=(3, 3), padding='same', activation='relu', strides=1))\n",
    "    model.add(tf.keras.layers.MaxPooling2D(pool_size=2, strides=2, padding='valid'))\n",
    "    model.add(tf.keras.layers.Dropout(0.3))\n",
    "    model.add(tf.keras.layers.Flatten())\n",
    "    model.add(tf.keras.layers.Dense(128, activation='relu'))\n",
    "    model.add(tf.keras.layers.Dropout(0.1))\n",
    "    model.add(tf.keras.layers.Dense(64, activation='relu'))\n",
    "    model.add(tf.keras.layers.Dense(10, activation='softmax'))\n",
    "\n",
    "    model.compile(optimizer=\"rmsprop\", loss=\"categorical_crossentropy\", metrics=[\"accuracy\"])\n",
    "    \n",
    "    return shfl.model.DeepLearningModel(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Federated Aggregator: RFOut\n",
    "\n",
    "In this point, we provide the implementation of the RFOut aggregation mechanism. For that purpose, we overwrite the implementation of the FedAvg aggregator in [rfout.py](./rfout.py).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [],
   "source": [
    "from rfout import RFOut1d\n",
    "\n",
    "aggregator = RFOut1d()\n",
    "federated_government = shfl.federated_government.FederatedGovernment(model_builder, federated_data, aggregator)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run the federated algorithm\n",
    "\n",
    "Finally, we run 10 rounds of learning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "pycharm": {
     "is_executing": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy round 0\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b9d0>: [0.2388855218887329, 0.9332000017166138]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b880>: [0.2581176459789276, 0.9220499992370605]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b190>: [0.19621045887470245, 0.9393249750137329]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287380a0>: [0.1872367411851883, 0.942550003528595]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287382b0>: [0.21427756547927856, 0.9347249865531921]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9250>: [0.22970852255821228, 0.930899977684021]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93a0>: [0.19189073145389557, 0.9419500231742859]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9550>: [0.1961621344089508, 0.941474974155426]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9610>: [0.243849515914917, 0.9299250245094299]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb96d0>: [0.2081487774848938, 0.9358999729156494]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9520>: [0.21334826946258545, 0.9330999851226807]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9880>: [0.20309950411319733, 0.9448750019073486]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9970>: [0.22021955251693726, 0.9330750107765198]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9a60>: [0.2360302209854126, 0.9356250166893005]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9b50>: [0.23469223082065582, 0.9321249723434448]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93d0>: [0.21361806988716125, 0.937624990940094]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb94f0>: [0.22649258375167847, 0.935325026512146]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9d90>: [0.23842595517635345, 0.925000011920929]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9f10>: [0.1911039650440216, 0.9461749792098999]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9ee0>: [0.19144952297210693, 0.9452499747276306]\n",
      "Global model test performance : [0.4851139187812805, 0.9234750270843506]\n",
      "\n",
      "\n",
      "\n",
      "Accuracy round 1\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b9d0>: [0.13547809422016144, 0.9614499807357788]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b880>: [0.14166778326034546, 0.958774983882904]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b190>: [0.14548717439174652, 0.9544000029563904]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287380a0>: [0.1236949935555458, 0.9639250040054321]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287382b0>: [0.14975126087665558, 0.9545750021934509]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9250>: [0.12347474694252014, 0.9631249904632568]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93a0>: [0.12353714555501938, 0.9624249935150146]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9550>: [0.14121533930301666, 0.9569000005722046]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9610>: [0.1377611756324768, 0.9607750177383423]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb96d0>: [0.1582116335630417, 0.9499750137329102]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9520>: [0.1283305138349533, 0.9623000025749207]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9880>: [0.12885522842407227, 0.9620000123977661]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9970>: [0.13007882237434387, 0.9637249708175659]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9a60>: [0.13267189264297485, 0.9607499837875366]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9b50>: [0.12400049716234207, 0.9633499979972839]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93d0>: [0.1374894231557846, 0.9593499898910522]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb94f0>: [0.15398560464382172, 0.9559749960899353]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9d90>: [0.12565010786056519, 0.9619500041007996]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9f10>: [0.11731094121932983, 0.9678750038146973]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9ee0>: [0.12904739379882812, 0.9624500274658203]\n",
      "Global model test performance : [0.12271680682897568, 0.9687749743461609]\n",
      "\n",
      "\n",
      "\n",
      "Accuracy round 2\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b9d0>: [0.10976628214120865, 0.9708499908447266]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b880>: [0.10231529176235199, 0.9699500203132629]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b190>: [0.10691407322883606, 0.9682250022888184]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287380a0>: [0.10135801136493683, 0.9688000082969666]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287382b0>: [0.13061022758483887, 0.9585000276565552]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9250>: [0.10152841359376907, 0.970300018787384]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93a0>: [0.10805460065603256, 0.9691749811172485]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9550>: [0.11366046220064163, 0.9683499932289124]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9610>: [0.11002598702907562, 0.9685249924659729]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb96d0>: [0.10064857453107834, 0.9695749878883362]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9520>: [0.11793921142816544, 0.9664750099182129]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9880>: [0.13460400700569153, 0.9590749740600586]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9970>: [0.11025239527225494, 0.9678750038146973]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9a60>: [0.11161519587039948, 0.9671249985694885]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9b50>: [0.12087365984916687, 0.9638749957084656]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93d0>: [0.10741893202066422, 0.968874990940094]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb94f0>: [0.11775379627943039, 0.965624988079071]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9d90>: [0.11527737975120544, 0.9677749872207642]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9f10>: [0.09912654012441635, 0.9733250141143799]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9ee0>: [0.0968470573425293, 0.9704499840736389]\n",
      "Global model test performance : [0.08968925476074219, 0.9774500131607056]\n",
      "\n",
      "\n",
      "\n",
      "Accuracy round 3\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b9d0>: [0.11364084482192993, 0.966949999332428]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b880>: [0.0954194888472557, 0.9711750149726868]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b190>: [0.08156702667474747, 0.9751999974250793]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287380a0>: [0.07841644436120987, 0.9758999943733215]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287382b0>: [0.10380958020687103, 0.9695500135421753]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9250>: [0.09021474421024323, 0.974399983882904]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93a0>: [0.08978726714849472, 0.9757500290870667]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9550>: [0.09040649235248566, 0.9750999808311462]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9610>: [0.09839345514774323, 0.9718250036239624]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb96d0>: [0.09859319776296616, 0.9693250060081482]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9520>: [0.08538071811199188, 0.975974977016449]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9880>: [0.10990029573440552, 0.9686999917030334]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9970>: [0.08661627024412155, 0.9757000207901001]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9a60>: [0.08874532580375671, 0.9746999740600586]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9b50>: [0.08148611336946487, 0.9755749702453613]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93d0>: [0.09293752163648605, 0.9748749732971191]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb94f0>: [0.10566122829914093, 0.9671499729156494]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9d90>: [0.10426794737577438, 0.9722999930381775]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9f10>: [0.07729529589414597, 0.9790250062942505]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9ee0>: [0.09955435246229172, 0.9687749743461609]\n",
      "Global model test performance : [0.07383355498313904, 0.9812750220298767]\n",
      "\n",
      "\n",
      "\n",
      "Accuracy round 4\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b9d0>: [0.08662769198417664, 0.9760749936103821]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b880>: [0.08742935210466385, 0.974174976348877]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b190>: [0.0822490006685257, 0.9761750102043152]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287380a0>: [0.08422570675611496, 0.9740250110626221]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287382b0>: [0.08457591384649277, 0.9745500087738037]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9250>: [0.10081102699041367, 0.97222501039505]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93a0>: [0.07818493247032166, 0.9783250093460083]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9550>: [0.08748605847358704, 0.9751750230789185]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9610>: [0.08800072968006134, 0.9745000004768372]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb96d0>: [0.07566069066524506, 0.9775999784469604]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9520>: [0.08875451982021332, 0.9752749800682068]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9880>: [0.08760051429271698, 0.9749249815940857]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9970>: [0.10274142771959305, 0.9738500118255615]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9a60>: [0.07957231998443604, 0.9764249920845032]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9b50>: [0.08021637052297592, 0.9757500290870667]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93d0>: [0.0720076709985733, 0.9780250191688538]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb94f0>: [0.10299696028232574, 0.9699249863624573]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9d90>: [0.08522294461727142, 0.9745000004768372]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9f10>: [0.07490862160921097, 0.978600025177002]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9ee0>: [0.07998666167259216, 0.9773250222206116]\n",
      "Global model test performance : [0.0644276887178421, 0.9828500151634216]\n",
      "\n",
      "\n",
      "\n",
      "Accuracy round 5\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b9d0>: [0.0942881628870964, 0.9725750088691711]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b880>: [0.07579708844423294, 0.9771749973297119]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b190>: [0.07772012054920197, 0.9779000282287598]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287380a0>: [0.0790097787976265, 0.9746249914169312]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287382b0>: [0.08162403851747513, 0.9763000011444092]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9250>: [0.07384593784809113, 0.9796749949455261]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93a0>: [0.0729253739118576, 0.9790250062942505]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9550>: [0.07747974991798401, 0.9785000085830688]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9610>: [0.0860111340880394, 0.9753249883651733]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb96d0>: [0.07991674542427063, 0.9767000079154968]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9520>: [0.08632246404886246, 0.9773250222206116]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9880>: [0.07948481291532516, 0.9787750244140625]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9970>: [0.07777303457260132, 0.9799249768257141]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9a60>: [0.07665253430604935, 0.977649986743927]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9b50>: [0.09540059417486191, 0.9725000262260437]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93d0>: [0.0703655406832695, 0.9801250100135803]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb94f0>: [0.07696633040904999, 0.979200005531311]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9d90>: [0.0755414217710495, 0.978725016117096]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9f10>: [0.06608203053474426, 0.9812999963760376]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9ee0>: [0.06668052822351456, 0.9807999730110168]\n",
      "Global model test performance : [0.05901665613055229, 0.9849500060081482]\n",
      "\n",
      "\n",
      "\n",
      "Accuracy round 6\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b9d0>: [0.0905396118760109, 0.9763000011444092]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b880>: [0.06392548233270645, 0.9820500016212463]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b190>: [0.07779323309659958, 0.9772999882698059]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287380a0>: [0.08462753146886826, 0.9742500185966492]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287382b0>: [0.07240232825279236, 0.978725016117096]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9250>: [0.0735410675406456, 0.980774998664856]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93a0>: [0.0633547231554985, 0.9822750091552734]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9550>: [0.08062846958637238, 0.975724995136261]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9610>: [0.06901977211236954, 0.980650007724762]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb96d0>: [0.062738336622715, 0.9818500280380249]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9520>: [0.07763431966304779, 0.9787750244140625]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9880>: [0.07298872619867325, 0.9785500168800354]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9970>: [0.07010279595851898, 0.9814249873161316]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9a60>: [0.0694321021437645, 0.9804750084877014]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9b50>: [0.0909695103764534, 0.9729250073432922]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93d0>: [0.07424259930849075, 0.9815250039100647]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb94f0>: [0.0675642192363739, 0.9820749759674072]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9d90>: [0.0823562815785408, 0.9794250130653381]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9f10>: [0.059910938143730164, 0.982824981212616]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9ee0>: [0.07550592720508575, 0.9784500002861023]\n",
      "Global model test performance : [0.054279301315546036, 0.9863499999046326]\n",
      "\n",
      "\n",
      "\n",
      "Accuracy round 7\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b9d0>: [0.07781845331192017, 0.9795500040054321]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b880>: [0.06956273317337036, 0.9807000160217285]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b190>: [0.06851397454738617, 0.9798250198364258]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287380a0>: [0.06555607169866562, 0.9805750250816345]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287382b0>: [0.10768882930278778, 0.9672499895095825]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9250>: [0.10321928560733795, 0.9733750224113464]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93a0>: [0.062295496463775635, 0.9831249713897705]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9550>: [0.07237130403518677, 0.9779499769210815]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9610>: [0.07913729548454285, 0.9777250289916992]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb96d0>: [0.06271641701459885, 0.9824000000953674]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9520>: [0.07860808074474335, 0.9772499799728394]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9880>: [0.07382326573133469, 0.9792500138282776]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9970>: [0.10013846307992935, 0.9769499897956848]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9a60>: [0.06945876777172089, 0.9809749722480774]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9b50>: [0.09238098561763763, 0.9765999913215637]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93d0>: [0.06172364577651024, 0.9832249879837036]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb94f0>: [0.067879818379879, 0.9801750183105469]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9d90>: [0.08859730511903763, 0.9737499952316284]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9f10>: [0.06505817919969559, 0.9828749895095825]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9ee0>: [0.07730469107627869, 0.9786750078201294]\n",
      "Global model test performance : [0.05281278118491173, 0.9865750074386597]\n",
      "\n",
      "\n",
      "\n",
      "Accuracy round 8\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b9d0>: [0.08706668019294739, 0.9794999957084656]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b880>: [0.085691437125206, 0.9771999716758728]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b190>: [0.05836420878767967, 0.9835000038146973]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287380a0>: [0.06000976637005806, 0.9831749796867371]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287382b0>: [0.09888938814401627, 0.9727500081062317]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9250>: [0.08110594749450684, 0.9793999791145325]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93a0>: [0.0671541839838028, 0.9809749722480774]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9550>: [0.0602683424949646, 0.9830750226974487]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9610>: [0.06889133155345917, 0.9813249707221985]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb96d0>: [0.06423650681972504, 0.9815250039100647]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9520>: [0.06644138693809509, 0.9824249744415283]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9880>: [0.11808507144451141, 0.9707249999046326]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9970>: [0.08392486721277237, 0.979200005531311]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9a60>: [0.06113360449671745, 0.9835000038146973]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9b50>: [0.05811706557869911, 0.9833499789237976]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93d0>: [0.05898582190275192, 0.9834250211715698]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb94f0>: [0.0711522176861763, 0.9801250100135803]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9d90>: [0.08423629403114319, 0.9742500185966492]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9f10>: [0.05968204885721207, 0.9836750030517578]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9ee0>: [0.061004918068647385, 0.9815499782562256]\n",
      "Global model test performance : [0.05135490000247955, 0.9872000217437744]\n",
      "\n",
      "\n",
      "\n",
      "Accuracy round 9\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b9d0>: [0.07311484217643738, 0.982824981212616]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b880>: [0.06950072944164276, 0.9833499789237976]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f852878b190>: [0.083771251142025, 0.9767000079154968]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287380a0>: [0.05693419277667999, 0.9836249947547913]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f85287382b0>: [0.069190114736557, 0.9816750288009644]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9250>: [0.10311960428953171, 0.9767249822616577]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93a0>: [0.055994246155023575, 0.9855250120162964]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9550>: [0.06352025270462036, 0.9840999841690063]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9610>: [0.06562283635139465, 0.9816750288009644]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb96d0>: [0.05545197054743767, 0.9842000007629395]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9520>: [0.06858314573764801, 0.9827250242233276]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9880>: [0.07089690864086151, 0.9814000129699707]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9970>: [0.0736437663435936, 0.979075014591217]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9a60>: [0.09386664628982544, 0.979200005531311]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9b50>: [0.06405432522296906, 0.9822750091552734]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb93d0>: [0.07091104984283447, 0.981124997138977]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb94f0>: [0.06632877886295319, 0.9813249707221985]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9d90>: [0.0778079554438591, 0.9779000282287598]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9f10>: [0.05608365312218666, 0.9846749901771545]\n",
      "Test performance client <shfl.private.federated_operation.FederatedDataNode object at 0x7f84a7bb9ee0>: [0.07624192535877228, 0.9784749746322632]\n",
      "Global model test performance : [0.04923280328512192, 0.9880750179290771]\n",
      "\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "federated_government.run_rounds(10, test_data, test_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "pycharm": {
   "stem_cell": {
    "cell_type": "raw",
    "metadata": {
     "collapsed": false
    },
    "source": []
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}