WeightWatcher-Lightning-Flash.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "95fa8042",
   "metadata": {},
   "source": [
    "# Analyze Lightning Flash Tasks with WeightWatcher"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a273fdc",
   "metadata": {},
   "source": [
    "[Lightning Flash](https://github.com/PyTorchLightning/lightning-flash/tree/master/flash_examples) offers a variety of tasks backed by [PyTorch Lightning](https://github.com/PyTorchLightning/pytorch-lightning). Since these tasks are just ``nn.Modules`` we can analyze these tasks!\n",
    "\n",
    "Below we analyze the [Object Detection](https://lightning-flash.readthedocs.io/en/latest/reference/object_detection.html) Task, comparing the [alpha PL exponent](https://github.com/CalculatedContent/WeightWatcher#detecting-overtraining) across two different backbones."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b37e596e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a2eb07f",
   "metadata": {},
   "source": [
    "#### Import WeightWatcher & Flash ObjectDetector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ececca0c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import weightwatcher as ww\n",
    "\n",
    "from flash.image import ObjectDetector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "136694e0",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pytorch_lightning.utilities.distributed:Using 'efficientdet' provided by airctic/IceVision (https://github.com/airctic/icevision) and rwightman/efficientdet-pytorch (https://github.com/rwightman/efficientdet-pytorch).\n",
      "INFO:weightwatcher:\n",
      "\n",
      "python      version 3.8.8 (default, Apr 13 2021, 19:58:26) \n",
      "[GCC 7.3.0]\n",
      "numpy       version 1.21.2\n",
      "tensforflow version 2.6.0\n",
      "keras       version 2.6.0\n",
      "INFO:weightwatcher:Saving all images to ww-img\n",
      "INFO:weightwatcher:params {'glorot_fix': False, 'normalize': False, 'conv2d_norm': True, 'randomize': False, 'savedir': 'ww-img', 'savefig': 'ww-img', 'rescale': True, 'deltaEs': False, 'intra': False, 'channels': None, 'conv2d_fft': False, 'ww2x': False, 'min_evals': 0, 'max_evals': None, 'plot': False, 'mp_fit': False, 'layers': []}\n",
      "INFO:weightwatcher:Saving all images to ww-img\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 85%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 12%"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "/home/sean/anaconda3/lib/python3.8/site-packages/powerlaw.py:1615: RuntimeWarning: invalid value encountered in true_divide\n",
      "  CDF = CDF/norm\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 66%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "Assuming nested distributions\n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 95%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 80%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 97%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(40, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(240, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 95%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(240, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 79%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(40, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(240, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(240, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 27%"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(80, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 39%"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 35%"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(80, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 40%"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 78%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(80, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 98%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 94%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 99%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(112, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 99%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 96%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 99%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(112, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 99%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 22%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 56%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(112, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 27%"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 95%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 96%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 99%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 99%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 97%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 50%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 99%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 97%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 62%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 99%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 95%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 97%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 99%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 99%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(1152, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 97%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 99%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 57%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 39%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 98%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 23%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 42%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 98%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 98%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 03%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 11%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 00%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 06%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 01%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 03%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "Assuming nested distributions\n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 97%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "Assuming nested distributions\n"
     ]
    }
   ],
   "source": [
    "model = ObjectDetector(\n",
    "    head=\"efficientdet\", \n",
    "    backbone=\"d0\", \n",
    "    num_classes=91, # number of features in the CoCo dataset \n",
    "    image_size=128\n",
    ")\n",
    "watcher = ww.WeightWatcher(model=model)\n",
    "effecientdet_details = watcher.analyze()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f258a56a",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pytorch_lightning.utilities.distributed:Using 'faster_rcnn' provided by airctic/IceVision (https://github.com/airctic/icevision) and PyTorch/torchvision (https://github.com/pytorch/vision).\n",
      "INFO:weightwatcher:\n",
      "\n",
      "python      version 3.8.8 (default, Apr 13 2021, 19:58:26) \n",
      "[GCC 7.3.0]\n",
      "numpy       version 1.21.2\n",
      "tensforflow version 2.6.0\n",
      "keras       version 2.6.0\n",
      "INFO:weightwatcher:Saving all images to ww-img\n",
      "INFO:weightwatcher:params {'glorot_fix': False, 'normalize': False, 'conv2d_norm': True, 'randomize': False, 'savedir': 'ww-img', 'savefig': 'ww-img', 'rescale': True, 'deltaEs': False, 'intra': False, 'channels': None, 'conv2d_fft': False, 'ww2x': False, 'min_evals': 0, 'max_evals': None, 'plot': False, 'mp_fit': False, 'layers': []}\n",
      "INFO:weightwatcher:Saving all images to ww-img\n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 93%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 95%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 95%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 58%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 96%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 96%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 35%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 52%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 98%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 98%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 79%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 58%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 98%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 98%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 98%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 37%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 63%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 44%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 98%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 73%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 76%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n",
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 73%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 21%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 99%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 67%"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 87%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "/home/sean/anaconda3/lib/python3.8/site-packages/powerlaw.py:1408: RuntimeWarning: overflow encountered in double_scalars\n",
      "  C = ( self.Lambda**(1-self.alpha) /\n",
      "/home/sean/anaconda3/lib/python3.8/site-packages/powerlaw.py:1408: RuntimeWarning: invalid value encountered in double_scalars\n",
      "  C = ( self.Lambda**(1-self.alpha) /\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 91%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 99%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "'nan' in fit cumulative distribution values.\n",
      "Likely underflow or overflow error: the optimal fit for this distribution gives values that are so extreme that we lack the numerical precision to calculate them.\n",
      "WARNING:weightwatcher:pytorch layer: BatchNorm2d(1280, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  type LAYER_TYPE.UNKNOWN not found \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 99%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/sean/anaconda3/lib/python3.8/site-packages/powerlaw.py:1184: RuntimeWarning: overflow encountered in power\n",
      "  return x**-self.alpha\n",
      "/home/sean/anaconda3/lib/python3.8/site-packages/powerlaw.py:835: RuntimeWarning: invalid value encountered in multiply\n",
      "  likelihoods = f*C\n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 90%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n",
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 99%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 99%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 98%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xmin progress: 99%\r"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Assuming nested distributions\n"
     ]
    }
   ],
   "source": [
    "model = ObjectDetector(\n",
    "    head=\"faster_rcnn\", \n",
    "    backbone=\"mobilenet\", \n",
    "    num_classes=91, # number of features in the CoCo dataset \n",
    "    image_size=128\n",
    ")\n",
    "watcher = ww.WeightWatcher(model=model)\n",
    "faster_rcnn_details = watcher.analyze()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "aecda234",
   "metadata": {},
   "outputs": [],
   "source": [
    "params = {\n",
    "    'legend.fontsize': 'x-large',\n",
    "    'figure.figsize': (7,5),\n",
    "    'axes.labelsize': 'x-large',\n",
    "    'axes.titlesize':'xx-large',\n",
    "    'xtick.labelsize':'xx-large',\n",
    "    'ytick.labelsize':'xx-large'\n",
    "}\n",
    "plt.rcParams.update(params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "8303ed80",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfEAAAFcCAYAAADYsIdNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAmgUlEQVR4nO3de0BUdd7H8Q9C4AJe0GCwNFuTrIzMsos9FhsuUuEdTfGSmZds1TIzJTNLS3NR29S20twsrVhTM00qS1ylnsr0MRdztYuFgsqgghfEuAzn+cOabeQ2MB6Zg+/XP80553fO+c532z6cOTPn52MYhiEAAGA59Wq7AAAAUDOEOAAAFkWIAwBgUYQ4AAAWRYgDAGBRhDgAABblV9sFVNfhwyfdHhsSEqi8vAITq6nb6J/n6KFn6J9n6J9nvKl/oaENyl1fp6/E/fx8a7sES6N/nqOHnqF/nqF/nrFC/+p0iAMAUJcR4gAAWJSpIZ6WlqbY2FjFxMRo0aJFZbZv2bJFN954o3r06KEePXropZdeMrMcAADqFNO+2OZwODR9+nQtWbJENptNffr0UXR0tFq3bu0yrkOHDlq4cKFZZQAAUGeZdiWenp6uli1bqkWLFvL391dcXJxSU1PNOh0AABcc067E7Xa7wsPDncs2m03p6ellxu3YsUPdu3dXWFiYJk2apIiIiEqPGxISWK1vDFb0tXy4h/55jh56hv55hv55xtv7Z1qIlzfDqY+Pj8ty27ZttXHjRgUFBWnz5s0aPXq0Pvnkk0qPW53f7IWGNqjW78rhiv55jh56hv55hv55xpv6d95/Jx4eHq7s7Gznst1uV1hYmMuY4OBgBQUFSZKioqJUUlKi3Nxcs0oCAFjUxo0bNHBgH40d+6Ak6emnJ2vIkP5avvztah9r1KgHalzHhx9+oCNHDpe7bfv2bZo4cVyl+//ww3f68svPa3z+s5l2JR4ZGamMjAxlZmbKZrMpJSVFc+fOdRlz+PBhXXzxxfLx8VF6erpKS0sVEhJiVkkAgGoKTJp5To9XMHFyjfZbt26NHnssUTfc0EFHjx7Rt9+ma9WqdTU61quvvl6j/aQzId6q1RW6+OLQGu3/ww/fa8+e/6hjx041ruH3TAtxPz8/TZ06VcOHD5fD4VB8fLwiIiKUnJwsSUpISND69euVnJwsX19f1a9fXy+88EKZj9wBABeW9es/1MqV/1RxcYmuuaatLr44VDt37tDs2QfUqVOUvv76S+Xl5en++wfo0Ucf18UXh2ru3L/q2LE81a9fX5MmTVHLlpcrN/eoZs9+XgcPHpAkTZiQqMjIdoqJuV2ffvqZJOmdd5Zq48YNKi4u0h133Klhwx7UoUMHNWHCw7r55pu0dev/KTQ0VLNmzdUXX3yu777brWnTpiggoL4WLnxd33yzXfPnz1WjRo3Vps1Vzvdw+vRp/e1vSfrpp71yOEr0wAMjdeut/6PFi19VUVGh0tP/rcGD71fnzl086pWpz06PiopSVFSUy7qEhATn60GDBmnQoEFmlgAAsJCMjJ+VmvqpXnnldfn5+WnOnFlq1uwStWlztcaMGaerrrpGvXv31cSJ4/TGG+9Ikh555CFNmPCEWrS4TLt2fau5c2dp/vxX9eKLc9S+/Q16/vk5cjgcOn36tMu5vv76K2VmZuq1196UYRhKTByvHTu2y2YLV1ZWpubNe1GPPDJJTz2VqE2bNio29h6tWvWus47CwkIlJc3QvHmvqHnzFpo69QnnsZcufV033niTJk9+WidPntSIEUPUocMtGj58lPbs+Y/Gj590TvpluQlQAAB11//939f67rvdGj78PklSYeEvld5mLSgo0M6d6XrqqUTnuuLiIknS9u1bNWXKNEmSr6+vgoODXfb9+uuvtHXrVxo6dKAk6fTpAmVl7ZfNFq5mzS7R1VdfrcOHT6pNm6t06NDBMufevz9DzZpdohYtLpMkxcberbVrVzuP/fnnm5Wc/JYkqaioUHZ7dpljeIoQr6HApJk1vrcDACifYRi6++6uGjVqjMv6MWNGVjC+VA0aBDuvyqt7rkGD7lfPnvEu6w8dOqiLLrrIuVyvnq8cjsJyj1HRLWDDMDRjRpIuu+xyl/X/+c+31a6zMjw7HQDgNW688WZt2pSqvLwzv1Q6ceK4srMPVTg+KChYzZpdqo0bN0g6E54//PD9r8e6Se+/v1LSmaeInjqV77LvLbd0VErKWhUUnPnp8uHDOc7zViQwMMg5/rLLLtfBgwd04ECWJOnTT9e7HHvlyuXOn1t///2eX/cPdO5/LhDiAACv8cc/ttKIEQ/p0UfHaMiQ/ho3brSOHDlS6T5Tpz6rdevWaMiQBA0efK8+/3yzJOmRRyZo+/Ztuu++fho2bLB+/vknl/1uvvlWxcTcpVGjhuq++/ppypRJVQbsPfd01ezZM3X//QMkGZo48Uk9/vgjeuihYQoPb+Ycd//9w1RSUqIhQ/pr8OB7tXjxq5KkG27ooIyMn3X//QOUmlr5c1Hc4WOU91QWL1adH96b+UP9C+HjdG960IFV0UPP0D/P0D/PeFP/zvvDXgAAgLkIcQAALIoQBwDAoghxAAAsihAHAMCiCHEAACyKEAcAeJUVK/6pgQP7aNq0KdXa7+TJk3rvvRUmVeWdeOwqAKBCYS83PKfHy/nLiSrHrF69QnPmzNcll1xarWPn55/U6tUr1Lt3X7f3MQxDhmGoXr3yr2kdDod8fX2rVcf5RIgDALzG7NkzdfDgASUmjldcXHdFRrbT/PkvqLDwFwUE1NfkyVN12WWX66ef9ur556epuLhEhlGq555L0uLFr+jAgQO6//4BuummWzR69COVTjXavn0H7dqVruefn+vytLU+fbopLq67vvlmq7p3j1dwcAMtWvR3ORylaty4sebNe0X/+MdC2e3ZOnjwgOx2u+69N0F9+/Z3Hvu6667Xzp3pzmlMAwLqm9IvQhwA4DUef3yytmz5UvPnL1Tjxo116lS+Xnppkfz8/LR16xYtXPh3zZgxW2vWrFLfvgnq0uVuFRcXq7TUoVGjxuqnn/Y6J0OpbKrR/fv36YknntaECYnl1uHv76/k5GR9//1+DRs2SC+9tEiXXHKpTpw47hyzf/8+zZ//qgoKCjRgQLx69eojScrKytQzz8zQpElTXKYxNQMhDgDwWvn5+XruuWeUlbVfPj4+KikpkSS1bXudli59XTk5dkVFRTunA/29yqYaDQ9vpmuvjazwvJ07d5Ek7dq1U+3atXd+tN+wYSPnmI4d/0f+/v7y9/dXSEiIcnOPSpKaNbtEERFtJKnCaUzPFUIcAOC1Fi9+VTfc0EHPPz9Hhw4d1NixD0qSunS5S23bXqsvvvhc48ePVWLilDL30CubarR+/co/3q5f/w+/HaXC6UYvusjf+bpevXpyOBy/rndvGtNzgW+nAwC8Vn5+vkJDQyVJH374gXP9gQNZuuSSS9W3b3916nSH9u79ocw0nzWZavRsbdtepx07tuvgwQOS5PJxujfgShwA4LUGDrxPzz33jJYvf1s33HCTc/3GjZ9q/fqP5OfnpyZNmmro0OFq2LCRIiPbafDge3Xrrf+j0aMfUUbGzxo1aqgk6Q9/CNTUqc9W+E308oSEhOjxxyfryScfV2mpoZCQEL344svn/H3WFFOR1hBTkcId9NAz9M8z9M8z3tQ/piIFAKCOIcQBALAoQhwAAIsixAEAsChCHAAAiyLEAQCwKEIcAACLIsQBALAontjmgbPn2XVnnlwAAM4VrsQBALAoQhwAAIsixAEAsChCHAAAiyLEAQCwKEIcAACLIsQBALAoQhwAAIsixAEAsChCHAAAiyLEAQCwKEIcAACLIsQBALAoQhwAAIsixAEAsChCHAAAiyLEAQCwKEIcAACLIsQBALAoQhwAAIsyNcTT0tIUGxurmJgYLVq0qMJx6enpuvrqq/Xxxx+bWQ4AAHWKaSHucDg0ffp0LV68WCkpKVq3bp1+/PHHcsfNmTNHnTp1MqsUAADqJNNCPD09XS1btlSLFi3k7++vuLg4paamlhm3bNkyxcbGqmnTpmaVAgBAnWRaiNvtdoWHhzuXbTab7HZ7mTEbNmxQ//79zSoDAIA6y8+sAxuGUWadj4+Py/KMGTM0YcIE+fr6un3ckJBA+fm5Pz40tIHbY6slKOD8nasW1cX3dL7RQ8/QP8/QP894e/9MC/Hw8HBlZ2c7l+12u8LCwlzGfPvttxo/frwkKS8vT5s3b5afn5/+/Oc/V3jcvLwCt2sIDW2gw4dPVrNy9wSeKpSCXdeZda7aYmb/LhT00DP0zzP0zzPe1L+K/pgwLcQjIyOVkZGhzMxM2Ww2paSkaO7cuS5jNm7c6HydmJioP/3pT5UGOAAA+C/TQtzPz09Tp07V8OHD5XA4FB8fr4iICCUnJ0uSEhISzDo1AAAXBNNCXJKioqIUFRXlsq6i8J41a5aZpQAAUOfwxDYAACyKEAcAwKIIcQAALIoQBwDAoghxAAAsihAHAMCiCHEAACyKEAcAwKIIcQAALIoQBwDAoghxAAAsihAHAMCiCHEAACyKEAcAwKIIcQAALIoQBwDAoghxAAAsihAHAMCiCHEAACzKr7YLqEsCk2Y6XxdMnFyLlQAALgRciQMAYFGEOAAAFkWIAwBgUYQ4AAAWRYgDAGBRhDgAABZFiAMAYFGEOAAAFkWIAwBgUYQ4AAAWRYgDAGBRhDgAABZFiAMAYFGEOAAAFkWIAwBgUYQ4AAAWRYgDAGBRhDgAABZFiJskMGlmbZcAAKjjCHEAACyKEAcAwKIIcQAALIoQBwDAoghxAAAsihAHAMCiCHEAACyKEAcAwKIIcQAALIoQBwDAokwN8bS0NMXGxiomJkaLFi0qs33Dhg3q1q2bevTood69e2vbtm1mlgMAQJ3iZ9aBHQ6Hpk+friVLlshms6lPnz6Kjo5W69atnWM6duyozp07y8fHR3v27NG4ceP08ccfm1USAAB1imlX4unp6WrZsqVatGghf39/xcXFKTU11WVMUFCQfHx8JEmnT592vgYAAFUz7UrcbrcrPDzcuWyz2ZSenl5m3Keffqq5c+cqNzdXCxcuNKscAADqHLdCfNiwYRo0aJD+9Kc/uX21bBhGmXXl7RsTE6OYmBht3bpV8+bN0xtvvFHpcUNCAuXn5+tWDZIUGtrA7bHVEhRQdtVZ64LMOvd5ZFr/LiD00DP0zzP0zzPe3j+3Qrxfv35688039dxzz6lfv37q27evQkJCKt0nPDxc2dnZzmW73a6wsLAKx990003av3+/cnNz1aRJkwrH5eUVuFOypDPNP3z4pNvjqyPwVKEU7Lru1KlCl+UCk859vpjZvwsFPfQM/fMM/fOMN/Wvoj8m3Lon3qVLF73xxht67bXXlJOTo65du2rixIn69ttvK9wnMjJSGRkZyszMVFFRkVJSUhQdHe0yZt++fc4r9l27dqm4uLjKPw4AAMAZNbonftFFFykgIECTJk3S7bffrsTExLIH9vPT1KlTNXz4cDkcDsXHxysiIkLJycmSpISEBK1fv15r1qyRn5+f6tevr7/97W98uQ0AADe5FeKffPKJ3nrrLR09elQDBgxQSkqKgoKCVFJSoi5dupQb4pIUFRWlqKgol3UJCQnO1yNHjtTIkSM9KN9zgUkzJUkFEydXaxsAALXNrRBfuXKlRowYodtvv911Zz8/TZkyxZTCAABA5dwK8YULF1b4MffZ97kBAMD54dYX2wYMGKDjx487l48dO6aBAweaVhQAAKiaWyFeUFCgRo0aOZcbN26s/Px804oCAABVcyvES0tLVVDw399nnzp1Sg6Hw7SiAABA1dy6J961a1c98MADzm+WJycnq3v37qYWBgAAKudWiD/44IMKCwvTxo0bZRiG+vfvr549e5pcGgAAqIzbD3vp1auXevXqZWYtAACgGtwK8aNHj2rZsmXKzMxUSUmJc/28efNMKwwAAFTOrRAfO3asrrjiCnXs2FG+vu7PIAYAAMzjVoifOHFCzz77rNm1AACAanArxCMiImS322Wz2cyup04Le7mhy3LOX07UUiUAgLrA7Svx7t27q3379goICHCu5544AAC1x+3fiXft2tXsWgAAQDW4FeL8tAwAAO/j1mNXMzIylJCQ4JyxbNeuXVqwYIGphQEAgMq5FeLPPPOMHnroITVo0ECSdPXVV+vjjz82tTAAAFA5t0L85MmTuuOOO5xziterV08XXXSRqYUBAIDKuXVP3NfXV8XFxc4Qt9vtqlfPrfyvMwKTZkqSCiZOrvG+Cj6XFQEALnRuJfGAAQM0ZswY5eXlacGCBRowYIAeeOABs2sDAACVcOtKvGfPnmrevLn+9a9/6fTp0/rrX/+qDh06mF0bAACohNuzmHXo0IHgBgDAi7gV4vHx8c774b+3cuXKc14QAABwj1shPmnSJOfrwsJCpaSkKCwszLSiAABA1dwK8ZtvvtlluVOnTnyxDQCAWlaj34nl5+crMzPzXNcCAACqodr3xEtLS5WVlaWhQ4eaWhgAAKhcte+J+/r6qnnz5swtDgBALavRPXEAAFD73ArxW2+9tdyfmBmGIR8fH3355ZfnvDAAAFA5t0I8ISFBx44dU79+/WQYhlatWiWbzaZ77rnH7PoAAEAF3ArxrVu36q233nIuT5kyRYMGDdKIESNMK6wuCHu54ZkXTHwCADCBWz8xy8nJUW5urnM5NzdXhw8fNq0oAABQNbeuxIcMGaIePXrozjvvlCRt3rxZDz74oKmFAQCAyrkV4gMHDtSNN96orVu3yjAMDRw4UG3atDG7NgAAUAm3ZzFr3ry5HA6H2rZta2Y9AADATW7dE9+8ebPi4uI0duxYSdLOnTs1atQoUwsDAACVcyvE58+fr5UrV6phwzPfto6MjNT+/ftNLcybBCbNdHn9+2UAAGqL2xOghIaGuiz7+/uf82IAAID73ArxoKAgHTlyxPnUti1btqhBgwamFgYAACrn1hfbJkyYoBEjRigrK0uDBw9WRkaGXnnlFbNrAwAAlagyxEtLS+Xv76+lS5dq+/btkqT27ds7748DAIDaUWWI16tXT08++aRWrVqlqKio81ETAABwg1v3xK+44gplZWWZXQsAAKgGt+6J5+bmqnv37rrxxhsVGBjoXD9v3jzTCgMAAJWrNMRnzZqlxMRExcXFKSYmhp+VAQDgRSoN8S1btkiSevXqpV69emn16tXnpSgAAFC1Su+JG4ZR7msAAFD7Kr0SLyoq0t69e2UYhsvr37Ru3dr0AgEAQPkqDfFffvlFI0aMcC7//rWPj49SU1PNqwwAAFSq0hDfuHGjRwdPS0vTjBkzVFpaqr59+2rkyJEu29euXavXXntN0plHuz7zzDO66qqrPDonAAAXCrfnE68uh8Oh6dOna8mSJbLZbOrTp4+io6NdPoJv3ry53nrrLTVq1EibN2/WU089pRUrVphVUrUFB89yWc7PT6ylSgAAKMvtWcyqKz09XS1btlSLFi3k7++vuLi4Mh+/33DDDWrUqJEk6frrr1d2drZZ5QAAUOeYdiVut9sVHh7uXLbZbEpPT69w/MqVK3XHHXdUedyQkED5+fm6XUdoaBWzrQUFnPlHVeMkBf06tqbbz1ZlbV7ACjV6O3roGfrnGfrnGW/vn2khXt5P0n6byvRsX331lVauXKl33nmnyuPm5RW4XUNoaAMdPnyy0jGBpwolSQVVjJOkU7+OdQqu3vazVVVbbXOnf6gcPfQM/fMM/fOMN/Wvoj8mTAvx8PBwl4/H7Xa7wsLCyozbs2ePpkyZotdee00hISFmlQMAQJ1j2j3xyMhIZWRkKDMzU0VFRUpJSVF0dLTLmIMHD2rs2LFKSkrSH//4R7NKAQCgTjLtStzPz09Tp07V8OHD5XA4FB8fr4iICCUnJ0uSEhIS9Pe//13Hjh3TtGnTJEm+vr567733zCoJAIA6xbQQl6SoqKgyc5AnJCQ4X8+YMUMzZswwswQAAOos0z5OBwAA5iLEAQCwKEIcAACLIsQBALAoQhwAAIsixAEAsChCHAAAiyLEAQCwKEIcAACLIsQBALAoQhwAAIsixAEAsChTJ0C50AQHz6rtEgAAFxCuxAEAsChCHAAAiyLEAQCwKEIcAACLIsQBALAoQhwAAIsixAEAsChCHAAAiyLEAQCwKEIcAACLIsQBALAoQhwAAIsixAEAsChCHAAAiyLEAQCwKEIcAACLIsQBALAoQhwAAIvyq+0C4L18pvm4LOf85UQtVQIAKA9X4gAAWBQhDgCARRHiAABYFCFejsCkmQpMmnlezgMAQE0R4gAAWBQhDgCARRHiAABYFCEOAIBFEeIAAFgUIQ4AgEUR4gAAWBQhDgCARRHiAABYFCEOAIBFEeIAAFgUIQ4AgEUR4gAAWJSpIZ6WlqbY2FjFxMRo0aJFZbbv3btX/fr107XXXqt//OMfZpYCAECd42fWgR0Oh6ZPn64lS5bIZrOpT58+io6OVuvWrZ1jGjdurCeffFKpqalmlQEAQJ1lWoinp6erZcuWatGihSQpLi5OqampLiHetGlTNW3aVJs3bzarDK8WHDxLenmWcznnLydctoe93NBl+eztAIALm2khbrfbFR4e7ly22WxKT0/3+LghIYHy8/N1e3xoaIPKBwQFnPnH78f9uq7M0ArWnytV1VrlezFZbZ/fquibZ+ifZ+ifZ7y9f6aFuGEYZdb5+Ph4fNy8vAK3x4aGNtDhwycrHRN4qlCSVPC7cb+tU7Dr2FO/rf/NWds9VVWtVW03W22f34rc+XcQFaN/nqF/nvGm/lX0x4RpX2wLDw9Xdna2c9lutyssLMys050zgUkza7sEAADcYlqIR0ZGKiMjQ5mZmSoqKlJKSoqio6PNOh0AABcc0z5O9/Pz09SpUzV8+HA5HA7Fx8crIiJCycnJkqSEhAQdPnxY8fHxys/PV7169fTmm2/qww8/VHDwOf6cGgCAOsi0EJekqKgoRUVFuaxLSEhwvg4NDVVaWpqZJQAAUGfxxDYAACyKEAcAwKIIcQAALIoQBwDAoghxAAAsihAHAMCiCHEAACyKEAcAwKIIcQAALIoQBwDAoghxAAAsihAHAMCiCHEAACzK1FnM6prg4Fm1XUKlwl5uWOn2nL+cOE+VAADOB67EAQCwKEIcAACLIsQBALAoQhwAAIsixAEAsChCHAAAiyLEAQCwKEIcAACLIsQBALAoQhwAAIsixAEAsChCHAAAiyLEAQCwKEIcAACLIsQBALAoQhwAAIsixAEAsChCHAAAiyLEAQCwKL/aLgAA3BH2ckOX5Zy/nKilSuq+ut7rc/3+arNfhLgXCUya6boiuOJxBRMnu+zz23Jt8IYa6gL6CKC6+DgdAACLIsQBALAoQhwAAIsixAEAsChCHAAAiyLEAQCwKEIcAACLIsQBALAoQhwAAIsixAEAsChCHAAAiyLEAQCwKEIcAACLIsQBALAoU0M8LS1NsbGxiomJ0aJFi8psNwxDzz33nGJiYtStWzft2rXLzHIAAKhTTAtxh8Oh6dOna/HixUpJSdG6dev0448/uoxJS0tTRkaGPvnkEz377LN65plnzCoHAIA6x7QQT09PV8uWLdWiRQv5+/srLi5OqampLmNSU1PVs2dP+fj46Prrr9eJEyeUk5NjVkkAANQppoW43W5XeHi4c9lms8lut1c6Jjw8vMwYAABQPj+zDmwYRpl1Pj4+1R5zttDQBtWqo8rxs5+XJAWdtSxJhp4vO/48qvD8s58vU2+QJOPpsv306PzuHu/sHsKF2//O0sdy/da/c/3v94Wiuv/NlOp+r6vz/tzpX232y7Qr8fDwcGVnZzuX7Xa7wsLCKh2TnZ1dZgwAACifaSEeGRmpjIwMZWZmqqioSCkpKYqOjnYZEx0drffff1+GYWjHjh1q0KABIQ4AgJtM+zjdz89PU6dO1fDhw+VwOBQfH6+IiAglJydLkhISEhQVFaXNmzcrJiZGf/jDHzRz5kyzygEAoM7xMcq7MQ0AALweT2wDAMCiCHEAACzKtHvitS0tLU0zZsxQaWmp+vbtq5EjR9Z2SV7tiSee0KZNm9S0aVOtW7dOknTs2DE9+uijOnDggC699FK9+OKLatSoUS1X6p0OHTqkiRMn6siRI6pXr57uvfdeDRkyhB66qbCwUAMHDlRRUZEcDodiY2P18MMP079q+u37RzabTQsXLqR/1RQdHa2goCDVq1dPvr6+eu+997y+h3XyStydR77CVe/evbV48WKXdYsWLVLHjh31ySefqGPHjuU+/x5n+Pr6KjExUR999JGWL1+ud955Rz/++CM9dJO/v7/efPNNrV27Vu+//74+++wz7dixg/5V09KlS3XFFVc4l+lf9b355ptas2aN3nvvPUne38M6GeLuPPIVrm666aYyf13+9lhcSerZs6c2bNhQC5VZQ1hYmNq2bStJCg4OVqtWrWS32+mhm3x8fBQUdOYxNyUlJSopKZGPjw/9q4bs7Gxt2rRJffr0ca6jf57z9h7WyRB355GvqNrRo0edv9sPCwtTbm5uLVdkDVlZWdq9e7fatWtHD6vB4XCoR48euu2223TbbbfRv2qaOXOmHn/8cdWr99//rNO/6hs2bJh69+6t5cuXS/L+HtbJe+I1eZwrcC6cOnVKDz/8sCZPnqzg4ODaLsdSfH19tWbNGp04cUKjR4/W999/X9slWca//vUvNWnSRNdee622bNlS2+VYVnJysmw2m44ePaqhQ4eqVatWtV1SlepkiLvzyFdUrWnTpsrJyVFYWJhycnLUpEmT2i7JqxUXF+vhhx9Wt27d1KVLF0n0sCYaNmyoW265RZ999hn9c9P27du1ceNGpaWlqbCwUPn5+ZowYQL9qyabzSbpzP9vY2JilJ6e7vU9rJMfp7vzyFdU7bfH4krS+++/r86dO9duQV7MMAw9+eSTatWqlYYOHepcTw/dk5ubqxMnTkiSfvnlF33xxRdq1aoV/XPTY489prS0NG3cuFEvvPCCbr31Vs2ZM4f+VUNBQYHy8/Odr//3f/9XERERXt/DOvvEts2bN2vmzJnOn1w89NBDtV2SVxs/fry+/vpr5eXlqWnTpho7dqz+/Oc/a9y4cTp06JCaNWumefPmqXHjxrVdqlfatm2bBg4cqCuvvNJ5T3L8+PG67rrr6KEb9uzZo8TERDkcDhmGobvuuktjxoxRXl4e/aumLVu26PXXX9fChQvpXzVkZmZq9OjRks58P6Nr16566KGHvL6HdTbEAQCo6+rkx+kAAFwICHEAACyKEAcAwKIIcQAALIoQBwDAoghxAAAsihAHAMCiCHHABG3atNGpU6fO2biqjBkzRunp6VWOKyoqUu/evXXy5MkKx0RHR+uuu+5S9+7d1bVrV6WkpDjXXwjPM1+wYIGKiopquwzALYQ4YHH//ve/dfr0aV133XVVjvX391f37t21ZMmSSsfNnz9fa9euVVJSkp544gmvm7nJTC+99JKKi4truwzALYQ44IHHHntMvXv3Vrdu3TR69GgdP368zJg2bdpowYIF6t+/v2JjY7V+/XqX7cuWLVN8fLw6d+7sss2dY0vS8uXL1bVrV5d1OTk5GjdunHr27KkePXro+++/V//+/VVaWqquXbtq5cqVbr2/a665RkFBQcrKynJrvHTmj4rBgwerd+/e6t27tzZt2iRJ2rt3r6KionTgwAFJZ654H330UUmV9ygtLU09e/ZUt27dNGTIEO3bt8+5rU2bNnr11VfL7V9FdVS237Rp0yRJ/fv3V48ePZzPcwe8lgGgxo4ePep8/cILLxizZ882DMMwrrzySiM/P9/5esGCBYZhGMbevXuNm2++2Thy5Ihz27JlywzDMIxt27YZnTp1qvLYZ+vcubPx448/OpdLS0uNPn36GGvWrDEMwzCmTJliREdHG5s2bXKOiY6Odtnn9+68807ju+++MwzDML788kujffv2xvHjx13WV+T48eNGjx49DLvdbhiGYdjtduP22283jh8/bhiGYaxevdro27ev8dlnnxldunQxTp48WWmPjhw5Ytxyyy3GDz/8YBiGYbz77rtGnz59nOerqH9V1VFZ33//vx3g7erkVKTA+bJmzRp98MEHKi4uVkFBgS6//PJyx/Xt21eS1KpVK11zzTXasWOHczake+65R5J0/fXXKycnR4WFhQoICHD72NnZ2br44oudyzt27FBhYaG6d+8uSQoLC9Mll1yiqKgo55jQ0FDZ7XZdccUV5R7z4YcfVkBAgIKDg7VgwQI1bNjQrX588803ysrK0ogRI5zrfHx8tG/fPkVGRqpnz5766quvNHr0aL399tsuc66X1yMfHx9dddVVat26tSQpPj5e06ZNU35+vnPf8vpXVR0V7RcQEODW+wS8BSEO1NC2bduUnJysf/7zn2rSpIk++OADvfvuu1XuZxiGfHx8nMu/BYevr68kqaSkRDt37nT72PXr11dhYaFzOT09Xddff71zeffu3YqPj3fZp6rAmj9/vq688soq30t5761NmzZ6++23y91eVFSkH374QQ0aNNDRo0crPY6Pj0+ZXpWnvP5VVUdF+xHisBruiQM1dOLECQUHB6tx48YqKirSqlWrKhz727aMjAzt3r1b7dq1O2fHvvLKK/Xzzz87l/38/HTw4EFJ0oYNG/TZZ5+5fFHL4XAoMzOzRiFdlfbt22vfvn366quvnOvS09Nl/DpZYlJSktq2baslS5bo6aefVnZ2tnNceT1q3769du/erb1790qSVq9erWuuucblCr4mdVQmKCjIOa804O24Egdq6I477tDatWt19913y2az6dprr9XOnTvLHevv76/+/fsrLy9P06dPV9OmTc/Zsbt06aLPP/9ct9xyiyTp7rvv1kcffaS77rpLzZo105w5czRjxgxFRkbqqquu0vbt29WuXTs1aNCg2u956NChzitXSfrggw/UqFEj53KjRo308ssva/bs2Zo5c6aKi4vVokULvfrqq0pNTdXXX3+tFStWKCAgQKNHj9b48eO1dOnSSnuUlJSkCRMmqKSkRE2aNNHs2bOrrLOyOqq6sn/ggQd03333qX79+lq2bJnbtxKA2sB84oDJ2rRpo+3btysoKMiU4+fn5yshIUErVqxQ/fr1qxz/2GOPKT4+Xrfddpsp9dSE2T0C6io+TgcsLjg4WImJiW79DKyoqEgdOnTwqgAHUHNciQMAYFFciQMAYFGEOAAAFkWIAwBgUYQ4AAAWRYgDAGBRhDgAABZFiAMAYFGEOAAAFvX/Fe3YfH33Ni4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x396 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "effecientdet_details.alpha.plot.hist(bins=100, color='red', alpha=0.5, density=True, label='effecientdet')\n",
    "faster_rcnn_details.alpha.plot.hist(bins=100, color='green', density=True, label='faster rcnn')\n",
    "plt.legend()\n",
    "plt.xlabel(r\"alpha $(\\alpha)$ PL exponent\")\n",
    "plt.style.use(\"seaborn\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f31afe94",
   "metadata": {},
   "source": [
    "The alpha metric can be used to detect overfitting and other generalization analytics. See the [README](https://github.com/CalculatedContent/WeightWatcher#advanced-usage) for more information and visit the [Lightning Flash documentation](https://lightning-flash.readthedocs.io/en/latest/index.html) to see more tasks!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}