AE_Intel.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Artifacts Evaluation Instructions: #26 Automating Reinforcement Learning Architecture Design for Code Optimization\n",
    "\n",
    "## Preliminaries\n",
    "\n",
    "This interactive Jupyter notebook provides a small-scale demo for showcasing the task definition, client RL search, client RL parameter tuning and evaluation of case studies reported in the paper.\n",
    "\n",
    "The main results of our CC 2022 paper are comparing the performance of our Supersonic-tuned client RL against prior search-based techniques. \n",
    "The evaluation presented in our paper ran on a much larger dataset and for longer. The idea is that this notebook contains minimal working examples which can be evaluated in a reasonable amount of time. \n",
    "\n",
    "## Instructions for Experimental Workflow:\n",
    "\n",
    "Before you start, please first make a copy of the notebook by going to the landing page. Then select the checkbox next to the notebook titled *main.ipynb*, then click \"**Duplicate**\".\n",
    "\n",
    "Click the name of the newly created Jupyter Notebook, e.g. **AE_Intel-Copy1.ipynb**. Next, select \"**Kernel**\" > \"**Restart & Clear Output**\". Then, repeatedly press the play button (the tooltip is \"run cell, select below\") to step through each cell of the notebook.\n",
    "\n",
    "Alternatively, select each cell in turn and use \"**Cell**\"> \"**Run Cell**\" from the menu to run specific cells. Note that some cells depend on previous cells being executed. If any errors occur, ensure all previous cells have been executed.\n",
    "\n",
    "## Important Notes\n",
    "\n",
    "**Some cells can take more than half an hour to complete; please wait for the results until step to the next cell.** \n",
    "\n",
    "High load can lead to a long wait for results. This may occur if multiple reviewers are simultaneously trying to generate results. \n",
    "\n",
    "The experiments are customisable as the code provided in the Jupyter Notebook can be edited on the spot. Simply type your changes into the code blocks and re-run using **Cell > Run Cells** from the menu.\n",
    "\n",
    "## Links to The Paper\n",
    "\n",
    "For each step, we note the section number of the submitted version where the relevant technique is described or data is presented.\n",
    "\n",
    "The main results are presented in Figures 3-6 of the submitted paper. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Demo 1: Client RL Search\n",
    "\n",
    "This demo corresponds to the simplified search space definition example given in Figure 2. Note that we have refactored the code; hence there are small changes in the API. This is a small-scale demo for case study 4 of superoptimization. The full-scale evaluation used in the paper takes over 24 hours to run. \n",
    "\n",
    "## Step 1. Task definition\n",
    "\n",
    "A compiler developer can use the Supersonic API to define the optimization problem. This is done by creating an RL policy interface. The definition includes a list of client RL components for the meta-optimizer to search over.\n",
    "\n",
    "#### *statefs*: \n",
    "\n",
    "This parameter defines the candidate state functions. \n",
    "\n",
    "- *Word2vec* for embedding programs with word2vec model;\n",
    "- *Doc2vec* for embedding programs with Doc2vec model;\n",
    "- *Bert* for embedding programs with Bert model;\n",
    "- *Actionhistory* for a hash function to the action history to represent the program.\n",
    "\n",
    "#### *rewards*: \n",
    "\n",
    "This parameter defines the candidate reward functions. \n",
    "\n",
    "- *relative_measure* for using speedup, code size reduction ratio as a reward;\n",
    "- *tan* for using a tan function to process original feedback, e.g. tan(run time) ;\n",
    "- *func* for using a mapping function to process original feedback, e.g. better performance gives 1, worse performance gives 0;\n",
    "- *weight* for assigning a weight number to different feedback. e.g. (weight_1*runtime + weight_2*memory_use + weight_3*code_size)\n",
    "\n",
    "#### *rl_algs*: \n",
    "\n",
    "This parameter defines candidate RL algorithms. Possible values are:\n",
    "\n",
    "- *\"MCTS\", \"PPO\", \"APPO\", \"A2C\", \"DQN\", \"QLearning\", \"MARWIL\", \"PG\", \"SimpleQ\", \"A3C\", \"ARS\", \"ES\", \"BC\"*  We don't recommend using algorithms A3C, ARS and ES due to long search time. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The search space consists of 47 combinations of RL components.\n",
      "A few examples include:\n",
      "---------------------------------------------------------------------------------------------------------\n",
      "{'StatList': 'Word2vec', 'ActList': 'init', 'RewList': 'relative_measure', 'AlgList': 'MCTS'}\n",
      "{'StatList': 'Word2vec', 'ActList': 'init', 'RewList': 'relative_measure', 'AlgList': 'DQN'}\n",
      "{'StatList': 'Word2vec', 'ActList': 'init', 'RewList': 'tan', 'AlgList': 'MCTS'}\n",
      "{'StatList': 'Word2vec', 'ActList': 'init', 'RewList': 'tan', 'AlgList': 'DQN'}\n",
      "{'StatList': 'Word2vec', 'ActList': 'init', 'RewList': 'func', 'AlgList': 'MCTS'}\n",
      "{'StatList': 'Word2vec', 'ActList': 'init', 'RewList': 'func', 'AlgList': 'DQN'}\n",
      "{'StatList': 'Word2vec', 'ActList': 'init', 'RewList': 'weight', 'AlgList': 'MCTS'}\n",
      "{'StatList': 'Word2vec', 'ActList': 'init', 'RewList': 'weight', 'AlgList': 'DQN'}\n",
      "{'StatList': 'Doc2vec', 'ActList': 'init', 'RewList': 'relative_measure', 'AlgList': 'MCTS'}\n",
      "{'StatList': 'Doc2vec', 'ActList': 'init', 'RewList': 'relative_measure', 'AlgList': 'DQN'}\n",
      "{'StatList': 'Doc2vec', 'ActList': 'init', 'RewList': 'tan', 'AlgList': 'MCTS'}\n",
      "{'StatList': 'Doc2vec', 'ActList': 'init', 'RewList': 'tan', 'AlgList': 'DQN'}\n",
      "{'StatList': 'Doc2vec', 'ActList': 'init', 'RewList': 'func', 'AlgList': 'MCTS'}\n",
      "{'StatList': 'Doc2vec', 'ActList': 'init', 'RewList': 'func', 'AlgList': 'DQN'}\n",
      "{'StatList': 'Doc2vec', 'ActList': 'init', 'RewList': 'weight', 'AlgList': 'MCTS'}\n",
      "{'StatList': 'Doc2vec', 'ActList': 'init', 'RewList': 'weight', 'AlgList': 'DQN'}\n",
      "{'StatList': 'Bert', 'ActList': 'init', 'RewList': 'relative_measure', 'AlgList': 'MCTS'}\n",
      "{'StatList': 'Bert', 'ActList': 'init', 'RewList': 'relative_measure', 'AlgList': 'DQN'}\n",
      "{'StatList': 'Bert', 'ActList': 'init', 'RewList': 'tan', 'AlgList': 'MCTS'}\n",
      "{'StatList': 'Bert', 'ActList': 'init', 'RewList': 'tan', 'AlgList': 'DQN'}\n",
      "{'StatList': 'Bert', 'ActList': 'init', 'RewList': 'func', 'AlgList': 'MCTS'}\n",
      "{'StatList': 'Bert', 'ActList': 'init', 'RewList': 'func', 'AlgList': 'DQN'}\n",
      "{'StatList': 'Bert', 'ActList': 'init', 'RewList': 'weight', 'AlgList': 'MCTS'}\n",
      "{'StatList': 'Bert', 'ActList': 'init', 'RewList': 'weight', 'AlgList': 'DQN'}\n",
      "---------------------------------------------------------------------------------------------------------\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import SuperSonic.policy_definition.policy_define as policy_define\n",
    "\n",
    "#Candidate state functions\n",
    "statefs=[\"Word2vec\", \"Doc2vec\", \"Bert\"]\n",
    "#Candidate reward functions\n",
    "rewards=[\"relative_measure\", \"tan\", \"func\", \"weight\"]\n",
    "#Candidate RL algorithms\n",
    "rl_algs=[\"MCTS\", \"PPO\", \"DQN\", \"QLearning\"]\n",
    "actions=[\"init\"]\n",
    "\n",
    "PolicyCandidate = policy_define.SuperOptimizer(\n",
    "        StateFunctions=statefs,\n",
    "        RewardFunctions=rewards,\n",
    "        RLAlgorithms=rl_algs,\n",
    "        ActionFunctions=actions,\n",
    "    ).PolicyDefined()\n",
    "\n",
    "PolicySearchSpace = PolicyCandidate[0]\n",
    "PolicySearchCount = PolicyCandidate[1]\n",
    "\n",
    "print(\"The search space consists of %d combinations of RL components.\" % PolicySearchCount)\n",
    "print(\"A few examples include:\")\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "for i in range(len(PolicySearchSpace)):\n",
    "    # We reduce the output to 1/5 of original to avoid logging too long.\n",
    "    if i%2==0:\n",
    "        print(PolicySearchSpace[i])\n",
    "print(\"---------------------------------------------------------------------------------------------------------\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2. Client RL search\n",
    "\n",
    "The policy_search() function invokes the SuperSonic meta-optimizer to search the client RL architecture. The user can also limit the number of trials spent on client RL searching. \n",
    "\n",
    "Once again, this example works for case study 4 - Superoptimization and use Hacker benchmarks. \n",
    "\n",
    "**approximate runtime ~ 30 minutes (please wait before moving to the next cell)**\n",
    "\n",
    "##### Note: \n",
    "You may encounter an error of failed tests. This is because we reduce the RL client search steps to make the search time manageable for the demo. Such failure did occur during our full-scale evaluation. \n",
    "\n",
    "#### *mode*: \n",
    "This defines module selecting of Supersonic. Possible values are:\n",
    "- *policy* for showing client RL Policy Search module; \n",
    "- *config* for showing parameters Tuning module; \n",
    "- *deploy* for showing the deployment of a tuned client RL module; \n",
    "\n",
    "#### *iterations*: \n",
    "This defines the number of trials for client RL search/ parameters tuning/ deployment; we set it to 100 for paramete tuning and 1000+ for deployment in the paper but use a smaller number for demo.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Start RL Client searching on superoptimization (please wait - approximate runtime = 30 minutes)...\n",
      "---------------------------------------------------------------------------------------------------------\n",
      "Policy search engine started...\n",
      "\n",
      "Start client RL training...\n",
      "exist_policy:{'StatList': 'Word2vec', 'ActList': 'init', 'RewList': 'weight', 'AlgList': 'QLearning'}\n",
      "Opened database successfully\n",
      "== Status ==\n",
      "Memory usage on this node: 31.2/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.23 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------+\n",
      "| Trial name               | status   | loc   |\n",
      "|--------------------------+----------+-------|\n",
      "| SAC_stoke_rl_b90af_00000 | RUNNING  |       |\n",
      "+--------------------------+----------+-------+\n",
      "\n",
      "\n",
      "\u001b[2m\u001b[36m(pid=101361)\u001b[0m 2022-03-03 16:34:03,688\tWARNING sac_tf_policy.py:51 -- When not using a state-preprocessor with SAC, `fcnet_hiddens` will be set to an empty list! Any hidden layer sizes are defined via `policy_model.hidden_layer_sizes` and `Q_model.hidden_layer_sizes`.\n",
      "\u001b[2m\u001b[36m(pid=101361)\u001b[0m 2022-03-03 16:34:06,495\tWARNING util.py:37 -- Install gputil for GPU system monitoring.\n",
      "== Status ==\n",
      "Memory usage on this node: 31.6/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.23 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_b90af_00000 | RUNNING  | 172.17.0.7:101361 |      1 |            5.938 |   50 |  20.9754 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.6/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.23 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_b90af_00000 | RUNNING  | 172.17.0.7:101361 |      3 |          14.1134 |  150 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.6/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.23 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_b90af_00000 | RUNNING  | 172.17.0.7:101361 |      5 |          22.1839 |  250 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.6/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.23 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_b90af_00000 | RUNNING  | 172.17.0.7:101361 |      7 |          29.8874 |  350 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.23 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_b90af_00000 | RUNNING  | 172.17.0.7:101361 |      9 |          37.8213 |  450 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.23 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_b90af_00000 | RUNNING  | 172.17.0.7:101361 |     11 |          46.0837 |  550 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.23 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_b90af_00000 | RUNNING  | 172.17.0.7:101361 |     13 |           54.102 |  650 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.23 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_b90af_00000 | RUNNING  | 172.17.0.7:101361 |     15 |           62.205 |  750 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.23 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_b90af_00000 | RUNNING  | 172.17.0.7:101361 |     17 |          70.6815 |  850 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.23 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_b90af_00000 | RUNNING  | 172.17.0.7:101361 |     19 |          79.5913 |  950 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 0/50 CPUs, 0/0 GPUs, 0.0/59.23 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 TERMINATED)\n",
      "+--------------------------+------------+-------+--------+------------------+------+----------+\n",
      "| Trial name               | status     | loc   |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+------------+-------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_b90af_00000 | TERMINATED |       |     20 |          83.6235 | 1000 |        1 |\n",
      "+--------------------------+------------+-------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "Iteration numbers : 1\n",
      "exist_policy:{'StatList': 'Word2vec', 'ActList': 'init', 'RewList': 'weight', 'AlgList': 'QLearning'}\n",
      "Opened database successfully\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------+\n",
      "| Trial name               | status   | loc   |\n",
      "|--------------------------+----------+-------|\n",
      "| SAC_stoke_rl_f34cd_00000 | RUNNING  |       |\n",
      "+--------------------------+----------+-------+\n",
      "\n",
      "\n",
      "\u001b[2m\u001b[36m(pid=102138)\u001b[0m 2022-03-03 16:35:41,910\tWARNING sac_tf_policy.py:51 -- When not using a state-preprocessor with SAC, `fcnet_hiddens` will be set to an empty list! Any hidden layer sizes are defined via `policy_model.hidden_layer_sizes` and `Q_model.hidden_layer_sizes`.\n",
      "\u001b[2m\u001b[36m(pid=102138)\u001b[0m 2022-03-03 16:35:44,829\tWARNING util.py:37 -- Install gputil for GPU system monitoring.\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_f34cd_00000 | RUNNING  | 172.17.0.7:102138 |      1 |          5.24408 |   50 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_f34cd_00000 | RUNNING  | 172.17.0.7:102138 |      3 |          13.2132 |  150 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.8/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_f34cd_00000 | RUNNING  | 172.17.0.7:102138 |      5 |          20.9638 |  250 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_f34cd_00000 | RUNNING  | 172.17.0.7:102138 |      7 |          28.5969 |  350 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.8/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_f34cd_00000 | RUNNING  | 172.17.0.7:102138 |      9 |          36.3953 |  450 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.3/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 0/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 ERROR)\n",
      "+--------------------------+----------+-------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc   |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_f34cd_00000 | ERROR    |       |     10 |          40.5196 |  500 |        1 |\n",
      "+--------------------------+----------+-------+--------+------------------+------+----------+\n",
      "Number of errored trials: 1\n",
      "+--------------------------+--------------+----------------------------------------------------------------------------------------------------------+\n",
      "| Trial name               |   # failures | error file                                                                                               |\n",
      "|--------------------------+--------------+----------------------------------------------------------------------------------------------------------|\n",
      "| SAC_stoke_rl_f34cd_00000 |            1 | /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC/SAC_stoke_rl_0_2022-03-03_16-35-35kz2pji4j/error.txt |\n",
      "+--------------------------+--------------+----------------------------------------------------------------------------------------------------------+\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.3/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 0/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 ERROR)\n",
      "+--------------------------+----------+-------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc   |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_f34cd_00000 | ERROR    |       |     10 |          40.5196 |  500 |        1 |\n",
      "+--------------------------+----------+-------+--------+------------------+------+----------+\n",
      "Number of errored trials: 1\n",
      "+--------------------------+--------------+----------------------------------------------------------------------------------------------------------+\n",
      "| Trial name               |   # failures | error file                                                                                               |\n",
      "|--------------------------+--------------+----------------------------------------------------------------------------------------------------------|\n",
      "| SAC_stoke_rl_f34cd_00000 |            1 | /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC/SAC_stoke_rl_0_2022-03-03_16-35-35kz2pji4j/error.txt |\n",
      "+--------------------------+--------------+----------------------------------------------------------------------------------------------------------+\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "('Trials did not complete', [SAC_stoke_rl_f34cd_00000])\n",
      "Iteration numbers : 2\n",
      "exist_policy:{'StatList': 'Doc2vec', 'ActList': 'init', 'RewList': 'weight', 'AlgList': 'QLearning'}\n",
      "Opened database successfully\n",
      "== Status ==\n",
      "Memory usage on this node: 31.4/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------+\n",
      "| Trial name               | status   | loc   |\n",
      "|--------------------------+----------+-------|\n",
      "| SAC_stoke_rl_1367c_00000 | RUNNING  |       |\n",
      "+--------------------------+----------+-------+\n",
      "\n",
      "\n",
      "\u001b[2m\u001b[36m(pid=102139)\u001b[0m 2022-03-03 16:36:35,514\tWARNING sac_tf_policy.py:51 -- When not using a state-preprocessor with SAC, `fcnet_hiddens` will be set to an empty list! Any hidden layer sizes are defined via `policy_model.hidden_layer_sizes` and `Q_model.hidden_layer_sizes`.\n",
      "\u001b[2m\u001b[36m(pid=102139)\u001b[0m 2022-03-03 16:36:38,295\tWARNING util.py:37 -- Install gputil for GPU system monitoring.\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_1367c_00000 | RUNNING  | 172.17.0.7:102139 |      1 |          5.34852 |   50 |    20.98 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_1367c_00000 | RUNNING  | 172.17.0.7:102139 |      3 |          13.1818 |  150 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_1367c_00000 | RUNNING  | 172.17.0.7:102139 |      5 |          21.0224 |  250 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_1367c_00000 | RUNNING  | 172.17.0.7:102139 |      7 |          28.4663 |  350 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_1367c_00000 | RUNNING  | 172.17.0.7:102139 |      9 |           36.242 |  450 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_1367c_00000 | RUNNING  | 172.17.0.7:102139 |     11 |          44.2566 |  550 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_1367c_00000 | RUNNING  | 172.17.0.7:102139 |     13 |          51.8158 |  650 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_1367c_00000 | RUNNING  | 172.17.0.7:102139 |     15 |          59.8976 |  750 | 0.993846 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_1367c_00000 | RUNNING  | 172.17.0.7:102139 |     17 |          68.6213 |  850 | 0.999423 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_1367c_00000 | RUNNING  | 172.17.0.7:102139 |     19 |          76.7059 |  950 |    0.995 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.7/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 0/50 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 TERMINATED)\n",
      "+--------------------------+------------+-------+--------+------------------+------+----------+\n",
      "| Trial name               | status     | loc   |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+------------+-------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_1367c_00000 | TERMINATED |       |     20 |          80.6315 | 1000 |        1 |\n",
      "+--------------------------+------------+-------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "Iteration numbers : 3\n",
      "exist_policy:{'StatList': 'Doc2vec', 'ActList': 'init', 'RewList': 'weight', 'AlgList': 'QLearning'}\n",
      "Opened database successfully\n",
      "== Status ==\n",
      "Memory usage on this node: 31.8/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------+\n",
      "| Trial name               | status   | loc   |\n",
      "|--------------------------+----------+-------|\n",
      "| SAC_stoke_rl_4c14c_00000 | RUNNING  |       |\n",
      "+--------------------------+----------+-------+\n",
      "\n",
      "\n",
      "\u001b[2m\u001b[36m(pid=103051)\u001b[0m 2022-03-03 16:38:10,921\tWARNING sac_tf_policy.py:51 -- When not using a state-preprocessor with SAC, `fcnet_hiddens` will be set to an empty list! Any hidden layer sizes are defined via `policy_model.hidden_layer_sizes` and `Q_model.hidden_layer_sizes`.\n",
      "\u001b[2m\u001b[36m(pid=103051)\u001b[0m 2022-03-03 16:38:13,954\tWARNING util.py:37 -- Install gputil for GPU system monitoring.\n",
      "== Status ==\n",
      "Memory usage on this node: 31.8/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_4c14c_00000 | RUNNING  | 172.17.0.7:103051 |      1 |          5.05766 |   50 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.8/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_4c14c_00000 | RUNNING  | 172.17.0.7:103051 |      3 |          12.7672 |  150 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.8/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_4c14c_00000 | RUNNING  | 172.17.0.7:103051 |      5 |          20.5962 |  250 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.8/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_4c14c_00000 | RUNNING  | 172.17.0.7:103051 |      7 |          29.2007 |  350 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.8/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_4c14c_00000 | RUNNING  | 172.17.0.7:103051 |      9 |          37.4238 |  450 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.8/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_4c14c_00000 | RUNNING  | 172.17.0.7:103051 |     11 |          45.9137 |  550 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.8/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_4c14c_00000 | RUNNING  | 172.17.0.7:103051 |     13 |          53.9474 |  650 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.8/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_4c14c_00000 | RUNNING  | 172.17.0.7:103051 |     15 |          62.1972 |  750 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.8/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_4c14c_00000 | RUNNING  | 172.17.0.7:103051 |     16 |          67.1019 |  800 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.8/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_4c14c_00000 | RUNNING  | 172.17.0.7:103051 |     18 |          75.2911 |  900 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.8/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_4c14c_00000 | RUNNING  | 172.17.0.7:103051 |     20 |          83.2248 | 1000 | 0.996154 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.8/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 0/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 TERMINATED)\n",
      "+--------------------------+------------+-------+--------+------------------+------+----------+\n",
      "| Trial name               | status     | loc   |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+------------+-------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_4c14c_00000 | TERMINATED |       |     20 |          83.2248 | 1000 | 0.996154 |\n",
      "+--------------------------+------------+-------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "Iteration numbers : 4\n",
      "exist_policy:{'StatList': 'Word2vec', 'ActList': 'init', 'RewList': 'weight', 'AlgList': 'QLearning'}\n",
      "Opened database successfully\n",
      "== Status ==\n",
      "Memory usage on this node: 31.8/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------+\n",
      "| Trial name               | status   | loc   |\n",
      "|--------------------------+----------+-------|\n",
      "| SAC_stoke_rl_869c8_00000 | RUNNING  |       |\n",
      "+--------------------------+----------+-------+\n",
      "\n",
      "\n",
      "\u001b[2m\u001b[36m(pid=103864)\u001b[0m 2022-03-03 16:39:49,042\tWARNING sac_tf_policy.py:51 -- When not using a state-preprocessor with SAC, `fcnet_hiddens` will be set to an empty list! Any hidden layer sizes are defined via `policy_model.hidden_layer_sizes` and `Q_model.hidden_layer_sizes`.\n",
      "\u001b[2m\u001b[36m(pid=103864)\u001b[0m 2022-03-03 16:39:52,198\tWARNING util.py:37 -- Install gputil for GPU system monitoring.\n",
      "== Status ==\n",
      "Memory usage on this node: 31.9/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_869c8_00000 | RUNNING  | 172.17.0.7:103864 |      1 |          4.87897 |   50 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.9/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_869c8_00000 | RUNNING  | 172.17.0.7:103864 |      3 |          12.9904 |  150 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.9/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_869c8_00000 | RUNNING  | 172.17.0.7:103864 |      5 |          21.0332 |  250 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.9/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_869c8_00000 | RUNNING  | 172.17.0.7:103864 |      7 |          29.4175 |  350 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.9/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_869c8_00000 | RUNNING  | 172.17.0.7:103864 |      9 |           37.462 |  450 |        1 |\n",
      "+--------------------------+----------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.4/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 0/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 ERROR)\n",
      "+--------------------------+----------+-------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc   |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_869c8_00000 | ERROR    |       |     10 |          41.9175 |  500 |        1 |\n",
      "+--------------------------+----------+-------+--------+------------------+------+----------+\n",
      "Number of errored trials: 1\n",
      "+--------------------------+--------------+----------------------------------------------------------------------------------------------------------+\n",
      "| Trial name               |   # failures | error file                                                                                               |\n",
      "|--------------------------+--------------+----------------------------------------------------------------------------------------------------------|\n",
      "| SAC_stoke_rl_869c8_00000 |            1 | /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC/SAC_stoke_rl_0_2022-03-03_16-39-42rt1zelbf/error.txt |\n",
      "+--------------------------+--------------+----------------------------------------------------------------------------------------------------------+\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.4/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 0/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 ERROR)\n",
      "+--------------------------+----------+-------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc   |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_869c8_00000 | ERROR    |       |     10 |          41.9175 |  500 |        1 |\n",
      "+--------------------------+----------+-------+--------+------------------+------+----------+\n",
      "Number of errored trials: 1\n",
      "+--------------------------+--------------+----------------------------------------------------------------------------------------------------------+\n",
      "| Trial name               |   # failures | error file                                                                                               |\n",
      "|--------------------------+--------------+----------------------------------------------------------------------------------------------------------|\n",
      "| SAC_stoke_rl_869c8_00000 |            1 | /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC/SAC_stoke_rl_0_2022-03-03_16-39-42rt1zelbf/error.txt |\n",
      "+--------------------------+--------------+----------------------------------------------------------------------------------------------------------+\n",
      "\n",
      "('Trials did not complete', [SAC_stoke_rl_869c8_00000])\n",
      "Iteration numbers : 5\n",
      "exist_policy:{'StatList': 'Word2vec', 'ActList': 'init', 'RewList': 'weight', 'AlgList': 'QLearning'}\n",
      "Opened database successfully\n",
      "== Status ==\n",
      "Memory usage on this node: 31.4/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------+\n",
      "| Trial name               | status   | loc   |\n",
      "|--------------------------+----------+-------|\n",
      "| SAC_stoke_rl_a6595_00000 | RUNNING  |       |\n",
      "+--------------------------+----------+-------+\n",
      "\n",
      "\n",
      "\u001b[2m\u001b[36m(pid=103884)\u001b[0m 2022-03-03 16:40:42,254\tWARNING sac_tf_policy.py:51 -- When not using a state-preprocessor with SAC, `fcnet_hiddens` will be set to an empty list! Any hidden layer sizes are defined via `policy_model.hidden_layer_sizes` and `Q_model.hidden_layer_sizes`.\n",
      "\u001b[2m\u001b[36m(pid=103884)\u001b[0m 2022-03-03 16:40:45,214\tWARNING util.py:37 -- Install gputil for GPU system monitoring.\n",
      "== Status ==\n",
      "Memory usage on this node: 31.4/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 0/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 ERROR)\n",
      "+--------------------------+----------+-------+\n",
      "| Trial name               | status   | loc   |\n",
      "|--------------------------+----------+-------|\n",
      "| SAC_stoke_rl_a6595_00000 | ERROR    |       |\n",
      "+--------------------------+----------+-------+\n",
      "Number of errored trials: 1\n",
      "+--------------------------+--------------+----------------------------------------------------------------------------------------------------------+\n",
      "| Trial name               |   # failures | error file                                                                                               |\n",
      "|--------------------------+--------------+----------------------------------------------------------------------------------------------------------|\n",
      "| SAC_stoke_rl_a6595_00000 |            1 | /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC/SAC_stoke_rl_0_2022-03-03_16-40-35zqr_8nen/error.txt |\n",
      "+--------------------------+--------------+----------------------------------------------------------------------------------------------------------+\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.4/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 0/50 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 ERROR)\n",
      "+--------------------------+----------+-------+\n",
      "| Trial name               | status   | loc   |\n",
      "|--------------------------+----------+-------|\n",
      "| SAC_stoke_rl_a6595_00000 | ERROR    |       |\n",
      "+--------------------------+----------+-------+\n",
      "Number of errored trials: 1\n",
      "+--------------------------+--------------+----------------------------------------------------------------------------------------------------------+\n",
      "| Trial name               |   # failures | error file                                                                                               |\n",
      "|--------------------------+--------------+----------------------------------------------------------------------------------------------------------|\n",
      "| SAC_stoke_rl_a6595_00000 |            1 | /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC/SAC_stoke_rl_0_2022-03-03_16-40-35zqr_8nen/error.txt |\n",
      "+--------------------------+--------------+----------------------------------------------------------------------------------------------------------+\n",
      "\n",
      "('Trials did not complete', [SAC_stoke_rl_a6595_00000])\n",
      "Iteration numbers : 6\n",
      "\n",
      "Total gradients at learning step 1:\n",
      " layer                 |        mean |        max\n",
      "-----------------------+-------------+------------\n",
      " feat_convs.0.0.weight | 0           | 0\n",
      " feat_convs.1.0.weight | 0.000910623 | 0.00894489\n",
      " feat_convs.2.0.weight | 0.00125335  | 0.0184943\n",
      " resnet1.0.1.weight    | 2.98372e-05 | 0.00112768\n",
      " resnet1.0.3.weight    | 0.000188095 | 0.00185946\n",
      " resnet1.1.1.weight    | 9.48644e-05 | 0.00298027\n",
      " resnet1.1.3.weight    | 0.000150409 | 0.00280567\n",
      " resnet1.2.1.weight    | 0.000123491 | 0.00250588\n",
      " resnet1.2.3.weight    | 0.000173113 | 0.00594817\n",
      " resnet2.0.1.weight    | 5.2201e-05  | 0.00108342\n",
      " resnet2.0.3.weight    | 0.000305338 | 0.00216515\n",
      " resnet2.1.1.weight    | 8.82574e-05 | 0.00234537\n",
      " resnet2.1.3.weight    | 0.000222186 | 0.0029794\n",
      " resnet2.2.1.weight    | 0.000139129 | 0.00427435\n",
      " resnet2.2.3.weight    | 0.000260501 | 0.00726163\n",
      " fc.weight             | 0.000534945 | 0.0198083\n",
      " policy.weight         | 0.00167661  | 0.113774\n",
      " baseline.weight       | 0.00757543  | 0.0604782 \n",
      "\n",
      "model has been saved in  ./logs/supersonic/MultiTaskPopArt/model.tar\n",
      "\n",
      "\n",
      "----------------------------------\n",
      "\n",
      "policy search process finished. The best policy strategy is : \n",
      " {'StatList': 'Word2vec', 'ActList': 'init', 'RewList': 'weight', 'AlgList': 'DQN'}\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------------------------------------------------------------------------------------------------\n",
      "Client RL search finished.\n"
     ]
    }
   ],
   "source": [
    "from demo import demo_run\n",
    "\n",
    "#Wait until you see the finish message before moving to the next cell.\n",
    "demo_run(mode=\"policy\",iterations=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3. Client RL Parameter Tuning\n",
    "\n",
    "After choosing an RL architecture, Supersonic will fine-tune a set of model-specific hyper-parameters of the selected\n",
    "client RL.\n",
    "\n",
    "This demo shows how to fine-tune the chosen client RL on the hacker benchmark dataset for superoptimization.\n",
    "\n",
    "*approximate runtime ~ 20 minutes*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Start client RL parameter tuning (please wait - approximate runtime = 20 minutes)...\n",
      "---------------------------------------------------------------------------------------------------------\n",
      "config search engine started...\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.0/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: None\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/59.23 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "| Trial name               | status   | loc   |   learning_starts |\n",
      "|--------------------------+----------+-------+-------------------|\n",
      "| SAC_stoke_rl_4bef6_00000 | RUNNING  |       |         0.0676315 |\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "\n",
      "\n",
      "\u001b[2m\u001b[36m(pid=104839)\u001b[0m 2022-03-03 16:45:20,258\tWARNING sac_tf_policy.py:51 -- When not using a state-preprocessor with SAC, `fcnet_hiddens` will be set to an empty list! Any hidden layer sizes are defined via `policy_model.hidden_layer_sizes` and `Q_model.hidden_layer_sizes`.\n",
      "\u001b[2m\u001b[36m(pid=104839)\u001b[0m 2022-03-03 16:45:22,906\tWARNING util.py:37 -- Install gputil for GPU system monitoring.\n",
      "== Status ==\n",
      "Memory usage on this node: 33.3/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/59.23 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_4bef6_00000 | RUNNING  | 172.17.0.7:104839 |         0.0676315 |      1 |          4.81404 |   50 |        1 |\n",
      "+--------------------------+----------+-------------------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.3/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/59.23 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_4bef6_00000 | RUNNING  | 172.17.0.7:104839 |         0.0676315 |      3 |          12.7224 |  150 |        1 |\n",
      "+--------------------------+----------+-------------------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.3/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 0/104 CPUs, 0/0 GPUs, 0.0/59.23 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 TERMINATED)\n",
      "+--------------------------+------------+-------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status     | loc   |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+------------+-------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_4bef6_00000 | TERMINATED |       |         0.0676315 |      3 |          12.7224 |  150 |        1 |\n",
      "+--------------------------+------------+-------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.4/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: None\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.98 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "| Trial name               | status   | loc   |   learning_starts |\n",
      "|--------------------------+----------+-------+-------------------|\n",
      "| SAC_stoke_rl_5b94c_00000 | RUNNING  |       |          0.580241 |\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "\n",
      "\n",
      "\u001b[2m\u001b[36m(pid=106310)\u001b[0m 2022-03-03 16:45:46,459\tWARNING sac_tf_policy.py:51 -- When not using a state-preprocessor with SAC, `fcnet_hiddens` will be set to an empty list! Any hidden layer sizes are defined via `policy_model.hidden_layer_sizes` and `Q_model.hidden_layer_sizes`.\n",
      "\u001b[2m\u001b[36m(pid=106310)\u001b[0m 2022-03-03 16:45:49,090\tWARNING util.py:37 -- Install gputil for GPU system monitoring.\n",
      "== Status ==\n",
      "Memory usage on this node: 33.4/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.98 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_5b94c_00000 | RUNNING  | 172.17.0.7:106310 |          0.580241 |      1 |          4.89308 |   50 |        1 |\n",
      "+--------------------------+----------+-------------------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.4/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.98 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------------------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc               |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------------------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_5b94c_00000 | RUNNING  | 172.17.0.7:106310 |          0.580241 |      3 |           12.632 |  150 |        1 |\n",
      "+--------------------------+----------+-------------------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.4/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 0/104 CPUs, 0/0 GPUs, 0.0/58.98 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 TERMINATED)\n",
      "+--------------------------+------------+-------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status     | loc   |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+------------+-------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_5b94c_00000 | TERMINATED |       |          0.580241 |      3 |           12.632 |  150 |        1 |\n",
      "+--------------------------+------------+-------+-------------------+--------+------------------+------+----------+\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.5/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: None\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.98 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "| Trial name               | status   | loc   |   learning_starts |\n",
      "|--------------------------+----------+-------+-------------------|\n",
      "| SAC_stoke_rl_6b42a_00000 | RUNNING  |       |          0.593558 |\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "\n",
      "\n",
      "\u001b[2m\u001b[36m(pid=1611)\u001b[0m 2022-03-03 16:46:13,412\tWARNING sac_tf_policy.py:51 -- When not using a state-preprocessor with SAC, `fcnet_hiddens` will be set to an empty list! Any hidden layer sizes are defined via `policy_model.hidden_layer_sizes` and `Q_model.hidden_layer_sizes`.\n",
      "\u001b[2m\u001b[36m(pid=1611)\u001b[0m 2022-03-03 16:46:16,216\tWARNING util.py:37 -- Install gputil for GPU system monitoring.\n",
      "== Status ==\n",
      "Memory usage on this node: 33.5/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.98 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc             |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_6b42a_00000 | RUNNING  | 172.17.0.7:1611 |          0.593558 |      1 |          4.69286 |   50 |        1 |\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.5/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.98 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc             |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_6b42a_00000 | RUNNING  | 172.17.0.7:1611 |          0.593558 |      3 |          12.5177 |  150 |        1 |\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.5/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 0/104 CPUs, 0/0 GPUs, 0.0/58.98 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 TERMINATED)\n",
      "+--------------------------+------------+-------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status     | loc   |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+------------+-------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_6b42a_00000 | TERMINATED |       |          0.593558 |      3 |          12.5177 |  150 |        1 |\n",
      "+--------------------------+------------+-------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.6/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: None\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "| Trial name               | status   | loc   |   learning_starts |\n",
      "|--------------------------+----------+-------+-------------------|\n",
      "| SAC_stoke_rl_7b5dd_00000 | RUNNING  |       |          0.641259 |\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "\n",
      "\n",
      "\u001b[2m\u001b[36m(pid=3114)\u001b[0m 2022-03-03 16:46:40,441\tWARNING sac_tf_policy.py:51 -- When not using a state-preprocessor with SAC, `fcnet_hiddens` will be set to an empty list! Any hidden layer sizes are defined via `policy_model.hidden_layer_sizes` and `Q_model.hidden_layer_sizes`.\n",
      "\u001b[2m\u001b[36m(pid=3114)\u001b[0m 2022-03-03 16:46:43,231\tWARNING util.py:37 -- Install gputil for GPU system monitoring.\n",
      "== Status ==\n",
      "Memory usage on this node: 33.6/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc             |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_7b5dd_00000 | RUNNING  | 172.17.0.7:3114 |          0.641259 |      1 |           5.0753 |   50 |        1 |\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.6/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc             |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_7b5dd_00000 | RUNNING  | 172.17.0.7:3114 |          0.641259 |      3 |          12.7683 |  150 |        1 |\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.6/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 0/104 CPUs, 0/0 GPUs, 0.0/58.89 GiB heap, 0.0/20.17 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 TERMINATED)\n",
      "+--------------------------+------------+-------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status     | loc   |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+------------+-------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_7b5dd_00000 | TERMINATED |       |          0.641259 |      3 |          12.7683 |  150 |        1 |\n",
      "+--------------------------+------------+-------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.7/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: None\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.84 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "| Trial name               | status   | loc   |   learning_starts |\n",
      "|--------------------------+----------+-------+-------------------|\n",
      "| SAC_stoke_rl_8b9be_00000 | RUNNING  |       |          0.378585 |\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\u001b[2m\u001b[36m(pid=4635)\u001b[0m 2022-03-03 16:47:07,823\tWARNING sac_tf_policy.py:51 -- When not using a state-preprocessor with SAC, `fcnet_hiddens` will be set to an empty list! Any hidden layer sizes are defined via `policy_model.hidden_layer_sizes` and `Q_model.hidden_layer_sizes`.\n",
      "\u001b[2m\u001b[36m(pid=4635)\u001b[0m 2022-03-03 16:47:10,811\tWARNING util.py:37 -- Install gputil for GPU system monitoring.\n",
      "== Status ==\n",
      "Memory usage on this node: 33.7/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.84 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc             |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_8b9be_00000 | RUNNING  | 172.17.0.7:4635 |          0.378585 |      1 |          5.27222 |   50 |        1 |\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.7/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.84 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc             |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_8b9be_00000 | RUNNING  | 172.17.0.7:4635 |          0.378585 |      3 |          13.5995 |  150 | 0.985385 |\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.7/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 0/104 CPUs, 0/0 GPUs, 0.0/58.84 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 TERMINATED)\n",
      "+--------------------------+------------+-------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status     | loc   |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+------------+-------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_8b9be_00000 | TERMINATED |       |          0.378585 |      3 |          13.5995 |  150 | 0.985385 |\n",
      "+--------------------------+------------+-------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.8/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: None\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "| Trial name               | status   | loc   |   learning_starts |\n",
      "|--------------------------+----------+-------+-------------------|\n",
      "| SAC_stoke_rl_9c758_00000 | RUNNING  |       |             0.845 |\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "\n",
      "\n",
      "\u001b[2m\u001b[36m(pid=6120)\u001b[0m 2022-03-03 16:47:35,868\tWARNING sac_tf_policy.py:51 -- When not using a state-preprocessor with SAC, `fcnet_hiddens` will be set to an empty list! Any hidden layer sizes are defined via `policy_model.hidden_layer_sizes` and `Q_model.hidden_layer_sizes`.\n",
      "\u001b[2m\u001b[36m(pid=6120)\u001b[0m 2022-03-03 16:47:38,786\tWARNING util.py:37 -- Install gputil for GPU system monitoring.\n",
      "== Status ==\n",
      "Memory usage on this node: 33.3/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: None\n",
      "Resources requested: 0/104 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 ERROR)\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "| Trial name               | status   | loc   |   learning_starts |\n",
      "|--------------------------+----------+-------+-------------------|\n",
      "| SAC_stoke_rl_9c758_00000 | ERROR    |       |             0.845 |\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "Number of errored trials: 1\n",
      "+--------------------------+--------------+--------------------------------------------------------------------------------------------------------------------------------+\n",
      "| Trial name               |   # failures | error file                                                                                                                     |\n",
      "|--------------------------+--------------+--------------------------------------------------------------------------------------------------------------------------------|\n",
      "| SAC_stoke_rl_9c758_00000 |            1 | /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC/SAC_stoke_rl_0_learning_starts=0.845_2022-03-03_16-47-283pk3jbcq/error.txt |\n",
      "+--------------------------+--------------+--------------------------------------------------------------------------------------------------------------------------------+\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.3/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: None\n",
      "Resources requested: 0/104 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 ERROR)\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "| Trial name               | status   | loc   |   learning_starts |\n",
      "|--------------------------+----------+-------+-------------------|\n",
      "| SAC_stoke_rl_9c758_00000 | ERROR    |       |             0.845 |\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "Number of errored trials: 1\n",
      "+--------------------------+--------------+--------------------------------------------------------------------------------------------------------------------------------+\n",
      "| Trial name               |   # failures | error file                                                                                                                     |\n",
      "|--------------------------+--------------+--------------------------------------------------------------------------------------------------------------------------------|\n",
      "| SAC_stoke_rl_9c758_00000 |            1 | /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC/SAC_stoke_rl_0_learning_starts=0.845_2022-03-03_16-47-283pk3jbcq/error.txt |\n",
      "+--------------------------+--------------+--------------------------------------------------------------------------------------------------------------------------------+\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.3/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: None\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "| Trial name               | status   | loc   |   learning_starts |\n",
      "|--------------------------+----------+-------+-------------------|\n",
      "| SAC_stoke_rl_a4f91_00000 | RUNNING  |       |         0.0209298 |\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\u001b[2m\u001b[36m(pid=6117)\u001b[0m 2022-03-03 16:47:50,002\tWARNING sac_tf_policy.py:51 -- When not using a state-preprocessor with SAC, `fcnet_hiddens` will be set to an empty list! Any hidden layer sizes are defined via `policy_model.hidden_layer_sizes` and `Q_model.hidden_layer_sizes`.\n",
      "\u001b[2m\u001b[36m(pid=6117)\u001b[0m 2022-03-03 16:47:53,035\tWARNING util.py:37 -- Install gputil for GPU system monitoring.\n",
      "== Status ==\n",
      "Memory usage on this node: 33.7/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc             |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_a4f91_00000 | RUNNING  | 172.17.0.7:6117 |         0.0209298 |      1 |          5.19478 |   50 |        1 |\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.7/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc             |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_a4f91_00000 | RUNNING  | 172.17.0.7:6117 |         0.0209298 |      3 |          13.1968 |  150 |        1 |\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.7/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 0/104 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 TERMINATED)\n",
      "+--------------------------+------------+-------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status     | loc   |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+------------+-------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_a4f91_00000 | TERMINATED |       |         0.0209298 |      3 |          13.1968 |  150 |        1 |\n",
      "+--------------------------+------------+-------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.8/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: None\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.74 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "| Trial name               | status   | loc   |   learning_starts |\n",
      "|--------------------------+----------+-------+-------------------|\n",
      "| SAC_stoke_rl_b5727_00000 | RUNNING  |       |          0.937877 |\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "\n",
      "\n",
      "\u001b[2m\u001b[36m(pid=7630)\u001b[0m 2022-03-03 16:48:17,902\tWARNING sac_tf_policy.py:51 -- When not using a state-preprocessor with SAC, `fcnet_hiddens` will be set to an empty list! Any hidden layer sizes are defined via `policy_model.hidden_layer_sizes` and `Q_model.hidden_layer_sizes`.\n",
      "\u001b[2m\u001b[36m(pid=7630)\u001b[0m 2022-03-03 16:48:21,022\tWARNING util.py:37 -- Install gputil for GPU system monitoring.\n",
      "== Status ==\n",
      "Memory usage on this node: 33.7/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.74 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc             |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_b5727_00000 | RUNNING  | 172.17.0.7:7630 |          0.937877 |      1 |          4.99191 |   50 |        1 |\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.7/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.74 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc             |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_b5727_00000 | RUNNING  | 172.17.0.7:7630 |          0.937877 |      3 |          12.9108 |  150 |        1 |\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.7/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 0/104 CPUs, 0/0 GPUs, 0.0/58.74 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 TERMINATED)\n",
      "+--------------------------+------------+-------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status     | loc   |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+------------+-------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_b5727_00000 | TERMINATED |       |          0.937877 |      3 |          12.9108 |  150 |        1 |\n",
      "+--------------------------+------------+-------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.9/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: None\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "| Trial name               | status   | loc   |   learning_starts |\n",
      "|--------------------------+----------+-------+-------------------|\n",
      "| SAC_stoke_rl_c5e7d_00000 | RUNNING  |       |          0.822631 |\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[2m\u001b[36m(pid=9131)\u001b[0m 2022-03-03 16:48:45,394\tWARNING sac_tf_policy.py:51 -- When not using a state-preprocessor with SAC, `fcnet_hiddens` will be set to an empty list! Any hidden layer sizes are defined via `policy_model.hidden_layer_sizes` and `Q_model.hidden_layer_sizes`.\n",
      "\u001b[2m\u001b[36m(pid=9131)\u001b[0m 2022-03-03 16:48:48,306\tWARNING util.py:37 -- Install gputil for GPU system monitoring.\n",
      "== Status ==\n",
      "Memory usage on this node: 33.7/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc             |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_c5e7d_00000 | RUNNING  | 172.17.0.7:9131 |          0.822631 |      1 |          5.31607 |   50 |        1 |\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.7/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc             |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_c5e7d_00000 | RUNNING  | 172.17.0.7:9131 |          0.822631 |      3 |          13.5185 |  150 |        1 |\n",
      "+--------------------------+----------+-----------------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.7/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 1.0\n",
      "Resources requested: 0/104 CPUs, 0/0 GPUs, 0.0/58.79 GiB heap, 0.0/20.12 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 TERMINATED)\n",
      "+--------------------------+------------+-------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status     | loc   |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+------------+-------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_c5e7d_00000 | TERMINATED |       |          0.822631 |      3 |          13.5185 |  150 |        1 |\n",
      "+--------------------------+------------+-------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.4/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: None\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/59.08 GiB heap, 0.0/20.21 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "| Trial name               | status   | loc   |   learning_starts |\n",
      "|--------------------------+----------+-------+-------------------|\n",
      "| SAC_stoke_rl_d6cb8_00000 | RUNNING  |       |           0.51402 |\n",
      "+--------------------------+----------+-------+-------------------+\n",
      "\n",
      "\n",
      "\u001b[2m\u001b[36m(pid=10610)\u001b[0m 2022-03-03 16:49:13,736\tWARNING sac_tf_policy.py:51 -- When not using a state-preprocessor with SAC, `fcnet_hiddens` will be set to an empty list! Any hidden layer sizes are defined via `policy_model.hidden_layer_sizes` and `Q_model.hidden_layer_sizes`.\n",
      "\u001b[2m\u001b[36m(pid=10610)\u001b[0m 2022-03-03 16:49:16,726\tWARNING util.py:37 -- Install gputil for GPU system monitoring.\n",
      "== Status ==\n",
      "Memory usage on this node: 33.7/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 20.98\n",
      "Resources requested: 1/104 CPUs, 0/0 GPUs, 0.0/59.08 GiB heap, 0.0/20.21 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+------------------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc              |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+------------------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_d6cb8_00000 | RUNNING  | 172.17.0.7:10610 |           0.51402 |      1 |          5.59271 |   50 |    20.98 |\n",
      "+--------------------------+----------+------------------+-------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.3/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 20.98\n",
      "Resources requested: 0/104 CPUs, 0/0 GPUs, 0.0/59.08 GiB heap, 0.0/20.21 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 ERROR)\n",
      "+--------------------------+----------+-------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc   |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_d6cb8_00000 | ERROR    |       |           0.51402 |      2 |          9.78831 |  100 |    10.99 |\n",
      "+--------------------------+----------+-------+-------------------+--------+------------------+------+----------+\n",
      "Number of errored trials: 1\n",
      "+--------------------------+--------------+----------------------------------------------------------------------------------------------------------------------------------+\n",
      "| Trial name               |   # failures | error file                                                                                                                       |\n",
      "|--------------------------+--------------+----------------------------------------------------------------------------------------------------------------------------------|\n",
      "| SAC_stoke_rl_d6cb8_00000 |            1 | /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC/SAC_stoke_rl_0_learning_starts=0.51402_2022-03-03_16-49-06bdx4p8rg/error.txt |\n",
      "+--------------------------+--------------+----------------------------------------------------------------------------------------------------------------------------------+\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 33.3/125.6 GiB\n",
      "Using AsyncHyperBand: num_stopped=0\n",
      "Bracket: Iter 256.000: None | Iter 64.000: None | Iter 16.000: None | Iter 4.000: None | Iter 1.000: 20.98\n",
      "Resources requested: 0/104 CPUs, 0/0 GPUs, 0.0/59.08 GiB heap, 0.0/20.21 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 ERROR)\n",
      "+--------------------------+----------+-------+-------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc   |   learning_starts |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+-------+-------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_d6cb8_00000 | ERROR    |       |           0.51402 |      2 |          9.78831 |  100 |    10.99 |\n",
      "+--------------------------+----------+-------+-------------------+--------+------------------+------+----------+\n",
      "Number of errored trials: 1\n",
      "+--------------------------+--------------+----------------------------------------------------------------------------------------------------------------------------------+\n",
      "| Trial name               |   # failures | error file                                                                                                                       |\n",
      "|--------------------------+--------------+----------------------------------------------------------------------------------------------------------------------------------|\n",
      "| SAC_stoke_rl_d6cb8_00000 |            1 | /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC/SAC_stoke_rl_0_learning_starts=0.51402_2022-03-03_16-49-06bdx4p8rg/error.txt |\n",
      "+--------------------------+--------------+----------------------------------------------------------------------------------------------------------------------------------+\n",
      "\n",
      "\n",
      "\n",
      "-------------------------------\n",
      "\n",
      "config search process finished. The best config is : \n",
      " {'env': <class 'SuperSonic.utils.environments.Stoke_env.stoke_rl'>, 'env_config': {'sql_path': '/home/sys/SUPERSONIC/SuperSonic/SQL/supersonic.db', 'target': 'localhost:50055', 'log_path': '/home/sys/SUPERSONIC/tasks/Stoke/result', 'obs_file': '/home/sys/SUPERSONIC/tasks/stoke/example/record/finish.txt', 'stop': {'time_total_s': 50}, 'stoke_path': '/home/sys/SUPERSONIC/tasks/stoke/example/p04', 'state_function': 'Doc2vec', 'action_function': 'Doc2vec', 'reward_function': 'usr_define', 'algorithm': 'QLearning', 'experiment': 'stoke', 'local_dir': '/home/sys/SUPERSONIC/SuperSonic/logs/model_save', 'training_iterations': 20}, 'num_workers': 0, 'timesteps_per_iteration': 1, 'learning_starts': 0.06763151336433991, 'normalize_actions': False, 'train_batch_size': 64, 'rollout_fragment_length': 50}\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------------------------------------------------------------------------------------------------\n",
      "Finish client RL parameter tuning.\n"
     ]
    }
   ],
   "source": [
    "from demo import demo_run\n",
    "\n",
    "#Wait until you see the finish message before moving to the next cell.\n",
    "demo_run(mode=\"config\", iterations=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4. Client RL Deployment\n",
    "\n",
    "The tuned client RL and its parameters are saved in the last step, which can be shipped with a compiler to optimise unseen programs at deployment time. \n",
    "\n",
    "This demo shows how to apply the saved client RL to optimize a test program for superoptimizaiton. \n",
    "\n",
    "*approximate runtime ~ 15 minutes*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Environment rebuilding for case study Superoptimization (~3 mins)\n",
      "Environment rebuilding successful!\n",
      "---------------------------------------------------------------------------------------------------------\n",
      "Start client RL demo (please wait - approximate runtime = 15 minutes)...\n",
      "---------------------------------------------------------------------------------------------------------\n",
      "deploying the best config and policy to RL client and optimize the program...\n",
      "\n",
      "Opened database successfully\n",
      "== Status ==\n",
      "Memory usage on this node: 31.2/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.38 GiB heap, 0.0/20.31 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------+\n",
      "| Trial name               | status   | loc   |\n",
      "|--------------------------+----------+-------|\n",
      "| SAC_stoke_rl_75b41_00000 | RUNNING  |       |\n",
      "+--------------------------+----------+-------+\n",
      "\n",
      "\n",
      "\u001b[2m\u001b[36m(pid=12093)\u001b[0m 2022-03-03 16:53:39,170\tWARNING sac_tf_policy.py:51 -- When not using a state-preprocessor with SAC, `fcnet_hiddens` will be set to an empty list! Any hidden layer sizes are defined via `policy_model.hidden_layer_sizes` and `Q_model.hidden_layer_sizes`.\n",
      "\u001b[2m\u001b[36m(pid=12093)\u001b[0m 2022-03-03 16:53:41,659\tWARNING util.py:37 -- Install gputil for GPU system monitoring.\n",
      "== Status ==\n",
      "Memory usage on this node: 31.6/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.38 GiB heap, 0.0/20.31 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc              |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_75b41_00000 | RUNNING  | 172.17.0.7:12093 |      1 |          5.61191 |   50 |    20.98 |\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.6/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.38 GiB heap, 0.0/20.31 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc              |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_75b41_00000 | RUNNING  | 172.17.0.7:12093 |      3 |          13.5956 |  150 |        1 |\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.6/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.38 GiB heap, 0.0/20.31 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc              |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_75b41_00000 | RUNNING  | 172.17.0.7:12093 |      5 |          21.8447 |  250 |        1 |\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.6/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.38 GiB heap, 0.0/20.31 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc              |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_75b41_00000 | RUNNING  | 172.17.0.7:12093 |      7 |          29.8658 |  350 |        1 |\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.6/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.38 GiB heap, 0.0/20.31 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc              |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_75b41_00000 | RUNNING  | 172.17.0.7:12093 |      9 |          37.8851 |  450 |        1 |\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.6/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 0/50 CPUs, 0/0 GPUs, 0.0/59.38 GiB heap, 0.0/20.31 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 TERMINATED)\n",
      "+--------------------------+------------+-------+--------+------------------+------+----------+\n",
      "| Trial name               | status     | loc   |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+------------+-------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_75b41_00000 | TERMINATED |       |     10 |          42.0703 |  500 |        1 |\n",
      "+--------------------------+------------+-------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "-------------------------------\n",
      "\n",
      "optimization finished, ready to use our script to evaluate performance\n",
      "-------------------------------\n",
      "\n",
      "start executing, usually takes 3 mins\n",
      "\n",
      "(stoke) start execute and calculate o3 baseline\n",
      "Run Time (Geomean) : 2.727 s\n",
      "(stoke) start execute and calculate RL result\n",
      "Run Time (Geomean) : 2.650000 s\n",
      "\n",
      "\n",
      "speedup:1.029057\n",
      "---------------------------------------------------------------------------------------------------------\n",
      "Demo finished\n"
     ]
    }
   ],
   "source": [
    "from demo import demo_run\n",
    "\n",
    "#Wait until you see the finish message before moving to the next cell.\n",
    "demo_run(mode=\"deploy\", iterations=10,benchmark='p20')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Demo 2: Experimental Evaluation\n",
    "\n",
    "Here, we provide a small-sized evaluation to showcase the working mechanism of the Supersonic-chosen RL on four case studies. A full-scale evaluation, which takes more than a week to run, is provided through the Docker image (with detailed instructions on our project Github). \n",
    "\n",
    "### Case Study 1: Optimizing Image Pipelines (Section 5.1)\n",
    "\n",
    "This task aims to improve the optimization heuristic of the Halide compiler framework. Halide is a domain-specific language and compiler for image processing pipelines (or graphs) with multiple computation stages. A Halide program separates the expression of the computation kernels and the application processing pipeline from the pipeline’s schedule. Here, the schedule defines the order of execution and placement of data on the hardware. The goal of this task is to automatically synthesize schedules to minimize the execution time of the benchmark.\n",
    "\n",
    "### Client RL Deployment Demo\n",
    "\n",
    "This demo shows how to apply the saved client RL to optimize a test program for Optimizing Image Pipelines. \n",
    "\n",
    "#### *benchmark*\n",
    "harris, interpolate, hist, max_filter, unsharp, nl_mean, lens_blur, local_laplacian, conv_layer, st_chain\n",
    "\n",
    "*approximate runtime ~ 20 minutes*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Environment rebuilding for case study Optimizing Image Pipelines. (~3 mins)\n",
      "Environment rebuilding successful!\n",
      "---------------------------------------------------------------------------------------------------------\n",
      "Start client RL demo (please wait - approximate runtime = 15 minutes)...\n",
      "---------------------------------------------------------------------------------------------------------\n",
      "deploying the best config and policy to RL client and optimize the program...\n",
      "\n",
      "Opened database successfully\n",
      "== Status ==\n",
      "Memory usage on this node: 31.2/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.33 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+-------+\n",
      "| Trial name               | status   | loc   |\n",
      "|--------------------------+----------+-------|\n",
      "| SAC_stoke_rl_6207a_00000 | RUNNING  |       |\n",
      "+--------------------------+----------+-------+\n",
      "\n",
      "\n",
      "\u001b[2m\u001b[36m(pid=12939)\u001b[0m 2022-03-03 17:00:15,719\tWARNING sac_tf_policy.py:51 -- When not using a state-preprocessor with SAC, `fcnet_hiddens` will be set to an empty list! Any hidden layer sizes are defined via `policy_model.hidden_layer_sizes` and `Q_model.hidden_layer_sizes`.\n",
      "\u001b[2m\u001b[36m(pid=12939)\u001b[0m 2022-03-03 17:00:18,388\tWARNING util.py:37 -- Install gputil for GPU system monitoring.\n",
      "== Status ==\n",
      "Memory usage on this node: 31.6/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.33 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc              |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_6207a_00000 | RUNNING  | 172.17.0.7:12939 |      1 |          6.46186 |   50 |    20.98 |\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.6/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.33 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc              |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_6207a_00000 | RUNNING  | 172.17.0.7:12939 |      3 |           14.748 |  150 |        1 |\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.6/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.33 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc              |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_6207a_00000 | RUNNING  | 172.17.0.7:12939 |      5 |          22.7198 |  250 |   1.0002 |\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.6/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.33 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc              |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_6207a_00000 | RUNNING  | 172.17.0.7:12939 |      7 |          30.7209 |  350 |        1 |\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.6/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 1/50 CPUs, 0/0 GPUs, 0.0/59.33 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 RUNNING)\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "| Trial name               | status   | loc              |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+----------+------------------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_6207a_00000 | RUNNING  | 172.17.0.7:12939 |      9 |          38.8035 |  450 |        1 |\n",
      "+--------------------------+----------+------------------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "== Status ==\n",
      "Memory usage on this node: 31.6/125.6 GiB\n",
      "Using FIFO scheduling algorithm.\n",
      "Resources requested: 0/50 CPUs, 0/0 GPUs, 0.0/59.33 GiB heap, 0.0/20.26 GiB objects\n",
      "Result logdir: /home/sys/SUPERSONIC/SuperSonic/logs/model_save/SAC\n",
      "Number of trials: 1 (1 TERMINATED)\n",
      "+--------------------------+------------+-------+--------+------------------+------+----------+\n",
      "| Trial name               | status     | loc   |   iter |   total time (s) |   ts |   reward |\n",
      "|--------------------------+------------+-------+--------+------------------+------+----------|\n",
      "| SAC_stoke_rl_6207a_00000 | TERMINATED |       |     10 |          42.7175 |  500 |        1 |\n",
      "+--------------------------+------------+-------+--------+------------------+------+----------+\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "-------------------------------\n",
      "\n",
      "optimization finished, ready to use our script to evaluate performance\n",
      "-------------------------------\n",
      "\n",
      "start executing, usually takes 3 mins\n",
      "\n",
      "(halide) start execute and calculte baseline\n",
      "(halide) start execute and calculte RL result \n"
     ]
    }
   ],
   "source": [
    "from demo import demo_run\n",
    "\n",
    "#Wait until you see the finish message before moving to the next cell.\n",
    "demo_run(iterations=10, case_study = 'case_1', benchmark = 'harris')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Performance evaluation on benchmarks\n",
    "\n",
    "The results correspond to Figure 3 of the submitted manuscript.\n",
    "\n",
    "*approximate runtime ~30 minutes for one benchmark*\n",
    "\n",
    "#### *data_nums*: \n",
    "This defines the number of test benchmarks. ( 1 - 10 )\n",
    "\n",
    "#### *times*: \n",
    "The number you want to execute the benchmark. ( we set to 100 in our paper, it takes more than 10 hours )\n",
    "\n",
    "Note: There may be other processes running at the same time on the same server, which could introduce noise to the data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "from AE.utils.Calculate import CalculateHalideDemo\n",
    "print(\"Start tuning on Optimizing Image Pipelines (please wait - approximate runtime = 30 minutes per benchmark)...\")\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "HalideSingleDemo = CalculateHalideDemo()\n",
    "HalideSingleDemo.ExecHalideMultipleDemo(times=5,data_nums=10)\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "print(\"Showing the results under different search time budgets (higher is better)\")\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "HalideSingleDemo.PrintTable(\"time\")\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "print(\"Showing the results under different search iteration budgets (higher is better)\")\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "HalideSingleDemo.PrintTable(\"cycle\")\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "print(\"Performance evaluation finished.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Full-scale evaluation data\n",
    "\n",
    "We now plot diagrams using full-scale evaluation data (it would take too long to run the experiment lively). The results correspond to Figure 3 (Section 5.1) of the submitted manuscript."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from AE.utils.Calculate import CalculateHalideDemo\n",
    "HalideDemo = CalculateHalideDemo()\n",
    "# Output Figure 3(a): speedup over Halide master for optimizing image pipelines under different search time(in intel).\n",
    "HalideDemo.plot(\"time\")\n",
    "# Output Figure 3(b): speedup over Halide master for optimizing image pipelines under different iterations(in intel).\n",
    "HalideDemo.plot(\"cycle\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Case Study 2: Neural Network Code Generation (Section 5.2)\n",
    "\n",
    "This task targets DNN back-end code generation to find a good schedule. e.g., instruction orders and data placement to reduce execution time on a multi-core CPU.\n",
    "\n",
    "This demo corresponds to Figure 4 of the submitted manuscript.\n",
    "\n",
    "*approximate runtime = 10 minutes for one benchmark*\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from AE.utils.Calculate_zjq import CalculateTvmDemo\n",
    "'''\n",
    "Class CalculateTvmDemo():TVM evaluation rountine\n",
    "Func ExecTvmDemo(): evaluation interface\n",
    "\n",
    "'''\n",
    "print(\"Start tuning benchmarks on Neural Network Code Generation (please wait - approximate runtime = 10 minutes per benchmark)...\")\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "TvmDemo = CalculateTvmDemo(\"cycle\") \n",
    "TvmDemo.execTvmDemo(cycles = [3,12,16])\n",
    "print(\"Showing the results under different search iteration budgets (higher is better)\")\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "TvmDemo.printTable()\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "print(\"Performing evaluation under different search time constraints.\")\n",
    "TvmDemo = CalculateTvmDemo(\"time\")\n",
    "TvmDemo.execTvmDemo(times = [20,40,60])\n",
    "print(\"Showing the results under different search time budgets (higher is better)\")\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "TvmDemo.printTable()\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "print(\"Performance evaluation finished.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Full-scale evaluation data\n",
    "\n",
    "We now plot the diagrams using full-scale evaluation data (it would take too long to run the experiment lively). The results correspond to Figure 4 (Section 5.2) of the submitted manuscript."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "TvmDemo = CalculateTvmDemo()\n",
    "# Output Figure 4(a): speedup over TVM default for Neural Network Code Generation under different search time(in intel).\n",
    "TvmDemo.plot(\"time\")\n",
    "# Output Figure 4(b): speedup over TVM default for Neural Network Code Generation under different search iterations(in intel).\n",
    "TvmDemo.plot(\"cycle\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Case Study 3: Code Size Reduction (Section 5.3)\n",
    "\n",
    "This task is concerned with determining the LLVM passes and their order to minimize the code size.\n",
    "\n",
    "This demo corresponds to Figure 5 of the submitted manuscript.\n",
    "\n",
    "*approximate runtime = 10 minutes for one benchmark*\n",
    "\n",
    "### Client RL Deployment Demo\n",
    "\n",
    "This demo shows how to apply the saved client RL to optimize a test program for Code Size Reduction. \n",
    "\n",
    "*approximate runtime ~ 15 minutes*\n",
    "\n",
    "#### *benchmark*\n",
    "mandel-text, revertBits, hello, spellcheck, wordfreq, reversefile, lists1 ,misr, whetstone, queens, lists, dt, wc, linpack-pc, perlin, objinst, puzzle, smallpt, blowfish,bzip2, crc32, dijkstra, ghostscript, gsm, ispell, jpeg-c, jpeg-d,lame,patricia,qsort,sha,stringsearch, stringsearch2, susan, tiff2bw, tiffdither, tiffmedian\n",
    "\n",
    "*approximate runtime ~ 20 minutes*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from demo import demo_run\n",
    "\n",
    "#Wait until you see the finish message before moving to the next cell.\n",
    "demo_run(mode=\"deploy\", iterations=10, case_study = 'case_3', benchmark = 'mandel-text')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Performance evaluation on benchmarks\n",
    "\n",
    "The results correspond to Figure 5 of the submitted manuscript.\n",
    "\n",
    "*approximate runtime ~30 minutes for one benchmark*\n",
    "\n",
    "#### *times*: \n",
    "The number you want to execute the benchmark. ( we set to 100 in our paper, it takes more than 10 hours )\n",
    "\n",
    "Note: There may be other processes running at the same time on the same server, which could introduce noise to the data. \n",
    "\n",
    "#### *data_num*: \n",
    "This defines the benchmark account you want to calculate speedup ( 1 - 43 ).  It takes the following values in this case study: 'adpcm', 'dry', 'bitcount', 'recursive', 'rijndael', 'tiff2rgba', total 43 benchmarks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from AE.utils.Calculate import CalculateCSRDemo\n",
    "'''\n",
    "Class CalculateCSRDemo(setting_list,set_up): code size reduction evaluation routine\n",
    ":param setting:for time'setting is [50,100,200,3600] and for cycle's setting is [100,200,400,7200]\n",
    ":param set_up: represent search times or iterations, choose between 'time' (i.e., the search overhead) or 'cycle (i.e., the number of search iterations)'.\n",
    "Func ExecCSRSingleDemo(data): this function perform code size reduction optimization on benchmarks. Our dataset includes 43 benchmarks: mandel-text,revertBits, hello,spellcheck,wordfreq,reversefile,lists1,misr,whetstone,queens,lists,dt,wc,linpack-pc,\n",
    "perlin,objinst,puzzle,smallpt,blowfish,bzip2,crc32,dijkstra,ghostscript,gsm,ispell,jpeg-c,jpeg-d,lame,patricia,qsort,sha,stringsearch,stringsearch2,susan,tiff2bw,tiffdither, and tiffmedian. \n",
    "'''\n",
    "print(\"Start tuning benchmarks on Code Size Reduction (please wait - approximate runtime = 10 minutes per benchmark)...\")\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "CSRTimeDemo = CalculateCSRDemo([\"50\",\"100\",\"200\",\"3600\"],\"time\")\n",
    "CSRCycleDemo = CalculateCSRDemo([\"100\",\"200\",\"400\",\"7200\"],\"cycle\")\n",
    "CSRTimeDemo.ExecCSRMultipleDemo(data_nums=10)\n",
    "print(\"Showing the results under different search iteration budgets (higher is better)\")\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "CSRTimeDemo.PrintTable()\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "CSRCycleDemo.ExecCSRMultipleDemo(data_nums=10)\n",
    "print(\"Showing the results under different search time budgets (higher is better)\")\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "CSRCycleDemo.PrintTable()\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "print(\"Performance evaluation finished.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Full-scale evaluation data\n",
    "\n",
    "We now plot the diagrams using full-scale evaluation data (it would take too long to run the experiment lively). The results correspond to Figure 5 (Section 5.3) of the submitted manuscript."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Output Figure 5(a): reduction over LLVM -Oz for code size reduction under different search time.\n",
    "CSRTimeDemo.plot()\n",
    "# Output Figure 5(b): reduction over LLVM -Oz for code size reduction under different iterations.\n",
    "CSRCycleDemo.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Case Study 4: Superoptimization (Section 5.4)\n",
    "\n",
    "This classical compiler optimization task\n",
    "finds a valid code sequence to maximize the performance\n",
    "of a loop-free sequence of instructions. Superoptimizaiton is an expensive optimization technique as the\n",
    "number of possible configurations grows exponentially as\n",
    "the instruction count to be optimized increases.\n",
    "\n",
    "This demo corresponds to Figure 5 of the submitted manuscript.\n",
    "\n",
    "*approximate runtime = 10 minutes for one benchmark*\n",
    "\n",
    "### Client RL Deployment Demo\n",
    "\n",
    "This demo shows how to apply the saved client RL to optimize a test program for Superoptimization. \n",
    "\n",
    "*approximate runtime ~ 15 minutes*\n",
    "\n",
    "#### *benchmark*\n",
    "\n",
    "p01 - p25"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from demo import demo_run\n",
    "\n",
    "#Wait until you see the finish message before moving to the next cell.\n",
    "demo_run(mode=\"deploy\", iterations=10, case_study = 'case_4', benchmark = 'p20')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Performance evaluation on benchmarks\n",
    "\n",
    "The results correspond to Figure 6 of the submitted manuscript.\n",
    "\n",
    "*approximate runtime ~30 minutes for one benchmark*\n",
    "\n",
    "#### *times*: \n",
    "The number you want to execute the benchmark. ( we set to 100 in our paper, it takes more than 10 hours )\n",
    "\n",
    "Note: There may be other processes running at the same time on the same server, which could introduce noise to the data. \n",
    "\n",
    "#### *data_nums*: \n",
    "This defines the benchmark account you want to calculate speedup. It takes the following values in this case study: p01 - p25"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "#Calculate all the result'time\n",
    "from AE.utils.Calculate import CalculateStokeDemo\n",
    "print(\"Start tuning benchmarks on Superoptimization (please wait - approximate runtime = 10 minutes per benchmark)...\")\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "StokeSingleDemo = CalculateStokeDemo()\n",
    "# StokeSingleDemo.ExecStokeSingleDemo(data=\"p20\",times=1)\n",
    "StokeSingleDemo.ExecStokeMultipleDemo(times=10,data_nums=10)\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "\n",
    "print(\"Showing the results under different search time budgets (higher is better)\")\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "StokeSingleDemo.PrintTable(\"time\")\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "print(\"Showing the results under different search iteration budgets (higher is better)\")\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "StokeSingleDemo.PrintTable(\"cycle\")\n",
    "print(\"---------------------------------------------------------------------------------------------------------\")\n",
    "\n",
    "#Wait until you see this message before moving to the next cell.\n",
    "print(\"Performance evaluation finished.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Full-scale evaluation data\n",
    "\n",
    "We now plot the diagrams using full-scale evaluation data (it would take too long to run the experiment lively). The results correspond to Figure 6 (Section 5.4) of the submitted manuscript."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from AE.utils.Calculate import CalculateStokeDemo\n",
    "StokeDemo = CalculateStokeDemo()\n",
    "# Output Figure 6(a): speedup over LLVM/GCC -O3 for superoptimization under different search time (on intel machine).\n",
    "StokeDemo.plot(\"time\")\n",
    "# Output Figure 6(b): speedup over LLVM/GCC -O3 for superoptimization under different iterations (on intel machine).\n",
    "StokeDemo.plot(\"cycle\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}