Add more examples

FREVA-CLINT · Mar 14, 2023 · e927cd4 · e927cd4
1 parent fbb8e67
commit e927cd4
Show file tree

Hide file tree

Showing 11 changed files with 29,124 additions and 146 deletions.
diff --git a/talks/Dkrz_User_Workshop/Figure_01.jpg b/talks/Dkrz_User_Workshop/Figure_01.jpg
diff --git a/talks/Dkrz_User_Workshop/Figure_02-a.jpg b/talks/Dkrz_User_Workshop/Figure_02-a.jpg
diff --git a/talks/Dkrz_User_Workshop/Figure_02.jpg b/talks/Dkrz_User_Workshop/Figure_02.jpg
diff --git a/....ipynb_checkpoints/Freva-checkpoint.ipynb → talks/Dkrz_User_Workshop/Freva.ipynb b/....ipynb_checkpoints/Freva-checkpoint.ipynb → talks/Dkrz_User_Workshop/Freva.ipynb
@@ -33,7 +33,7 @@
     }
    },
    "source": [
-    "<p style=\"color:black;font-size:18px;\">by: Brian Lewis, Christopher Kadow, Etor Lucio Eceiza, Mahesh Ramadoss, Martin Bergemann and many others</p>"
+    "<p style=\"color:black;font-size:18px;\">by: Brian Lewis, Christopher Kadow, Etor Lucio Eceiza, Andrej Fast, Martin Bergemann and many others</p>"
    ]
   },
   {
@@ -45,8 +45,8 @@
     }
    },
    "source": [
-    "<h4 style=\"text-align:center;\">I. Finding Data</h4>\n",
-    "<img src=\"Figure_01.png\" with=\"640px\"></img>"
+    "<h4 style=\"text-align:center;\">Common Problem I: Finding Data</h4>\n",
+    "<img src=\"Figure_01.jpg\" width=\"640px\"></img>"
    ]
   },
   {
@@ -58,8 +58,21 @@
     }
    },
    "source": [
-    "<h4 style=\"text-align:center;\">II. Make use of others code</h4>\n",
-    "<img src=\"Figure_02.png\" with=\"640px\"></img>"
+    "<h4 style=\"text-align:center;\">Common Problem II: Use code of others</h4>\n",
+    "<img src=\"Figure_02.jpg\" width=\"640px\"></img>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1149b078",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "subslide"
+    }
+   },
+   "source": [
+    "<h4 style=\"text-align:center;\">Common Problem III. Remember your own code</h4>\n",
+    "<img src=\"Figure_02.jpg\" width=\"640px\"></img>"
    ]
   },
   {
@@ -137,7 +150,7 @@
     }
    },
    "source": [
-    "## Example: Searching and reading CMIP6 data"
+    "## Example I: Searching and reading CMIP6 data"
    ]
   },
   {
@@ -192,7 +205,7 @@
    "source": [
     "<h4>Web UI</h4>\n",
     "<video controls loop height=\"550\">\n",
-    "    <source src=\"screencast_2.mp4\"type=\"video/mp4\">\n",
+    "    <source src=\"screencast_2.webm\"type=\"video/webm\">\n",
     "    Sorry, your browser doesn't support embedded videos.\n",
     "</video>"
    ]
@@ -219,7 +232,7 @@
     }
    },
    "source": [
-    "## Example Data analysis tool / plugins"
+    "## Example II: Data analysis tool / plugins"
    ]
   },
   {
@@ -303,7 +316,7 @@
     "```python\n",
     "from evaluation_system.api import plugin\n",
     "from evaluation_system.api.parameters import File, ParameterDictionary, SolrField\n",
-    "class PlottingTool(plugin.PluginAbstr|act):\n",
+    "class PlottingTool(plugin.PluginAbstract):\n",
     "\"\"\"Plugin that plots content of selected data.\"\"\"\n",
     "```"
    ]
@@ -317,7 +330,7 @@
     }
    },
    "source": [
-    "<p style=\"margin-bottom: -1.44cm; margin-top: -1.44cm;\"></p>\n",
+    "<p style=\"margin-bottom: -0.5cm; margin-top: -0.5cm;\"></p>\n",
     "```python\n",
     "# Plugin setup\n",
     "    tool_developer = {\"name\": \"Jane Doe\", \"email\": \"[email protected]\"}\n",
@@ -351,7 +364,7 @@
     }
    },
    "source": [
-    "<p style=\"margin-bottom: -1cm; margin-top: -1cm;\"></p>\n",
+    "<p style=\"margin-bottom: -0.2cm; margin-top: -0.2cm;\"></p>\n",
     "```python\n",
     "# Workflow definition of the actual tool\n",
     " def runTool(self, setup: dict[str, str] = {}) -> None:\n",
@@ -411,82 +424,90 @@
   },
   {
    "cell_type": "markdown",
-   "id": "8e02b904",
+   "id": "d2bfbc6d",
    "metadata": {
     "slideshow": {
      "slide_type": "slide"
     }
    },
    "source": [
-    "## A couple of additional features:\n",
-    "- Users can search their own plugin history and other users results\n",
-    "- Web UI suggests similar results on plugin config\n",
-    "- Full integration of workload managers"
+    "## Example III: Users adding their own data"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "3ca11b56",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "slide"
-    }
-   },
+   "id": "3f2e06b5",
+   "metadata": {},
    "source": [
-    "## What's next?"
+    "```python\n",
+    "import freva\n",
+    "import xarray as xr\n",
+    "\n",
+    "user_data = freva.UserData()\n",
+    "user_data.add(\"eur-11b\", \"/tmp/my_awesome_data/outfile_?.nc\",\n",
+    "              institute=\"clex\", model=\"UM-RA2T\",\n",
+    "              experiment=\"Bias-correct\")\n",
+    "dset = xr.open_mfdataset(freva.databrowser(experiment=\"bias*\", experiment=\"clex\"))\n",
+    "```"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "f589a7c8",
+   "id": "8e02b904",
    "metadata": {
     "slideshow": {
-     "slide_type": "subslide"
+     "slide_type": "slide"
     }
    },
    "source": [
-    "## Moving into the cloud\n",
-    "<img src=\"Clouds.jpg\" height=\"240px\"></img>\n",
-    "* Different Data Sources:\n",
-    "    * Cloud storage formats (S3/Swift)\n",
-    "    * ECMWF/ESGF data storage\n",
-    "    * Tape archives"
+    "## A couple of additional features:\n",
+    "- Users can search their own plugin history and other users results\n",
+    "- Web UI suggests similar results on plugin config\n",
+    "- Full integration of workload managers"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "39148a8e",
+   "id": "3ca11b56",
    "metadata": {
     "slideshow": {
-     "slide_type": "subslide"
+     "slide_type": "slide"
     }
    },
    "source": [
-    "## Containerising the User Plugins\n",
-    "<img src=\"docker.png\" height=\"240px\"></img>\n",
-    "* Advantages:\n",
-    "    * Host architecture independent (sort of)\n",
-    "    * Better reproducibility than anaconda\n",
-    "    * Scalable solutions (kubernetes)"
+    "## What's next?"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "e10e36fb",
+   "id": "f589a7c8",
    "metadata": {
     "slideshow": {
      "slide_type": "subslide"
     }
    },
    "source": [
-    "# More features coming up in Warm Wold"
+    "* Freva *has* to be come a RestAPI\n",
+    "    * Clients are easy to implement since we only need a couple of REST calls\n",
+    "* We teach freva the Common Workflow Language (cwl) via toil:\n",
+    "```yaml\n",
+    "cwlVersion: v1.0\n",
+    "class: CommandLineTool\n",
+    "hints:\n",
+    "   SoftwareRequirement:\n",
+    "      packages:\n",
+    "      - package: cdo\n",
+    "baseCommand: cdo\n",
+    "```\n",
+    "* We add a system for [ordering/pre-processing data](https://www-regiklim.dkrz.de/plugins/climpact/setup/)\n",
+    "* We add a simple interface for intake\n"
    ]
   }
  ],
  "metadata": {
   "celltoolbar": "Slideshow",
   "kernelspec": {
-   "display_name": "Plotting for (plotting)",
+   "display_name": "plotting",
    "language": "python",
    "name": "plotting"
   },
@@ -500,7 +521,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.2"
+   "version": "3.10.4"
   }
  },
  "nbformat": 4,