diff --git a/FAQ.html b/FAQ.html index 6848c87a..e6c34098 100644 --- a/FAQ.html +++ b/FAQ.html @@ -967,7 +967,7 @@

How can I use F Attributes: standard_name: precipitation_flux long_name: Precipitation - units: kg m-2 s-1 + dtype='float32', name='lat', length=412))
  • standard_name :
    precipitation_flux
    long_name :
    Precipitation
    units :
    kg m-2 s-1
    • diff --git a/FAQ.ipynb b/FAQ.ipynb index 9206728d..b0b311af 100644 --- a/FAQ.ipynb +++ b/FAQ.ipynb @@ -37,10 +37,10 @@ "id": "b0a1cf8a", "metadata": { "execution": { - "iopub.execute_input": "2024-09-27T05:11:19.551830Z", - "iopub.status.busy": "2024-09-27T05:11:19.551657Z", - "iopub.status.idle": "2024-09-27T05:11:20.203877Z", - "shell.execute_reply": "2024-09-27T05:11:20.203401Z" + "iopub.execute_input": "2024-09-27T05:16:52.712477Z", + "iopub.status.busy": "2024-09-27T05:16:52.712316Z", + "iopub.status.idle": "2024-09-27T05:16:53.470341Z", + "shell.execute_reply": "2024-09-27T05:16:53.469857Z" } }, "outputs": [ @@ -72,10 +72,10 @@ "id": "b7a20427", "metadata": { "execution": { - "iopub.execute_input": "2024-09-27T05:11:20.205771Z", - "iopub.status.busy": "2024-09-27T05:11:20.205351Z", - "iopub.status.idle": "2024-09-27T05:11:20.222728Z", - "shell.execute_reply": "2024-09-27T05:11:20.222298Z" + "iopub.execute_input": "2024-09-27T05:16:53.472383Z", + "iopub.status.busy": "2024-09-27T05:16:53.471881Z", + "iopub.status.idle": "2024-09-27T05:16:53.490766Z", + "shell.execute_reply": "2024-09-27T05:16:53.490223Z" } }, "outputs": [ @@ -288,10 +288,10 @@ "id": "dfad0452-92fd-48aa-a74e-feabd79c02ec", "metadata": { "execution": { - "iopub.execute_input": "2024-09-27T05:11:20.224378Z", - "iopub.status.busy": "2024-09-27T05:11:20.224056Z", - "iopub.status.idle": "2024-09-27T05:11:20.835260Z", - "shell.execute_reply": "2024-09-27T05:11:20.834628Z" + "iopub.execute_input": "2024-09-27T05:16:53.492736Z", + "iopub.status.busy": "2024-09-27T05:16:53.492400Z", + "iopub.status.idle": "2024-09-27T05:16:54.151527Z", + "shell.execute_reply": "2024-09-27T05:16:54.150968Z" } }, "outputs": [ @@ -676,7 +676,7 @@ "Attributes:\n", " standard_name: precipitation_flux\n", " long_name: Precipitation\n", - " units: kg m-2 s-1" + " dtype='float32', name='lat', length=412))
  • standard_name :
    precipitation_flux
    long_name :
    Precipitation
    units :
    kg m-2 s-1
    • " ], "text/plain": [ " Size: 54MB\n", @@ -938,10 +938,10 @@ "id": "26c898ed-a77a-442b-a93e-56fb69c88158", "metadata": { "execution": { - "iopub.execute_input": "2024-09-27T05:11:20.837485Z", - "iopub.status.busy": "2024-09-27T05:11:20.836977Z", - "iopub.status.idle": "2024-09-27T05:11:20.842289Z", - "shell.execute_reply": "2024-09-27T05:11:20.841753Z" + "iopub.execute_input": "2024-09-27T05:16:54.153390Z", + "iopub.status.busy": "2024-09-27T05:16:54.153050Z", + "iopub.status.idle": "2024-09-27T05:16:54.158184Z", + "shell.execute_reply": "2024-09-27T05:16:54.157650Z" } }, "outputs": [], @@ -986,10 +986,10 @@ "id": "6ab5a3ab-701a-4a17-beaa-1d9c073dead4", "metadata": { "execution": { - "iopub.execute_input": "2024-09-27T05:11:20.844015Z", - "iopub.status.busy": "2024-09-27T05:11:20.843685Z", - "iopub.status.idle": "2024-09-27T05:11:20.847350Z", - "shell.execute_reply": "2024-09-27T05:11:20.846835Z" + "iopub.execute_input": "2024-09-27T05:16:54.160126Z", + "iopub.status.busy": "2024-09-27T05:16:54.159730Z", + "iopub.status.idle": "2024-09-27T05:16:54.163840Z", + "shell.execute_reply": "2024-09-27T05:16:54.163211Z" } }, "outputs": [], @@ -1029,10 +1029,10 @@ "id": "0ce2c05f-bf41-4cde-8f10-43f4093fa962", "metadata": { "execution": { - "iopub.execute_input": "2024-09-27T05:11:20.848914Z", - "iopub.status.busy": "2024-09-27T05:11:20.848757Z", - "iopub.status.idle": "2024-09-27T05:11:20.852502Z", - "shell.execute_reply": "2024-09-27T05:11:20.852023Z" + "iopub.execute_input": "2024-09-27T05:16:54.165589Z", + "iopub.status.busy": "2024-09-27T05:16:54.165290Z", + "iopub.status.idle": "2024-09-27T05:16:54.169205Z", + "shell.execute_reply": "2024-09-27T05:16:54.168679Z" } }, "outputs": [], diff --git a/Freva.html b/Freva.html index 5fd2f521..46bf8ae1 100644 --- a/Freva.html +++ b/Freva.html @@ -536,7 +536,7 @@

    Searching for dataResults

    -
    {'future': {}, 'future_id': {}, 'model': {'access-cm2': 1, 'cpc': 24, 'mpi-esm1-2-lr': 1, 'mpi-m-mpi-esm-lr-clmcom-cclm4-8-17-v1': 10, 'ncc-noresm1-m-gerics-remo2015-v1': 2, 'nodc': 1}, 'variable': {'hc700': 1, 'pr': 26, 'tas': 10, 'ua': 2}, 'realm': {'atmos': 38, 'ocean': 1}, 'institute': {'clmcom': 10, 'cpc': 24, 'csiro-arccss': 1, 'gerics': 2, 'mpi-m': 1, 'noaa': 1}, 'project': {'cmip6': 2, 'cordex': 12, 'observations': 24, 'reanalysis': 1}, 'grid_label': {'gn': 39}, 'experiment': {'amip': 2, 'cmorph': 24, 'historical': 10, 'oc5': 1, 'rcp85': 2}, 'time_aggregation': {'mean': 39}, 'cmor_table': {'30min': 24, '3hr': 2, 'aday': 10, 'amon': 2, 'omon': 1}, 'product': {'cmip': 2, 'eur-11': 12, 'grid': 24, 'reanalysis': 1}, 'fs_type': {'posix': 39}, 'time_frequency': {'1day': 10, '30min': 24, '3hr': 2, 'mon': 3}, 'ensemble': {'r1i1p1': 37, 'r1i1p1f1': 1, 'r2i1p1f1': 1}}
+
    {'fs_type': {'posix': 39}, 'project': {'cmip6': 2, 'cordex': 12, 'observations': 24, 'reanalysis': 1}, 'future': {}, 'model': {'access-cm2': 1, 'cpc': 24, 'mpi-esm1-2-lr': 1, 'mpi-m-mpi-esm-lr-clmcom-cclm4-8-17-v1': 10, 'ncc-noresm1-m-gerics-remo2015-v1': 2, 'nodc': 1}, 'experiment': {'amip': 2, 'cmorph': 24, 'historical': 10, 'oc5': 1, 'rcp85': 2}, 'variable': {'hc700': 1, 'pr': 26, 'tas': 10, 'ua': 2}, 'time_frequency': {'1day': 10, '30min': 24, '3hr': 2, 'mon': 3}, 'product': {'cmip': 2, 'eur-11': 12, 'grid': 24, 'reanalysis': 1}, 'future_id': {}, 'time_aggregation': {'mean': 39}, 'ensemble': {'r1i1p1': 37, 'r1i1p1f1': 1, 'r2i1p1f1': 1}, 'realm': {'atmos': 38, 'ocean': 1}, 'institute': {'clmcom': 10, 'cpc': 24, 'csiro-arccss': 1, 'gerics': 2, 'mpi-m': 1, 'noaa': 1}, 'grid_label': {'gn': 39}, 'cmor_table': {'30min': 24, '3hr': 2, 'aday': 10, 'amon': 2, 'omon': 1}}
 
    
 
    
 
@@ -598,7 +598,7 @@ 
    Searching for dataResults
    
-
    <generator object SolrFindFiles._search at 0x7f976d3e27a0>
+
    <generator object SolrFindFiles._search at 0x7f174cece7a0>
 /home/runner/work/freva/freva/.docker/data/observations/grid/CPC/CPC/cmorph/30min/atmos/30min/r1i1p1/v20210618/pr/pr_30min_CPC_cmorph_r1i1p1_201609022300-201609022330.nc
 /home/runner/work/freva/freva/.docker/data/observations/grid/CPC/CPC/cmorph/30min/atmos/30min/r1i1p1/v20210618/pr/pr_30min_CPC_cmorph_r1i1p1_201609022200-201609022230.nc
 
    
@@ -684,7 +684,7 @@ 
    Searching for dataResults
    
-
    {'future': [], 'future_id': [], 'model': ['cpc'], 'variable': ['pr'], 'realm': ['atmos'], 'institute': ['cpc'], 'project': ['observations'], 'grid_label': ['gn'], 'experiment': ['cmorph'], 'time_aggregation': ['mean'], 'cmor_table': ['30min'], 'product': ['grid'], 'fs_type': ['posix'], 'time_frequency': ['30min'], 'ensemble': ['r1i1p1']}
+
    {'fs_type': ['posix'], 'project': ['observations'], 'future': [], 'model': ['cpc'], 'experiment': ['cmorph'], 'variable': ['pr'], 'time_frequency': ['30min'], 'product': ['grid'], 'future_id': [], 'time_aggregation': ['mean'], 'ensemble': ['r1i1p1'], 'realm': ['atmos'], 'institute': ['cpc'], 'grid_label': ['gn'], 'cmor_table': ['30min']}
 {'time_frequency': ['30min'], 'variable': ['pr']}
 
    
 
    
@@ -696,7 +696,7 @@ 
    Searching for dataResults
    
-
    ['future', 'future_id', 'model', 'variable', 'realm', 'institute', 'project', 'grid_label', 'experiment', 'time_aggregation', 'cmor_table', 'product', 'fs_type', 'time_frequency', 'ensemble']
+
    ['fs_type', 'project', 'future', 'model', 'experiment', 'variable', 'time_frequency', 'product', 'future_id', 'time_aggregation', 'ensemble', 'realm', 'institute', 'grid_label', 'cmor_table']
 
    
 
    
 
    Reverse search: retrieving meta data from a known file.
    
@@ -708,7 +708,7 @@ 
    Searching for dataResults
    
-
    {'future': [], 'future_id': [], 'model': ['cpc'], 'variable': ['pr'], 'realm': ['atmos'], 'institute': ['cpc'], 'project': ['observations'], 'grid_label': ['gn'], 'experiment': ['cmorph'], 'time_aggregation': ['mean'], 'cmor_table': ['30min'], 'product': ['grid'], 'fs_type': ['posix'], 'time_frequency': ['30min'], 'ensemble': ['r1i1p1']}
+
    {'fs_type': ['posix'], 'project': ['observations'], 'future': [], 'model': ['cpc'], 'experiment': ['cmorph'], 'variable': ['pr'], 'time_frequency': ['30min'], 'product': ['grid'], 'future_id': [], 'time_aggregation': ['mean'], 'ensemble': ['r1i1p1'], 'realm': ['atmos'], 'institute': ['cpc'], 'grid_label': ['gn'], 'cmor_table': ['30min']}
 
    
 
    
 
@@ -931,7 +931,7 @@ 
    Searching for data
    Scheduled job with history id: 2118
 You can view the job's status with the command squeue
 Your job's progress will be shown with the command
-tail -f /tmp/share/slurm/animator/Animator-3427.out
+tail -f /tmp/share/slurm/animator/Animator-3393.out
 ☀️  Waiting for plugin to finish... ok
 
    
 
    
@@ -987,7 +987,7 @@ 
    Searching for dataResults
    
-
    Processing output in /tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051247/2119
+
    Processing output in /tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051824/2119
 finished
 
    
 
    
@@ -1003,7 +1003,7 @@ 
    Searching for dataResults
    
-
    [05:12:47] INFO     freva - INFO - history of {plugin}, limit=1, since=None,    
+
    [05:18:24] INFO     freva - INFO - history of {plugin}, limit=1, since=None,    
                     until=None, entry_ids=None                                  
 
    
 
    
@@ -1058,7 +1058,7 @@ 
    Searching for dataResults
    
-
    Processing output in /tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051247/2120
+
    Processing output in /tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051824/2120
 pr
 
    
 
    
@@ -1103,8 +1103,8 @@ 
    Searching for dataResults
    
-
    Processing output in /tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051247/2121
-Processing output in /tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051247/2121
+
    Processing output in /tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051825/2121
+Processing output in /tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051825/2121
 
 
    
 
    
@@ -1144,7 +1144,7 @@ 
    Searching for data
    Scheduled job with history id: 2122
 You can view the job's status with the command squeue
 Your job's progress will be shown with the command
-tail -f /tmp/share/slurm/dummypluginfolders/DummyPluginFolders-3607.out
+tail -f /tmp/share/slurm/dummypluginfolders/DummyPluginFolders-3574.out
 ☀️  Waiting for plugin to finish... ok
 
    
 
    
@@ -1199,8 +1199,8 @@ 
    Searching for dataResults
    
 
    <class 'list'> 3
 dict_keys(['status_dict', 'id', 'timestamp', 'tool', 'version', 'version_details_id', 'configuration', 'slurm_output', 'host', 'uid_id', 'status', 'flag', 'caption', 'result'])
-{'variable': 'pr', 'outputdir': '/tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051247'}
-[05:12:49] INFO     freva - INFO - history, limit=3, since=None, until=None,    
+{'variable': 'pr', 'outputdir': '/tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051824'}
+[05:18:26] INFO     freva - INFO - history, limit=3, since=None, until=None,    
                     entry_ids=None                                              
 
    
 
    
@@ -1662,7 +1662,7 @@ 
    Returns:#Searching for data: the 
    
 
    
 
    Results
    
-
    grid_label: gn
-product: cmip,eur-11,eur-11b,grid,reanalysis
-project: cmip6,cordex,observations,reanalysis,user-runner
-time_aggregation: mean
-ensemble: r0i0p0,r1i1p1,r1i1p1f1,r2i1p1f1
-time_frequency: 1day,1hr,30min,3hr,mon
-model: access-cm2,cpc,mpi-esm1-2-lr,mpi-m-mpi-esm-lr-clmcom-cclm4-8-17-v1,ncc-noresm1-m-gerics-remo2015-v1,nodc,um-ra2t
+
    time_frequency: 1day,1hr,30min,3hr,mon
 cmor_table: 1hr,30min,3hr,aday,amon,omon
-realm: atmos,ocean,user_data
+fs_type: posix
 experiment: amip,bias-correct,cmorph,historical,oc5,rcp85
-institute: clex,clmcom,cpc,csiro-arccss,gerics,mpi-m,noaa
 variable: hc700,pr,tas,ua
-future: 
-fs_type: posix
+model: access-cm2,cpc,mpi-esm1-2-lr,mpi-m-mpi-esm-lr-clmcom-cclm4-8-17-v1,ncc-noresm1-m-gerics-remo2015-v1,nodc,um-ra2t
+realm: atmos,ocean,user_data
+product: cmip,eur-11,eur-11b,grid,reanalysis
+grid_label: gn
 future_id: 
+future: 
+project: cmip6,cordex,observations,reanalysis,user-runner
+institute: clex,clmcom,cpc,csiro-arccss,gerics,mpi-m,noaa
+ensemble: r0i0p0,r1i1p1,r1i1p1f1,r2i1p1f1
+time_aggregation: mean
 
 
    
 
    
@@ -826,21 +826,21 @@ 
    Searching for data: the 
    
 
    
 
    Results
    
-
    future: 
-future_id: 
+
    fs_type: posix
+project: observations
+future: 
 model: cpc
+experiment: cmorph
 variable: pr
+time_frequency: 30min
+product: grid
+future_id: 
+time_aggregation: mean
+ensemble: r1i1p1
 realm: atmos
 institute: cpc
-project: observations
 grid_label: gn
-experiment: cmorph
-time_aggregation: mean
 cmor_table: 30min
-product: grid
-fs_type: posix
-time_frequency: 30min
-ensemble: r1i1p1

    @@ -1022,7 +1022,7 @@

    Running data analysis plugins: the 
    Scheduled job with history id: 2124
 You can view the job's status with the command squeue
 Your job's progress will be shown with the command
-tail -f /tmp/share/slurm/animator/Animator-3830.out
+tail -f /tmp/share/slurm/animator/Animator-3799.out
    @@ -1057,8 +1057,8 @@

    Running data analysis plugins: the Results

    -
    Processing output in /tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051353/2125
-/tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051353/2125/data.nc
+
    Processing output in /tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051934/2125
+/tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051934/2125/data.nc
 
    
 
    
 
    
@@ -1115,7 +1115,7 @@ 
    Inspecting previous analysis jobs: the Results
    
-
    2115) dummyplugin [2024-09-27 05:10:20] finished {"number": null, "the_number": 5, "something": "test", "other": 1.4...
+
    2115) dummyplugin [2024-09-27 05:15:49] finished {"number": null, "the_number": 5, "something": "test", "other": 1.4...
 
 
    
 
    
@@ -1127,9 +1127,9 @@ 
    Inspecting previous analysis jobs: the Results
    
-
    2125) dummypluginfolders(0, 1, 2) ede8070fec25d5677d36be8062aae49ff2cf58cc [2024-09-27 05:13:53] finished 
+
    2125) dummypluginfolders(0, 1, 2) 8a959e48d299db5cbfd1dccd25e2536e64aeb7fd [2024-09-27 05:19:34] finished 
 Configuration:
-      outputdir=/tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051353
+      outputdir=/tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051934
        variable=tas
 
 
    
@@ -1140,7 +1140,7 @@ 
    Inspecting previous analysis jobs: the Results
    
-
    freva-plugin dummypluginfolders variable=tas outputdir=/tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051353
+
    freva-plugin dummypluginfolders variable=tas outputdir=/tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051934
 
 
    
 
    
@@ -1152,10 +1152,10 @@ 
    Inspecting previous analysis jobs: the Results
    
-
    /tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051353/2125/data.nc
-/tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051353/2125/plot.png
+
    /tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051934/2125/data.nc
+/tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051934/2125/plot.png
 /tmp/eval_conf/work/runner/freva-ces/output/animator/2123/pr_20160902T0000-20160902T2330.mp4
-[05:13:57] INFO     freva - INFO - history, limit=3, since=None, until=None,    
+[05:19:39] INFO     freva - INFO - history, limit=3, since=None, until=None,    
                     entry_ids=None                                              
 
    
 
    
@@ -1167,7 +1167,7 @@ 
    Inspecting previous analysis jobs: the Results
    
-
    Processing output in /tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051357/2126
+
    Processing output in /tmp/eval_conf/work/runner/freva-ces/output/dummypluginfolders/20240927_051939/2126
 [
    {
       "status_dict": {
@@ -1179,32 +1179,32 @@ 
    Inspecting previous analysis jobs: the Inspecting previous analysis jobs: the Inspecting previous analysis jobs: the Inspecting previous analysis jobs: the Inspecting previous analysis jobs: the Inspecting previous analysis jobs: the Inspecting previous analysis jobs: the Inspecting previous analysis jobs: the Inspecting previous analysis jobs: the Inspecting previous analysis jobs: the Inspecting previous analysis jobs: the Inspecting previous analysis jobs: the Inspecting previous analysis jobs: the Inspecting previous analysis jobs: the Inspecting previous analysis jobs: the Searching for data: the databrowser method
 
    -<generator object SolrFindFiles._search at 0x7f2ece623760>
+<generator object SolrFindFiles._search at 0x7f45e6c47760>
 
    
 
    
 
    This will return a so called iterator. The advantage of an iterator is that the data can be loaded into memory if needed. Nothing is pre loaded. To access the files you can either loop through the Iterator or convert it to a list:
    
@@ -662,7 +662,7 @@ 
    Searching for data: the databrowser method
 
    -<generator object SolrFindFiles._search at 0x7f2ece623a00>
+<generator object SolrFindFiles._search at 0x7f45e6c47a00>
 
    
 
    
 
    Sometimes it might be useful to subset the data you’re interested in by time. To do so you can use the time search key to subset time steps and whole time ranges. For example let’s get the for certain time range:
    
@@ -718,7 +718,7 @@ 
    Searching for data: the databrowser method
 
    -<generator object SolrFindFiles._search at 0x7f2ece623ca0>
+<generator object SolrFindFiles._search at 0x7f45e6c47ca0>
 
    
 
    
 
    Likewise you can inspect all model facet flags in the databrowser:
    
@@ -736,7 +736,7 @@ 
    Searching for data: the databrowser method
 
    -<generator object SolrFindFiles._search at 0x7f2ece623df0>
+<generator object SolrFindFiles._search at 0x7f45e6c47df0>
 
    
 
    
 
    Note: If you don’t give a search constraints like in the case above the command will query the whole data server.
    
@@ -755,7 +755,7 @@ 
    Searching for data: the databrowser method
 
    -<generator object SolrFindFiles._search at 0x7f2ece623b50>
+<generator object SolrFindFiles._search at 0x7f45e6c47b50>
 
    
 
    
 
    Reverse search is also be possible. You can query the metadata of a given file:
    
@@ -791,7 +791,7 @@ 
    Searching for data: the databrowser method
 
    -<generator object SolrFindFiles._search at 0x7f2ece58c040>
+<generator object SolrFindFiles._search at 0x7f45e666c040>
 
    
 
    
 
    
@@ -1201,7 +1201,7 @@ 
    Example: Using the databrowser to open datasets with xarray
  • lon
    (lon)
    float32
    255.0 255.1 255.2 ... 304.9 305.0
    standard_name :
    longitude
    long_name :
    longitude
    units :
    degrees_east
    axis :
    X
    array([255.04851, 255.12126, 255.19402, ..., 304.81406, 304.8868 , 304.9596 ],
-      dtype=float32)
  • lat
    (lat)
    float32
    15.06 15.14 15.21 ... 44.9 44.97
    standard_name :
    latitude
    long_name :
    latitude
    units :
    degrees_north
    axis :
    Y
    array([15.063675, 15.136447, 15.209218, ..., 44.827168, 44.89994 , 44.97271 ],
-      dtype=float32)