Merge pull request #263 from Ouranosinc/require_all_on_doc

add doc for require_all_on

HISTORY.rst

@@ -4,7 +4,7 @@ History
 
 v0.8.0 (unreleased)
 -------------------
-Contributors to this version: Gabriel Rondeau-Genesse (:user:`RondeauG`), Pascal Bourgault (:user:`aulemahal`).
+Contributors to this version: Gabriel Rondeau-Genesse (:user:`RondeauG`), Pascal Bourgault (:user:`aulemahal`), Juliette Lavoie (:user:`juliettelavoie`).
 
 Announcements
 ^^^^^^^^^^^^^
@@ -15,6 +15,7 @@ New features and enhancements
 * Added the ability to search for simulations that reach a given warming level. (:pull:`251`).
 * ``xs.spatial_mean`` now accepts the ``region="global"`` keyword to perform a global average (:issue:`94`, :pull:`260`).
 * ``xs.spatial_mean`` with ``method='xESMF'`` will also automatically segmentize polygons (down to a 1° resolution) to ensure a correct average (:pull:`260`).
+* Added documentation for `require_all_on` in `search_data_catalogs`. (:pull:`263`).
 
 Breaking changes
 ^^^^^^^^^^^^^^^^

docs/notebooks/1_catalog.ipynb

@@ -209,6 +209,55 @@
     "print(DC.search(source=\"NorESM2.*\").unique(\"source\"))"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "5d8b7995",
+   "metadata": {},
+   "source": [
+    "Notice that the search function returns everything available that matches some of the criteria."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "29bd4010",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# r1i1p1f1 sftlf is not available\n",
+    "DC.search(\n",
+    "    source=\"NorESM2-MM\",\n",
+    "    experiment=\"historical\",\n",
+    "    member=[\"r1i1p1f1\", \"r2i1p1f1\"],\n",
+    "    variable=[\"sftlf\", \"pr\"],\n",
+    ").df"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8abaa553",
+   "metadata": {},
+   "source": [
+    "You can restrict your search to only keep entries that matches all the criteria across a list of columns."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "253d4868",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Only returns variables that have all members, source and experiment asked for. In this case, pr, but not sftlf.\n",
+    "DC.search(\n",
+    "    source=\"NorESM2-MM\",\n",
+    "    experiment=\"historical\",\n",
+    "    member=[\"r1i1p1f1\", \"r2i1p1f1\"],\n",
+    "    variable=[\"sftlf\", \"pr\"],\n",
+    "    require_all_on=[\"variable\"],\n",
+    ").df"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "7f27214c-2524-4fb7-9b95-4a384ed13a53",
@@ -246,7 +295,7 @@
     "Due to how different reference datasets are from climate simulations, this function might have to be called multiple times and the results concatenated into a single dictionary. The main arguments are:\n",
     "\n",
     "- `variables_and_freqs` is used to indicate which variable and which frequency is required. <b> NOTE:</b> With the exception of fixed fields, where *'fx'* should be used, frequencies here use the `pandas` nomenclature ('D', 'H', '6H', 'MS', etc.).\n",
-    "- `other_search_criteria` is used to search for specific entries in other columns of the catalog, such as *activity*.\n",
+    "- `other_search_criteria` is used to search for specific entries in other columns of the catalog, such as *activity*. `require_all_on` can also be passed here.\n",
     "- `exclusions` is used to exclude certain simulations or keywords from the results.\n",
     "- `match_hist_and_fut` is used to indicate that RCP/SSP simulations should be matched with their *historical* counterparts.\n",
     "- `periods` is used to search for specific time periods.\n",
@@ -1102,7 +1151,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.12"
+   "version": "3.9.16"
   }
  },
  "nbformat": 4,

xscen/extract.py

@@ -524,6 +524,8 @@ def search_data_catalogs(
         Variables and freqs to search for, following a 'variable: xr-freq-compatible-str' format. A list of strings can also be provided.
     other_search_criteria : dict, optional
         Other criteria to search for in the catalogs' columns, following a 'column_name: list(subset)' format.
+        You can also pass 'require_all_on: list(columns_name)' in order to only return results that correspond to all other criteria across the listed columns.
+        More details available at https://intake-esm.readthedocs.io/en/stable/how-to/enforce-search-query-criteria-via-require-all-on.html .
     exclusions : dict, optional
         Same as other_search_criteria, but for eliminating results.
     match_hist_and_fut: bool, optional