Skip to content

Commit

Permalink
Indicator search (#1454)
Browse files Browse the repository at this point in the history
<!--Please ensure the PR fulfills the following requirements! -->
<!-- If this is your first PR, make sure to add your details to the
AUTHORS.rst! -->
### Pull Request Checklist:
- [x] This PR addresses an already opened issue (for bug fixes /
features)
    - This PR is a step towards fixing #1433
- [ ] Tests for the changes have been added (for bug fixes / features)
- [ ] (If applicable) Documentation has been added / updated (for bug
fixes / features)
- [x] CHANGES.rst has been updated (with summary of main changes)
- [ ] Link to issue (:issue:`number`) and pull request (:pull:`number`)
has been added

### What kind of change does this PR introduce?

* This rewrites the "Climate Indicators" page of the doc so that it
presents a searchable un-categorized list of indicators.
* I also added the "used variables" to the short description.

### Does this PR introduce a breaking change?
No.


### Other information:
The changes are usable on RTD's build :
https://xclim--1454.org.readthedocs.build/en/1454/indicators.html

The first commit is only a prototype. For now, it shows a list of
indicators with title, python name and variables. The list is searchable
by "free text", but it only looks in the titles. I'd like to:

- [x] Add keywords to the description
- [x] Add the abstract to the description. Should it be directly
inserted ? A foldable box ? A mouseover box ?
- [x] Search by keywords (this will be a bit more heavy on the JS
side...
    - [ ] by realms
    - [ ] by variables
- [ ] Add a mouseover short description of the variables
  • Loading branch information
aulemahal authored Sep 29, 2023
2 parents 6370346 + b3d0e6b commit 0fb7b46
Show file tree
Hide file tree
Showing 10 changed files with 199 additions and 176 deletions.
1 change: 1 addition & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ New features and enhancements
* `xclim` now has a dedicated console command for prefetching testing data from `xclim-testdata` with branch options (e.g.: `$ xclim prefetch_testing_data --branch some_development_branch`). This command can be used to download the testing data to a local cache, which can then be used to run the testing suite without internet access or in "offline" mode. For more information, see the contributing documentation section for `Updating Testing Data`. (:issue:`1468`, :pull:`1473`).
* The testing suite now offers a means of running tests in "offline" mode (using `pytest-socket <https://github.com/miketheman/pytest-socket>`_ to block external connections). This requires a local copy of `xclim-testdata` to be present in the user's home cache directory and for certain `pytest` options and markers to be set when invoked. For more information, see the contributing documentation section for `Running Tests in Offline Mode`. (:issue:`1468`, :pull:`1473`).
* The `SKIP_NOTEBOOKS` flag to speed up docs builds is now documented. See the contributing documentation section `Get Started!` for details. (:issue:`1470`, :pull:`1476`).
* Refactored the indicators page with the addition of a search bar.

Bug fixes
^^^^^^^^^
Expand Down
8 changes: 6 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ define BROWSER_PYSCRIPT
import os, webbrowser, sys
from urllib.request import pathname2url

webbrowser.open(f"file://{pathname2url(os.path.abspath(sys.argv[1]))}")
webbrowser.open(sys.argv[1])
endef
export BROWSER_PYSCRIPT

Expand Down Expand Up @@ -93,7 +93,11 @@ linkcheck: autodoc-custom-index ## run checks over all external links found thro
docs: autodoc-custom-index ## generate Sphinx HTML documentation, including API docs, but without indexes for for indices and indicators
$(MAKE) -C docs html
ifndef READTHEDOCS
$(BROWSER) docs/_build/html/index.html
## Start http server and show in browser.
## We want to have the cli command run in the foreground, so it's easy to kill.
## And we wait 2 sec for the server to start before opening the browser.
\{ sleep 2; $(BROWSER) http://localhost:54345 \} &
python -m http.server 54345 --directory docs/_build/html/
endif

servedocs: docs ## compile the docs watching for changes
Expand Down
95 changes: 95 additions & 0 deletions docs/_static/indsearch.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
/* Array of indicator objects */
let indicators = [];

/* MiniSearch object defining search mechanism */
let miniSearch = new MiniSearch({
fields: ['title', 'abstract', 'variables', 'keywords'], // fields to index for full-text search
storeFields: ['title', 'abstract', 'vars', 'realm', 'module', 'name', 'keywords'], // fields to return with search results
searchOptions: {
boost: {'title': 3, 'variables': 2},
fuzzy: 0.1,
prefix: true,
},
extractField: (doc, field) => {
if (field === 'variables') {
return Object.keys(doc['vars']).join(' ');
}
return MiniSearch.getDefault('extractField')(doc, field);
}
});

// Populate search object with complete list of indicators
fetch('indicators.json')
.then(data => data.json())
.then(data => {
indicators = Object.entries(data).map(([k, v]) => {
return {id: k.toLowerCase(), ...v}
});
miniSearch.addAll(indicators);
indFilter();
});


// Populate list of variables
//function getVariables() {
// fetch('variables.json')
// .then((res) => {
// return res.json();
// })
//}
//const variables = getVariables();


function makeKeywordLabel(ind) {
/* Print list of keywords only if there is at least one. */
if (ind.keywords[0].length > 0) {
const keywords = ind.keywords.map(v => `<code class="keywordlabel">${v.trim()}</code>`).join('');
return `<div class="keywords">Keywords: ${keywords}</div>`;
}
else {
return "";
}
}


function makeVariableList(ind) {
/* Print list of variables and include mouse-hover tooltip with variable description. */
return Object.entries(ind.vars).map((kv) => {
const tooltip = `<button class="indVarname" title="${kv[1]}" alt="${kv[1]}">${kv[0]}</button>`;
return tooltip
}).join('');
}

function indTemplate(ind) {
// const varlist = Object.entries(ind.vars).map((kv) => `<code class="indVarname">${kv[0]}</code>`).join('');
const varlist = makeVariableList(ind);
return `
<div class="indElem" id="${ind.id}">
<div class="indHeader">
<b class="indTitle">${ind.title}</b>
<a class="reference_internal indName" href="api.html#xclim.indicators.${ind.module}.${ind.name}" title="${ind.name}">
<code>${ind.module}.${ind.name}</code>
</a>
</div>
<div class="indVars">Uses: ${varlist}</div>
<div class="indDesc"><p>${ind.abstract}</p></div>
${makeKeywordLabel(ind)}
<div class="indID">Yaml ID: <code>${ind.id}</code></div>
</div>
`;
}

function indFilter() {
const input = document.getElementById("queryInput").value;
let inds = [];
if (input === "") {
inds = indicators;
} else {
inds = miniSearch.search(input);
}

const newTable = inds.map(indTemplate).join('');
const tableElem = document.getElementById("indTable");
tableElem.innerHTML = newTable;
return newTable;
}
33 changes: 33 additions & 0 deletions docs/_static/style.css
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,36 @@ ul.simple li {
#contents .toctree-wrapper:first-of-type ul {
margin-bottom: 0;
}

#queryInput {
width: 100%;
padding: 10px;
margin: 5px;
}

.indElem {
margin: 10px;
padding: 10px;
background-color: #ddd;
border-radius: 10px;
}

.indName {
float: right;
}

code > .indName {
background-color: #ccc;
}

.indVarname {
border-radius: 10px;
}

/* Rounded corners for keyword labels: */
.keywordlabel {
border-radius: 10px;
padding: 5px;
margin: 5px;
background-color: #ddd;
}
14 changes: 14 additions & 0 deletions docs/_templates/layout.html
Original file line number Diff line number Diff line change
@@ -1,2 +1,16 @@
{% extends "!layout.html" %}
{% set css_files = css_files + ["_static/style.css"] %}

<!-- Injection of the scripts for generating the indicators page data.
We do it this way to injected only where needed and to ensure they get
loaded before all other scripts. nbsphinx adds a line to load require.js which
breaks MiniSearch if loaded first.
-->

{% block scripts %}
{% if "indicators" in sourcename %}
<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/umd/index.min.js"></script>
<script type="text/javascript" src="./_static/indsearch.js"></script>
{% endif %}
{{ super() }}
{% endblock %}
86 changes: 39 additions & 47 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,13 @@
from __future__ import annotations

import datetime
import json
import os
import sys
import warnings
from collections import OrderedDict

import xarray
import yaml
from pybtex.plugin import register_plugin # noqa
from pybtex.style.formatting.alpha import Style as AlphaStyle # noqa
from pybtex.style.labels import BaseLabelStyle # noqa
Expand All @@ -29,6 +30,7 @@
xarray.CFTimeIndex.__module__ = "xarray"

import xclim # noqa
from xclim.core.indicator import registry # noqa

# If extensions (or modules to document with autodoc) are in another
# directory, add these directories to sys.path here. If the directory is
Expand All @@ -38,51 +40,40 @@
sys.path.insert(0, os.path.abspath(".."))
sys.path.insert(0, os.path.abspath("."))


def _get_indicators(module):
"""For all modules or classes listed, return the children that are instances of registered Indicator classes.
module : A xclim module.
"""
from xclim.core.indicator import registry

out = {}
for key, val in module.__dict__.items():
if hasattr(val, "_registry_id") and val._registry_id in registry: # noqa
out[key] = val

return OrderedDict(sorted(out.items()))


def _indicator_table(module):
"""Return a sequence of dicts storing metadata about all available indices in xclim."""
inds = _get_indicators(getattr(xclim.indicators, module))
table = {}
for ind_name, ind in inds.items():
# Apply default values
# args = {
# name: p.default if p.default != inspect._empty else f"<{name}>"
# for (name, p) in ind._sig.parameters.items()
# }
try:
table[ind_name] = ind.json() # args?
except KeyError as err:
warnings.warn(
f"{ind.identifier} could not be documented.({err})", UserWarning
)
else:
table[ind_name]["doc"] = ind.__doc__
if ind.compute.__module__.endswith("generic"):
table[ind_name][
"function"
] = f"xclim.indices.generic.{ind.compute.__name__}"
else:
table[ind_name]["function"] = f"xclim.indices.{ind.compute.__name__}"
return table


modules = ("atmos", "generic", "land", "seaIce", "cf", "icclim", "anuclim")
indicators = {module: _indicator_table(module) for module in modules}
# Indicator data for populating the searchable indicators page
# Get all indicators and some information about them
indicators = {}
# FIXME: Include cf module when its indicators documentation is improved.
for module in ("atmos", "generic", "land", "seaIce", "icclim", "anuclim"):
for key, ind in getattr(xclim.indicators, module).__dict__.items():
if hasattr(ind, "_registry_id") and ind._registry_id in registry: # noqa
indicators[ind._registry_id] = {
"realm": ind.realm,
"title": ind.title,
"name": key,
"module": module,
"abstract": ind.abstract,
"vars": {
param_name: f"{param.description}"
for param_name, param in ind._all_parameters.items()
if param.kind < 2 and not param.injected
},
"keywords": ind.keywords.split(","),
}
# Sort by title
indicators = dict(sorted(indicators.items(), key=lambda kv: kv[1]["title"]))

# Dump indicators to json. The json is added to the html output (html_extra_path)
# It is read by _static/indsearch.js to populate the table in indicators.rst
with open("indicators.json", "w") as f:
json.dump(indicators, f)


# Dump variables information
with open("variables.json", "w") as fout:
with open("../xclim/data/variables.yml") as fin:
data = yaml.safe_load(fin)
json.dump(data, fout)

# -- General configuration ---------------------------------------------

Expand Down Expand Up @@ -233,6 +224,7 @@ class XCStyle(AlphaStyle):
"""
nbsphinx_timeout = 300
nbsphinx_allow_errors = False
# nbsphinx_requirejs_path = "" # To make MiniSearch work in the indicators page

# Add any paths that contain templates here, relative to this directory.
templates_path = ["_templates"]
Expand Down Expand Up @@ -294,7 +286,7 @@ class XCStyle(AlphaStyle):

html_theme = "sphinx_rtd_theme"

html_context = {"indicators": indicators}
html_extra_path = ["indicators.json", "variables.json"]

# Theme options are theme-specific and customize the look and feel of a
# theme further. For a list of options available for each theme, see the
Expand Down
Loading

0 comments on commit 0fb7b46

Please sign in to comment.