refactor: Centralize Vega project versioning

NOTES_FOR_MAINTAINERS.md

@@ -27,11 +27,62 @@ The script output is designed to be deterministic; if the vega-lite version
 is not changed, then running the script should overwrite the schema wrappers
 with identical copies.
 
-## Updating the Vega-Lite version
+## Updating Vega versions
+All versions are maintained in [pyproject.toml](pyproject.toml).
 
-The vega & vega-lite versions for the Python code can be updated by manually
-changing the ``SCHEMA_VERSION`` definition within
-``tools/generate_schema_wrapper.py``, and then re-running the script.
+### Python Packages
+
+Projects which publish a package to PyPI are listed with a version bound in one of the following tables:
+
+- [`project.dependencies`](https://packaging.python.org/en/latest/specifications/pyproject-toml/#dependencies-optional-dependencies): Published dependencies.
+- [`project.optional-dependencies`](https://packaging.python.org/en/latest/specifications/pyproject-toml/#dependencies-optional-dependencies): Published optional dependencies, or "extras".
+- [`dependency-groups`](https://peps.python.org/pep-0735/): Local dependencies for development.
+
+> [!NOTE]
+> All are currently declared in sub-tables of `project.optional-dependencies`
+
+#### `vl-convert`
+
+We need to ensure that [vl-convert](https://github.com/vega/vl-convert) includes support for the new Vega-Lite version. 
+Check the [vl-convert releases](https://github.com/vega/vl-convert/releases) to find the minimum
+version of `vl-convert` that includes support for the desired version of Vega-Lite (and [open
+an issue](https://github.com/vega/vl-convert/issues) if this version hasn't been
+included in a released yet).
+
+#### `vegafusion`
+
+
+> [!WARNING]
+> *TODO: Find out how the constraint for `vegafusion` is decided*
+
+### Javascript/other
+
+Additional version constraints, including for [`Vega-Lite`](https://github.com/vega/vega-lite) itself are declared in `[tool.altair.vega]`.
+
+Whereas the [previous dependencies](#python-packages) are used at *install-time*; these are embedded into `altair` for use at *runtime* or when [generating the python code](#auto-generating-the-python-code):
+
+```toml
+[tool.altair.vega]
+vega-datasets     = "..." # https://github.com/vega/vega-datasets
+vega-embed        = "..." # https://github.com/vega/vega-embed
+vega-lite         = "..." # https://github.com/vega/vega-lite
+vegafusion        = "..." # https://github.com/vega/vegafusion
+vl-convert-python = "..." # https://github.com/vega/vl-convert
+```
+
+Some examples of where these propagate to:
+- [altair/jupyter/js/index.js](altair/jupyter/js/index.js)
+- [altair/utils/_importers.py](altair/utils/_importers.py)
+- [tools/generate_schema_wrapper.py](tools/generate_schema_wrapper.py)
+- [tools/versioning.py](tools/versioning.py)
+- [altair/utils/schemapi.py](https://github.com/vega/altair/blob/0e23fd33e9a755bab0ef73a856340c48c14897e6/altair/utils/schemapi.py#L1619-L1640)
+
+> [!IMPORTANT]
+> When updating **any** of these versions, be sure to [re-generate the python code](#auto-generating-the-python-code).
+
+#### Updating the Vega-Lite version
+
+The Vega-Lite version for the Python code propagates to `tools.generate_schema_wrapper.SCHEMA_VERSION`.
 
 This will update all of the automatically-generated files in the ``schema``
 directory for each version, but please note that it will *not* update other
@@ -50,30 +101,6 @@ of some docstrings.
 Major version updates (e.g. Vega-Lite 1.X->2.X) have required substantial
 rewrites, because the internal structure of the schema changed appreciably.
 
-### Updating the Vega-Lite in JupyterChart
-To update the Vega-Lite version used in JupyterChart, update the version in the 
-esm.sh URL in `altair/jupyter/js/index.js`.
-
-For example, to update to Vega-Lite 5.15.1, Vega 5 and Vega-Embed 6, the URL 
-should be:
-
-```javascript
-import embed from "https://esm.sh/vega-embed@6?deps=vega@5&[email protected]";
-```
-
-### Updating vl-convert version bound
-
-When updating the version of Vega-Lite, it's important to ensure that 
-[vl-convert](https://github.com/vega/vl-convert) includes support for the new Vega-Lite version. 
-Check the [vl-convert releases](https://github.com/vega/vl-convert/releases) to find the minimum
-version of vl-convert that includes support for the desired version of Vega-Lite (and [open
-an issue](https://github.com/vega/vl-convert/issues) if this version hasn't been
-included in a released yet.). Update the vl-convert version check in `altair/utils/_importers.py` 
-with the new minimum required version of vl-convert.
-
-Also, the version bound of the `vl-convert-python` package should be updated in the 
-`[project.optional-dependencies]/all` dependency group in `pyproject.toml`.
-
 ## Releasing the Package
 
 To cut a new release of Altair, follow the steps outlined in

RELEASING.md

@@ -1,16 +1,18 @@
-1. Make sure to have an environment set up with `hatch` installed. See `CONTRIBUTING.md`.
+1. Check all [Vega project](https://github.com/orgs/vega/repositories?type=source) versions are up-to-date. See [NOTES_FOR_MAINTAINERS.md](NOTES_FOR_MAINTAINERS.md)
+
+2. Make sure to have an environment set up with `hatch` installed. See [CONTRIBUTING.md](CONTRIBUTING.md).
    Remove any existing environments managed by `hatch` so that it will create new ones
    with the latest dependencies when executing the commands further below:
 
         hatch env prune
 
-2. Make certain your branch is in sync with head, and that you have no uncommitted modifications. If you work on a fork, replace `origin` with `upstream`:
+3. Make certain your branch is in sync with head, and that you have no uncommitted modifications. If you work on a fork, replace `origin` with `upstream`:
 
         git checkout main
         git pull origin main
         git status  # Should show "nothing to commit, working tree clean"
 
-3. Do a clean doc build:
+4. Do a clean doc build:
 
         hatch run doc:clean-all
         hatch run doc:build-html
@@ -19,62 +21,62 @@
    Navigate to http://localhost:8000 and ensure it looks OK (particularly
    do a visual scan of the gallery thumbnails).
 
-4. Create a new release branch:
+5. Create a new release branch:
 
         git switch -c version_5.0.0
 
-5. Update version to, e.g. 5.0.0:
+6. Update version to, e.g. 5.0.0:
 
    - in ``altair/__init__.py``
    - in ``doc/conf.py``
 
-6. Commit changes and push:
+7. Commit changes and push:
 
         git add . -u
         git commit -m "chore: Bump version to 5.0.0"
         git push
 
-7. Merge release branch into main, make sure that all required checks pass
+8. Merge release branch into main, make sure that all required checks pass
 
-8. On main, build source & wheel distributions. If you work on a fork, replace `origin` with `upstream`:
+9.  On main, build source & wheel distributions. If you work on a fork, replace `origin` with `upstream`:
 
         git switch main
         git pull origin main
         hatch clean  # clean old builds & distributions
         hatch build  # create a source distribution and universal wheel
 
-9. publish to PyPI (Requires correct PyPI owner permissions):
+10. publish to PyPI (Requires correct PyPI owner permissions):
 
         hatch publish
 
-10. build and publish docs (Requires write-access to altair-viz/altair-viz.github.io):
+11. build and publish docs (Requires write-access to altair-viz/altair-viz.github.io):
 
         hatch run doc:publish-clean-build
 
-11. On main, tag the release. If you work on a fork, replace `origin` with `upstream`:
+12. On main, tag the release. If you work on a fork, replace `origin` with `upstream`:
 
        git tag -a v5.0.0 -m "Version 5.0.0 release"
        git push origin v5.0.0
 
-12. Create a new branch:
+13. Create a new branch:
 
        git switch -c maint_5.1.0dev
 
-13. Update version and add 'dev' suffix, e.g. 5.1.0dev:
+14. Update version and add 'dev' suffix, e.g. 5.1.0dev:
 
     - in ``altair/__init__.py``
     - in ``doc/conf.py``
 
-14. Commit changes and push:
+15. Commit changes and push:
 
         git add . -u
         git commit -m "chore: Bump version to 5.1.0dev"
         git push
 
-15. Merge maintenance branch into main
+16. Merge maintenance branch into main
 
-16. Double-check that a conda-forge pull request is generated from the updated
+17. Double-check that a conda-forge pull request is generated from the updated
     pip package by the conda-forge bot (may take up to several hours):
     https://github.com/conda-forge/altair-feedstock/pulls
 
-17. Publish a new release in https://github.com/vega/altair/releases/
+18. Publish a new release in https://github.com/vega/altair/releases/

altair/utils/__init__.py

@@ -12,10 +12,18 @@
 from .deprecation import AltairDeprecationWarning, deprecated, deprecated_warn
 from .html import spec_to_html
 from .plugin_registry import PluginRegistry
-from .schemapi import Optional, SchemaBase, SchemaLike, Undefined, is_undefined
+from .schemapi import (
+    VERSIONS,
+    Optional,
+    SchemaBase,
+    SchemaLike,
+    Undefined,
+    is_undefined,
+)
 
 __all__ = (
     "SHORTHAND_KEYS",
+    "VERSIONS",
     "AltairDeprecationWarning",
     "Optional",
     "PluginRegistry",

altair/utils/_importers.py

@@ -5,12 +5,14 @@
 
 from packaging.version import Version
 
+from altair.utils.schemapi import VERSIONS
+
 if TYPE_CHECKING:
     from types import ModuleType
 
 
 def import_vegafusion() -> ModuleType:
-    min_version = "1.5.0"
+    min_version = VERSIONS["vegafusion"]
     try:
         import vegafusion as vf
 
@@ -45,7 +47,7 @@ def import_vegafusion() -> ModuleType:
 
 
 def import_vl_convert() -> ModuleType:
-    min_version = "1.6.0"
+    min_version = VERSIONS["vl-convert-python"]
     try:
         version = importlib_version("vl-convert-python")
         if Version(version) < Version(min_version):

altair/utils/schemapi.py

@@ -1614,3 +1614,27 @@ def with_property_setters(cls: type[TSchemaBase]) -> type[TSchemaBase]:
     for prop, propschema in schema.get("properties", {}).items():
         setattr(cls, prop, _PropertySetter(prop, propschema))
     return cls
+
+
+VERSIONS: Mapping[
+    Literal[
+        "vega-datasets", "vega-embed", "vega-lite", "vegafusion", "vl-convert-python"
+    ],
+    str,
+] = {
+    "vega-datasets": "v2.11.0",
+    "vega-embed": "6",
+    "vega-lite": "v5.20.1",
+    "vegafusion": "1.5.0",
+    "vl-convert-python": "1.7.0",
+}
+"""
+Version pins for non-``python`` `vega projects`_.
+
+Notes
+-----
+When cutting a new release, make sure to update ``[tool.altair.vega]`` in ``pyproject.toml``.
+
+.. _vega projects:
+    https://github.com/vega
+"""

pyproject.toml

@@ -84,6 +84,7 @@ dev = [
     "types-setuptools",
     "geopandas",
     "polars>=0.20.3",
+    "tomli; python_version<\"3.11\""
 ]
 doc = [
     "sphinx",
@@ -99,6 +100,14 @@ doc = [
     "scipy",
 ]
 
+[tool.altair.vega]
+# Minimum/exact versions, for projects under the `vega` organization
+vega-datasets     = "v2.11.0" # https://github.com/vega/vega-datasets
+vega-embed        = "6"       # https://github.com/vega/vega-embed
+vega-lite         = "v5.20.1" # https://github.com/vega/vega-lite
+vegafusion        = "1.5.0"   # https://github.com/vega/vegafusion
+vl-convert-python = "1.7.0"   # https://github.com/vega/vl-convert
+
 [tool.hatch.version]
 path = "altair/__init__.py"
 

tools/__init__.py

@@ -4,6 +4,7 @@
     markup,
     schemapi,
     update_init_file,
+    versioning,
 )
 
 __all__ = [
@@ -12,4 +13,5 @@
     "markup",
     "schemapi",
     "update_init_file",
+    "versioning",
 ]

tools/generate_schema_wrapper.py

@@ -41,6 +41,7 @@
     spell_literal,
 )
 from tools.vega_expr import write_expr_module
+from tools.versioning import VERSIONS, inline_versions_literal
 
 if TYPE_CHECKING:
     from collections.abc import Iterable, Iterator
@@ -50,7 +51,7 @@
 
 T = TypeVar("T", bound="str | Iterable[str]")
 
-SCHEMA_VERSION: Final = "v5.20.1"
+SCHEMA_VERSION: Final = VERSIONS["vega-lite"]
 
 
 HEADER_COMMENT = """\
@@ -632,9 +633,8 @@ def copy_schemapi_util() -> None:
         destination_fp.open("w", encoding="utf8") as dest,
     ):
         dest.write(HEADER_COMMENT)
-        dest.writelines(source.readlines())
-    if sys.platform == "win32":
-        ruff.format(destination_fp)
+        dest.writelines(chain(source.readlines(), inline_versions_literal("VERSIONS")))
+    ruff.format(destination_fp)
 
 
 def recursive_dict_update(schema: dict, root: dict, def_dict: dict) -> None:
@@ -1377,13 +1377,16 @@ def generate_encoding_artifacts(
 
 
 def main() -> None:
+    from tools import versioning
+
     parser = argparse.ArgumentParser(
         prog="generate_schema_wrapper.py", description="Generate the Altair package."
     )
     parser.add_argument(
         "--skip-download", action="store_true", help="skip downloading schema files"
     )
     args = parser.parse_args()
+    versioning.update_all_versions()
     copy_schemapi_util()
     vegalite_main(args.skip_download)
     write_expr_module(