diff --git a/.github/workflows/wiki.yaml b/.github/workflows/wiki.yaml deleted file mode 100644 index d85354895..000000000 --- a/.github/workflows/wiki.yaml +++ /dev/null @@ -1,63 +0,0 @@ -name: wiki -on: - release: - types: [released] - push: - paths: - - '.github/workflows/wiki.yaml' - - 'wiki/**' - - src/rez/rezconfig.py - -jobs: - build: - name: Build Wiki Artifact - runs-on: ubuntu-latest - - steps: - - uses: actions/checkout@v4 - with: - fetch-depth: 0 # required to generate credits list - - - uses: actions/setup-python@v4 - with: - python-version: 3.7 - - - name: Build Wiki - working-directory: wiki - run: | - python generate-wiki.py \ - --github-repo="${{ github.repository }}" \ - --out="${{ github.workspace }}/out" - - - uses: actions/upload-artifact@v3 - with: - name: wiki-markdown - path: out - - publish: - name: Publish to GitHub Wiki - runs-on: ubuntu-latest - needs: build - if: github.event_name == 'release' - - steps: - - name: Setup git config - run: | - git config --global user.name "github.com/${{ github.actor }}" - git config --global user.email "${{ github.actor }}@${{ github.sha }}" - - - name: Clone latest wiki repo - run: | - git clone https://${{ github.actor }}:${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}.wiki.git . - - - uses: actions/download-artifact@v3 - with: - name: wiki-markdown - path: . - - - name: Push wiki updates to git - run: | - git add --all - ( git commit -m "Generated from GitHub ${{ github.workflow }} Workflow" \ - && git push origin main \ - ) || echo "Nothing new to commit" diff --git a/CODEOWNERS b/CODEOWNERS index 92e3280dd..f2bf703f3 100644 --- a/CODEOWNERS +++ b/CODEOWNERS @@ -18,7 +18,7 @@ NOTICE @AcademySoftwareFoundation/rez-tsc /src/rez/data/ @AcademySoftwareFoundation/rez-tsc # Docs -/wiki/ @AcademySoftwareFoundation/rez-tsc +/docs/ @AcademySoftwareFoundation/rez-tsc # Core /src/rez/backport @AcademySoftwareFoundation/rez-tsc diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c1df2e49a..fc1dbd542 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -15,10 +15,10 @@ sending a pull request. Please follow these guidelines: 7. Use [this format](https://help.github.com/articles/closing-issues-using-keywords/) to mention the issue(s) your PR closes; 8. Add relevant tests to demonstrate that your changes work; -9. Add relevant documentation [here](wiki/pages) to document your changes, if applicable. Those +9. Add relevant documentation [here](docs/source) to document your changes, if applicable. Those markdown files prefixed with `_` are internal and should not be changed. 10. If your changes add a new rez config setting, update [rezconfig.py](src/rez/rezconfig.py) and - document the setting. The comments in this file are extracted and turned into Wiki content. Pay + document the setting. The comments in this file are extracted and turned into documentation. Pay attention to the comment formatting and follow the existing style closely. ## CLA diff --git a/INSTALL.md b/INSTALL.md index ebbfd3c82..59ce8c35e 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -1,89 +1 @@ -## Installation Script - -> [[media/icons/warning.png]] Only Python greater or eual to 3.7 is supported -> when installing via the install script! - -To install rez, download the source. Then from the root directory, run: - -``` -]$ python ./install.py -``` - -This installs rez to `/opt/rez`. See `install.py -h` for how to install to a -different location. - -Once the installation is complete, a message tells you how to run it: - -``` -SUCCESS! To activate Rez, add the following path to $PATH: -/opt/rez/bin/rez - -You may also want to source the completion script (for bash): -source /opt/rez/completion/complete.sh -``` - -> [[media/icons/warning.png]] Do _not_ move the installation - re-install to a new -> location if you want to change the install path. If you want to install rez for -> multiple operating systems, perform separate installs for each of those systems. - - -## Installation Via Pip - -It is possible to install rez with pip, like so: - -``` -]$ pip install rez -``` - -> [[media/icons/info.png]] While the `install.py` script only supoprts Python 3, -> rez is still usable with Python when installed via pip. In other words, the API -> is fully compatible with Python 2 and 3. - -However, this comes with a caveat - rez command line tools _are not guaranteed -to work correctly_ once inside a rez environment (ie after using the `rez-env` -command). The reasons are given in the next section. - -Pip installation is adequate however, if all you require is the rez API, or you -don't require its command line tools to be available within a resolved environment. - -Note that running pip-installed rez command line tools will print a warning like so: - -``` -Pip-based rez installation detected. Please be aware that rez command line tools -are not guaranteed to function correctly in this case. See -https://github.com/AcademySoftwareFoundation/rez/wiki/Installation#why-not-pip-for-production -for futher details. -``` - - -## Why Not Pip For Production? - -Rez is not a normal python package. Although it can successfully be installed -using standard mechanisms such as pip, this comes with a number of caveats. -Specifically: - -* When within a rez environment (ie after using the `rez-env` command), the rez - command line tools are not guaranteed to function correctly; -* When within a rez environment, other packages' tools (that were also installed - with pip) remain visible, but are not guaranteed to work. - -When you enter a rez environment, the rez packages in the resolve configure -that environment as they see fit. For example, it is not uncommon for a python -package to append to PYTHONPATH. Environment variables such as PYTHONPATH -affect the behaviour of tools, including rez itself, and this can cause it to -crash or behave abnormally. - -When you use the `install.py` script to install rez, some extra steps are taken -to avoid this problem. Specifically: - -* Rez is installed into a virtualenv so that it operates standalone; -* The rez tools are shebanged with `python -E`, in order to protect them from - environment variables that affect python's behaviour; -* The rez tools are stored in their own directory, so that other unrelated tools - are not visible. - -Due to the way standard wheel-based python installations work, it simply is not -possible to perform these extra steps without using a custom installation script. -Wheels do not give the opportunity to run post-installation code; neither do -they provide functionality for specifying interpreter arguments to be added for -any given entry point. +See https://rez.readthedocs.io/en/stable/installation.html for installation instructions. diff --git a/README.md b/README.md index 2cc9f0ba0..eecb96a86 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,6 @@ [![Windows](https://github.com/AcademySoftwareFoundation/rez/workflows/windows/badge.svg?branch=main)](https://github.com/AcademySoftwareFoundation/rez/actions?query=workflow%3AWindows+branch%3Amain)
[![Installation](https://github.com/AcademySoftwareFoundation/rez/workflows/installation/badge.svg?branch=main)](https://github.com/AcademySoftwareFoundation/rez/actions?query=workflow%3Ainstallation+branch%3Amain) [![Flake8](https://github.com/AcademySoftwareFoundation/rez/workflows/flake8/badge.svg?branch=main)](https://github.com/AcademySoftwareFoundation/rez/actions?query=workflow%3Aflake8+branch%3Amain) -[![Wiki](https://github.com/AcademySoftwareFoundation/rez/workflows/wiki/badge.svg)](https://github.com/AcademySoftwareFoundation/rez/actions?query=workflow%3Awiki+event%3Arelease) [![Pypi](https://github.com/AcademySoftwareFoundation/rez/workflows/pypi/badge.svg)](https://github.com/AcademySoftwareFoundation/rez/actions?query=workflow%3Apypi+event%3Arelease) [![Benchmark](https://github.com/AcademySoftwareFoundation/rez/workflows/benchmark/badge.svg)](https://github.com/AcademySoftwareFoundation/rez/actions?query=workflow%3Abenchmark+event%3Arelease)
[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=AcademySoftwareFoundation_rez&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=AcademySoftwareFoundation_rez) @@ -35,18 +34,18 @@ environments reference these existing packages. This means that configured envir are lightweight, and very fast to create, often taking just a few seconds to configure despite containing hundreds of packages. -See [the wiki](https://github.com/AcademySoftwareFoundation/rez/wiki) for full documentation. +See [the wiki](https://rez.readthedocs.io) for full documentation.

- - + +
Typical package managers install packages into an environment


- - + +
Rez installs packages once, and configures environments dynamically

@@ -90,7 +89,7 @@ and when re-evaluated later will reconstruct the same environment once more. ## Examples This example places the user into a resolved shell containing the requested packages, -using the [rez-env](https://github.com/AcademySoftwareFoundation/rez/wiki/Command-Line-Tools#rez-env) tool: +using the [rez-env](https://rez.readthedocs.io/en/stable/commands/rez-env.html) tool: ]$ rez-env requests-2.2+ python-2.6 'pymongo-0+<2.7' diff --git a/docs/source/getting_started.rst b/docs/source/getting_started.rst new file mode 100644 index 000000000..2872fe4fc --- /dev/null +++ b/docs/source/getting_started.rst @@ -0,0 +1,182 @@ +=============== +Getting started +=============== + +Essential packages +================== + +After installation, you need to create some essential Rez packages. The :ref:`rez-bind` +tool creates Rez packages that reference software already installed on your system. +Use the :option:`--quickstart ` argument to bind a set of standard packages: + +.. note:: + You may require administrative privileges for some of them + +.. code-block:: text + + ]$ rez-bind --quickstart + Binding platform into /home/ajohns/packages... + Binding arch into /home/ajohns/packages... + Binding os into /home/ajohns/packages... + Binding python into /home/ajohns/packages... + Binding rez into /home/ajohns/packages... + Binding rezgui into /home/ajohns/packages... + Binding setuptools into /home/ajohns/packages... + Binding pip into /home/ajohns/packages... + + Successfully converted the following software found on the current system into Rez packages: + + PACKAGE URI + ------- --- + arch /home/ajohns/packages/arch/x86_64/package.py + os /home/ajohns/packages/os/osx-10.11.5/package.py + pip /home/ajohns/packages/pip/8.0.2/package.py + platform /home/ajohns/packages/platform/osx/package.py + python /home/ajohns/packages/python/2.7.11/package.py + rez /home/ajohns/packages/rez/2.0.rc1.44/package.py + rezgui /home/ajohns/packages/rezgui/2.0.rc1.44/package.py + setuptools /home/ajohns/packages/setuptools/19.4/package.py + +Now you should be able to create an environment containing Python. Try this: + +.. code-block:: text + + ]$ rez-env python -- which python + /home/ajohns/packages/python-2.7.8/platform-linux/arch-x86_64/os-Ubuntu-12.04/bin/python + +Building packages +================= + +Before building your first rez package, ensure that: + +* The directory ``$HOME/packages`` exists and is writable; +* The `cmake `_ tool is available. + + .. note:: + CMake is not a hard requirement to build packages in rez. This example just + happens to use CMake, but you can use anything you want in your packages. + +The :ref:`rez-build` tool is used to build packages and install them locally (typically to ``$HOME/packages``). + +Once you've done that, you can use them via :ref:`rez-env`, just like any other package: + +.. code-block:: text + + ]$ cd example_packages/hello_world + ]$ rez-build --install + + -------------------------------------------------------------------------------- + Building hello_world-1.0.0... + -------------------------------------------------------------------------------- + Resolving build environment: python + resolved by ajohns@workstation.local, on Sun Jul 31 14:39:33 2016, using Rez v2.0.rc1.44 + + requested packages: + python + ~platform==osx (implicit) + ~arch==x86_64 (implicit) + ~os==osx-10.11.5 (implicit) + + resolved packages: + arch-x86_64 /home/ajohns/packages/arch/x86_64 (local) + os-osx-10.11.5 /home/ajohns/packages/os/osx-10.11.5 (local) + platform-osx /home/ajohns/packages/platform/osx (local) + python-2.7.11 /home/ajohns/packages/python/2.7.11/platform-osx/arch-x86_64/os-osx-10.11.5 (local) + + Invoking cmake build system... + Executing: /usr/local/bin/cmake -d /home/ajohns/workspace/rez/example_packages/hello_world -Wno-dev -DCMAKE_ECLIPSE_GENERATE_SOURCE_PROJECT=TRUE -D_ECLIPSE_VERSION=4.3 --no-warn-unused-cli -DCMAKE_INSTALL_PREFIX=/home/ajohns/packages/hello_world/1.0.0 -DCMAKE_MODULE_PATH=${CMAKE_MODULE_PATH} -DCMAKE_BUILD_TYPE=Release -DREZ_BUILD_TYPE=local -DREZ_BUILD_INSTALL=1 -G Unix Makefiles + Not searching for unused variables given on the command line. + -- Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE) + -- Configuring done + -- Generating done + -- Build files have been written to: /home/ajohns/workspace/rez/example_packages/hello_world/build + + Executing: make -j4 + [100%] Built target py + + Executing: make -j4 install + [100%] Built target py + Install the project... + -- Install configuration: "Release" + -- Installing: /home/ajohns/packages/hello_world/1.0.0/./python/hello_world.py + -- Installing: /home/ajohns/packages/hello_world/1.0.0/./python/hello_world.pyc + -- Installing: /home/ajohns/packages/hello_world/1.0.0/./bin/hello + + All 1 build(s) were successful. + +You have just built your first package, and installed it to the :data:`local package path `, +which defaults to (and is usually kept as) ``$HOME/packages``. + +For more information, see :doc:`building_packages`. + +Testing your package +==================== + +You can use the :ref:`rez-env` tool to request a configured environment containing your package: + +.. code-block:: text + + ]$ rez-env hello_world + + You are now in a rez-configured environment. + + resolved by ajohns@workstation.local, on Sun Jul 31 14:43:54 2016, using Rez v2.0.rc1.44 + + requested packages: + hello_world + ~platform==osx (implicit) + ~arch==x86_64 (implicit) + ~os==osx-10.11.5 (implicit) + + resolved packages: + arch-x86_64 /home/ajohns/packages/arch/x86_64 (local) + hello_world-1.0.0 /home/ajohns/packages/hello_world/1.0.0 (local) + os-osx-10.11.5 /home/ajohns/packages/os/osx-10.11.5 (local) + platform-osx /home/ajohns/packages/platform/osx (local) + python-2.7.11 /home/ajohns/packages/python/2.7.11/platform-osx/arch-x86_64/os-osx-10.11.5 (local) + + > ]$ █ + +Now you are within the configured environment. The caret (``>``) prefixed to your prompt is a visual +cue telling you that you're within a rez-configured subshell. Rez does not update the currect environment, +instead it configures a subshell and puts you within it. + +Now you can run the ``hello`` tool in our ``hello_world`` package: + +.. code-block:: text + + > ]$ hello + Hello world! + +If you're within a rez shell, and you forget what packages are currently available or want to see +the list again, you can use the :ref:`rez-context` tool. It prints the same information you see when you +initially created the environment: + +.. code-block:: text + + > ]$ rez-context + resolved by ajohns@workstation.local, on Sun Jul 31 14:43:54 2016, using Rez v2.0.rc1.44 + + requested packages: + hello_world + ~platform==osx (implicit) + ~arch==x86_64 (implicit) + ~os==osx-10.11.5 (implicit) + + resolved packages: + arch-x86_64 /home/ajohns/packages/arch/x86_64 (local) + hello_world-1.0.0 /home/ajohns/packages/hello_world/1.0.0 (local) + os-osx-10.11.5 /home/ajohns/packages/os/osx-10.11.5 (local) + platform-osx /home/ajohns/packages/platform/osx (local) + python-2.7.11 /home/ajohns/packages/python/2.7.11/platform-osx/arch-x86_64/os-osx-10.11.5 (local) + +To exit the configured environment, simply exist the shell using the ``exit`` command or :kbd:`Control-d`: + +You can also create a configured environment and run a command inside of it, with a single command. +When you use this form, the shell is immediately exited after the command runs: + +.. code-block:: text + + ]$ rez-env hello_world -- hello + Hello world! + ]$ █ diff --git a/docs/source/index.rst b/docs/source/index.rst index da4e133f5..278be5539 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -12,6 +12,7 @@ Welcome to rez's documentation! :hidden: installation + getting_started basic_concepts building_packages context diff --git a/docs/source/installation.rst b/docs/source/installation.rst index 87c085a2b..a9ef7e622 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -5,6 +5,9 @@ Installation Installation Script =================== +.. warning:: + The install script only supports Python 3.7+. + To install rez, download the source. Then from the root directory, run:: ]$ python ./install.py @@ -34,6 +37,11 @@ It is possible to install rez with pip, like so:: ]$ pip install rez +.. note:: + While the ``install.py`` script only supports Python 3.7+, + rez is still usable with Python 2.7 when installed via pip. In other words, the API + is fully compatible with Python 2 and 3. + However, this comes with a caveat. Rez command line tools **are not guaranteed to work correctly** once inside a rez environment (ie after using the :ref:`rez-env` command). The reasons are given in the next section. diff --git a/docs/source/package_definition.rst b/docs/source/package_definition.rst index 566ff4278..8223ccdbf 100644 --- a/docs/source/package_definition.rst +++ b/docs/source/package_definition.rst @@ -719,7 +719,7 @@ the data type, and includes a code snippet. .. code-block:: python - help = "https://github.com/__GITHUB_REPO__/wiki" + help = "https://example.com" .. code-block:: diff --git a/src/rez/bundle_context.py b/src/rez/bundle_context.py index f300c6514..a12c39025 100644 --- a/src/rez/bundle_context.py +++ b/src/rez/bundle_context.py @@ -41,7 +41,9 @@ def bundle_context(context, dest_dir, force=False, skip_non_relocatable=False, quiet (bool): Suppress all output patch_libs (bool): If True, modify libs and executables within the bundle to patch any references to external packages back to their - equivalents within the bundle. See the wiki for more details on this. + equivalents within the bundle. See + https://rez.readthedocs.io/en/stable/context_bundles.html#patching-libraries + for more details on this. verbose (bool): Verbose mode (quiet will override) """ bundler = _ContextBundler( diff --git a/src/rez/cli/_entry_points.py b/src/rez/cli/_entry_points.py index 8841932d9..8590e9d72 100644 --- a/src/rez/cli/_entry_points.py +++ b/src/rez/cli/_entry_points.py @@ -55,7 +55,7 @@ def check_production_install(): sys.stderr.write( "Pip-based rez installation detected. Please be aware that rez command " "line tools are not guaranteed to function correctly in this case. See " - "https://github.com/AcademySoftwareFoundation/rez/wiki/Installation#why-not-pip-for-production " + "https://rez.readthedocs.io/en/stable/installation.html#why-not-pip-for-production " " for futher details.\n" ) diff --git a/src/rez/package_cache.py b/src/rez/package_cache.py index 663e61330..88c2fba67 100644 --- a/src/rez/package_cache.py +++ b/src/rez/package_cache.py @@ -205,7 +205,7 @@ def add_variant(self, variant, force=False): # Package belongs to a temp repo (this occurs when a package is # tested on pre_build/pre_release - see - # https://github.com/AcademySoftwareFoundation/rez/wiki/Package-Definition-Guide#tests) + # https://rez.readthedocs.io/en/stable/package_definition.html#tests) # if package.repository.name() == "filesystem" and \ package.repository.location.startswith(config.tmpdir + os.sep): diff --git a/src/rez/rezconfig.py b/src/rez/rezconfig.py index 81f7347f1..f66cb467f 100644 --- a/src/rez/rezconfig.py +++ b/src/rez/rezconfig.py @@ -40,7 +40,7 @@ (based on Python's os.path.sep). So for Linux paths, / should be used. On Windows \ (unescaped) should be used. -Note: The comments in this file are extracted and turned into Wiki content. Pay +Note: The comments in this file are extracted and turned into documentation. Pay attention to the comment formatting and follow the existing style closely. """ diff --git a/src/rez/tests/util.py b/src/rez/tests/util.py index b851e3c38..fb0f40d9a 100644 --- a/src/rez/tests/util.py +++ b/src/rez/tests/util.py @@ -265,7 +265,7 @@ def wrapper(self, *args, **kwargs): else: self.skipTest( "Must be run via 'rez-selftest' tool, see " - "https://github.com/AcademySoftwareFoundation/rez/wiki/Installation#installation-script" + "https://rez.readthedocs.io/en/stable/installation.html#installation-script" ) return wrapper return decorator diff --git a/wiki/README.md b/wiki/README.md deleted file mode 100644 index 69aeab1f8..000000000 --- a/wiki/README.md +++ /dev/null @@ -1,16 +0,0 @@ -# rez-wiki - -This directory holds the content used to produce the Rez Wiki documentation -found [here](https://github.com/AcademySoftwareFoundation/rez/wiki). The wiki is updated by the -`Wiki` Github workflow, on release. - -You should include relevant wiki updates with your code PRs. - -## Testing - -To test wiki updates locally: - -* Make your changes to any pages/* or media/* files; -* Run `python ./generate-wiki.py` -* View the resulting content in the `out` directory using your markdown viewer - of choice (we suggest [grip](https://github.com/joeyespo/grip)). diff --git a/wiki/generate-wiki.py b/wiki/generate-wiki.py deleted file mode 100644 index c9651c7cc..000000000 --- a/wiki/generate-wiki.py +++ /dev/null @@ -1,435 +0,0 @@ -""" -Script to generate wiki content. -""" -from __future__ import absolute_import -from __future__ import division -from __future__ import print_function -from __future__ import unicode_literals - -import argparse -from collections import defaultdict -import errno -from io import open -import os -import re -import subprocess -import shutil -import sys - - -# py3.7+ only -if sys.version_info[:2] < (3, 7): - print("update-wiki.py: use python-3.7 or greater", file=sys.stderr) - sys.exit(1) - - -THIS_FILE = os.path.abspath(__file__) -THIS_DIR = os.path.dirname(THIS_FILE) -REZ_SOURCE_DIR = os.getenv("REZ_SOURCE_DIR", os.path.dirname(THIS_DIR)) - -OUT_DIR = "out" -GITHUB_REPO = "unknown/rez" -GITHUB_BRANCH = "main" -GITHUB_WORKFLOW = "Wiki" - - -def add_toc(txt): - """Add github-style ToC to start of md content. - """ - lines = txt.split('\n') - toc_lines = [] - mindepth = None - - for line in lines: - if not line.startswith('#'): - continue - - parts = line.split() - - hashblock = parts[0] - if set(hashblock) != set(["#"]): - continue - - depth = len(hashblock) - if mindepth is None: - mindepth = depth - depth -= mindepth - - toc_lines.append("%s- [%s](#%s)" % ( - ' ' * 4 * depth, - ' '.join(parts[1:]), - '-'.join(x.lower() for x in parts[1:]) - )) - - if not toc_lines: - return txt - - return '\n'.join(toc_lines) + "\n\n" + txt - - -def creating_configuring_rez_md(txt): - lines = txt.split('\n') - - start = None - end = None - for i, line in enumerate(lines): - if "__DOC_START__" in line: - start = i - elif "__DOC_END__" in line: - end = i - - lines = lines[start:end + 1] - assign_regex = re.compile("^([a-z0-9_]+) =") - settings = {} - - # parse out settings and their comment - for i, line in enumerate(lines): - m = assign_regex.match(line) - if not m: - continue - - start_defn = i - end_defn = i - while lines[end_defn].strip() and not lines[end_defn].startswith('#'): - end_defn += 1 - - defn_lines = lines[start_defn:end_defn] - defn_lines = [(" " + x) for x in defn_lines] # turn into code block - defn = '\n'.join(defn_lines) - - end_comment = i - while not lines[end_comment].startswith('#'): - end_comment -= 1 - - start_comment = end_comment - while lines[start_comment].startswith('#'): - start_comment -= 1 - start_comment += 1 - - comments = lines[start_comment:end_comment + 1] - comments = [x[2:] for x in comments] # drop leading '# ' - comment = '\n'.join(comments) - - varname = m.groups()[0] - settings[varname] = (defn, comment) - - # generate md text - md = [] - - for varname, (defn, comment) in sorted(settings.items()): - md.append("### %s" % varname) - md.append("") - md.append(defn) - md.append("") - md.append(comment) - md.append("") - - md = '\n'.join(md) - return md - - -def create_contributors_md(src_path): - # Make sure aliases KEY is fully lowercase to match correctly! - aliases = { - "allan.johns": "Allan Johns", - "allan johns": "Allan Johns", - "ajohns": "Allan Johns", - "nerdvegas": "Allan Johns", - "nerdvegas@gmail.com": "Allan Johns", - "method": "Allan Johns", - "rachel johns": "Allan Johns", - "root": "Allan Johns", - "(no author)": "Allan Johns", - - "mylene pepe": "Mylene Pepe", - "michael.morehouse": "Michael Morehouse", - "phunter.nz": "Philip Hunter", - "joe yu": "Joseph Yu", - "j0yu": "Joseph Yu", - "fpiparo": "Fabio Piparo" - } - out = subprocess.check_output( - ["git", "shortlog", "-sn", "HEAD"], - encoding="utf-8", - cwd=src_path, - ) - contributors = defaultdict(int) - regex = re.compile( - r'^\s*(?P\d+)\s+(?P.+)\s*$', - flags=re.MULTILINE | re.UNICODE - ) - - for match in regex.finditer(out): - author = match.group('author') - author_html = "%s
" % aliases.get(author.lower(), author) - contributors[author_html] += int(match.group('commits')) - - return '\n'.join( - author_html for author_html, commit_count in - sorted(contributors.items(), key=lambda x: x[1], reverse=True) - ) - - -def process_markdown_files(): - pagespath = os.path.join(THIS_DIR, "pages") - user, repo_name = GITHUB_REPO.split('/') - processed_files = {} - - src_path = REZ_SOURCE_DIR - if src_path is None: - print( - "Must provide REZ_SOURCE_DIR which points at root of " - "rez source clone", file=sys.stderr, - ) - sys.exit(1) - - def apply_replacements(filename, replacements=None): - srcfile = os.path.join(pagespath, filename) - - with open(srcfile, encoding='utf-8') as f: - txt = f.read() - - # add standard replacements - repls = { - "__GITHUB_REPO__": GITHUB_REPO, - "__GITHUB_USER__": user, - "__GITHUB_BRANCH__": GITHUB_BRANCH, - "__REPO_NAME__": repo_name - } - - repls.update(replacements or {}) - - for src_txt, repl_txt in repls.items(): - txt = txt.replace(src_txt, repl_txt) - - return txt - - # generate Configuring-Rez.md - filepath = os.path.join(src_path, "src", "rez", "rezconfig.py") - with open(filepath) as f: - txt = f.read() - - processed_files["Configuring-Rez.md"] = apply_replacements( - "Configuring-Rez.md", - { - "__REZCONFIG_MD__": creating_configuring_rez_md(txt) - } - ) - - # generate Credits.md - md = create_contributors_md(src_path) - processed_files["Credits.md"] = apply_replacements( - "Credits.md", - { - "__CONTRIBUTORS_MD__": md - } - ) - - # generate Command-Line-Tools.md - processed_files["Command-Line-Tools.md"] = apply_replacements( - "Command-Line-Tools.md", - { - "__GENERATED_MD__": create_clitools_markdown(src_path) - } - ) - - # all other markdown files - for name in os.listdir(pagespath): - if name not in processed_files: - processed_files[name] = apply_replacements(name) - - # iterate over every file, add a ToC, and write it out - for name, txt in processed_files.items(): - destfile = os.path.join(OUT_DIR, name) - txt = add_toc(txt) - with open(destfile, 'w') as out: - out.write(txt) - - -def create_clitools_markdown(src_path): - """Generate the formatted markdown for each rez cli tool. - - Hot-import rez cli library to get parsers. - - Args: - src_path (str): Full path to the rez source code repository. - - Returns: - str: Generated markdown text. - """ - sys.path.insert(0, os.path.join(src_path, "src")) - try: - from rez.cli._main import setup_parser - from rez.cli._util import LazySubParsersAction - - main_parser = setup_parser() - command_help = [] - parsers = [main_parser] - for action in main_parser._actions: - if isinstance(action, LazySubParsersAction): - parsers += action.choices.values() - - for arg_parser in parsers: - arg_parser.formatter_class = MarkdownHelpFormatter - command_help.append(arg_parser.format_help()) - finally: - sys.path.pop(0) - - return "\n\n\n".join(command_help) - - -class MarkdownHelpFormatter(argparse.ArgumentDefaultsHelpFormatter): - - def _format_usage(self, usage, actions, groups, prefix): - """Override to produce markdown title and code block formatting. - - Return: - str: Markdown title and code block formatted usage. - """ - prefix_was_none = prefix is None - if prefix_was_none: - prefix = "# {self._prog}\n```\n".format(self=self) - - super_format_usage = super(MarkdownHelpFormatter, self)._format_usage - formatted_usage = super_format_usage(usage, actions, groups, prefix) - - if prefix_was_none: - # Fix extra spaces calculated from old "usage: rez ..." prompt - extra_spaces = "{newline:<{count}} ".format( - newline="\n", - count=len("usage: {self._prog}".format(self=self)) - ) - formatted_usage = formatted_usage[:-1] + "```\n" - formatted_usage = formatted_usage.replace(extra_spaces, "\n") - - return formatted_usage - - def remap_heading(self, heading): - """Remap argparse headings to shorter, markdown formatted headings. - - Args: - heading (str): Original heading to remap and format. - - Returns: - str: Remapped and formatted heading, if any. - """ - if heading == "optional arguments": - return "\n**Flags**\n" - elif heading == "positional arguments": - return "" if self._prog == "rez" else "\n**Arguments**\n" - else: - return heading - - def start_section(self, heading): - """Extend to remap optional/positional arguments headings. - - Args: - heading (str): Section heading to parse. - """ - if self.remap_heading(heading) == heading: - super(MarkdownHelpFormatter, self).start_section(heading) - else: - self._indent() - self._add_item(self.remap_heading, [heading]) - super(MarkdownHelpFormatter, self).start_section(argparse.SUPPRESS) - - def _fill_text(self, text, width, indent): - """No indent for description, keep subsequent indents. - - Return: - str: Description but without leading indents. - """ - super_fill_text = super(MarkdownHelpFormatter, self)._fill_text - return super_fill_text(text, width, indent).lstrip() - - def _format_action(self, action): - """Extend to format rez sub commands as table of links. - - Returns: - str: Formatted help text for an action. - """ - backup_width = self._width - if self._prog == "rez" and action.nargs is None: - self._width = 2000 # Temporary thicc width to avoid wrapping - - try: - super_format = super(MarkdownHelpFormatter, self)._format_action - help_text = super_format(action) - finally: - self._width = backup_width - - if self._prog == "rez": - # Sub commands, format them with links - if action.nargs is None: - help_text = re.sub( - r'^\s+(\S+)(\s+)', - r'[\1](#rez-\1)\2| ', - help_text - ) - - # Sub commands heading, format as table heading - elif action.metavar == "COMMAND": - help_text = re.sub( - r'^\s+COMMAND', - "`COMMAND` | Description\n----|----", - help_text - ) - - return help_text - - -class UpdateWikiParser(argparse.ArgumentParser): - def __init__(self, **kwargs): - super(UpdateWikiParser, self).__init__(**kwargs) - - self.add_argument( - "--github-repo", - default=GITHUB_REPO, - dest="repo", - help="Url to GitHub repository without leading github.com/" - ) - self.add_argument( - "--github-branch", - default=GITHUB_BRANCH, - dest="branch", - help="Name of git branch that is generating the Wiki" - ) - self.add_argument( - "--github-workflow", - default=GITHUB_WORKFLOW, - dest="workflow", - help="Name of GitHub workflow that is generating the Wiki" - ) - self.add_argument( - "--out", - default=OUT_DIR, - dest="dir", - help="Output dir" - ) - - -if __name__ == "__main__": - # Quick check for "git" and throw meaningful error message - try: - subprocess.check_call(["git", "--version"]) - except OSError as error: - if error.errno == errno.ENOENT: - raise OSError(errno.ENOENT, '"git" needed but not found in PATH') - raise - - args = UpdateWikiParser().parse_args() - GITHUB_REPO = args.repo - GITHUB_BRANCH = args.branch - GITHUB_WORKFLOW = args.workflow - OUT_DIR = os.path.abspath(args.dir) - - if not os.path.exists(OUT_DIR): - os.makedirs(OUT_DIR) - - shutil.copytree( - os.path.join(THIS_DIR, 'media'), - os.path.join(OUT_DIR, 'media'), - ) - - os.chdir(OUT_DIR) - process_markdown_files() diff --git a/wiki/media/icons/info.png b/wiki/media/icons/info.png deleted file mode 100644 index 954f5e81c..000000000 Binary files a/wiki/media/icons/info.png and /dev/null differ diff --git a/wiki/media/icons/new.png b/wiki/media/icons/new.png deleted file mode 100644 index f44a9428e..000000000 Binary files a/wiki/media/icons/new.png and /dev/null differ diff --git a/wiki/media/icons/under_construction.png b/wiki/media/icons/under_construction.png deleted file mode 100644 index 24deabc5d..000000000 Binary files a/wiki/media/icons/under_construction.png and /dev/null differ diff --git a/wiki/media/icons/warning.png b/wiki/media/icons/warning.png deleted file mode 100644 index 91677b5f1..000000000 Binary files a/wiki/media/icons/warning.png and /dev/null differ diff --git a/wiki/media/other_pkg_mgr.png b/wiki/media/other_pkg_mgr.png deleted file mode 100644 index f3891daaf..000000000 Binary files a/wiki/media/other_pkg_mgr.png and /dev/null differ diff --git a/wiki/media/pkg_path_anatomy.png b/wiki/media/pkg_path_anatomy.png deleted file mode 100644 index d7b660495..000000000 Binary files a/wiki/media/pkg_path_anatomy.png and /dev/null differ diff --git a/wiki/media/rez_banner_128.png b/wiki/media/rez_banner_128.png deleted file mode 100644 index bc7769aec..000000000 Binary files a/wiki/media/rez_banner_128.png and /dev/null differ diff --git a/wiki/media/rez_deps_simple_eg.png b/wiki/media/rez_deps_simple_eg.png deleted file mode 100644 index 59e2ba485..000000000 Binary files a/wiki/media/rez_deps_simple_eg.png and /dev/null differ diff --git a/wiki/media/rez_env.png b/wiki/media/rez_env.png deleted file mode 100644 index ef6e19c0d..000000000 Binary files a/wiki/media/rez_env.png and /dev/null differ diff --git a/wiki/media/rez_pkg_mgr.png b/wiki/media/rez_pkg_mgr.png deleted file mode 100644 index 87e1b162d..000000000 Binary files a/wiki/media/rez_pkg_mgr.png and /dev/null differ diff --git a/wiki/pages/Basic-Concepts.md b/wiki/pages/Basic-Concepts.md deleted file mode 100644 index 95a8aa977..000000000 --- a/wiki/pages/Basic-Concepts.md +++ /dev/null @@ -1,369 +0,0 @@ -## Overview - -Rez manages packages. You request a list of packages from rez, and it resolves this request, if -possible. If the resolution is not possible, the system supplies you with the relevant information -to determine why this is so. You typically want to resolve a list of packages because you want to -create an environment in which you can use them in combination, without conflicts occurring. A -conflict occurs when there is a request for two or more different versions of the same package - a -version clash. - -Rez lets you describe the environment you want in a natural way. For example, you can say: -“I want an environment with...” - -* “...the latest version of houdini” -* “...maya-2009.1” -* “...the latest rv and the latest maya and houdini-11.something” -* “...rv-3.something or greater” -* “...the latest houdini which works with boost-1.37.0” -* “...PyQt-2.2 or greater, but less than PyQt-4.5.3” - -In many examples in this documentation we will use the -[rez-env](Command-Line-Tools#rez-env) command line tool. This tool takes a list of package -requests and creates the resulting configured environment. It places you in a subshell - simply -exit the shell to return to a non-configured environment. - -## Versions - -Rez version numbers are alphanumeric - they support any combination of numbers, letters and -underscores. A version number is a set of *tokens*, separated by either dot or dash. For example, -here is a list of valid package version numbers: - -* 1 -* 1.0.0 -* 3.2.build_13 -* 4.rc1 -* 10a-5 - -Version number tokens follow a strict ordering schema and are case sensitive. Underscore is the -smallest character, followed by letters (a-z and A-Z), followed by numbers. The ordering rules are -like so: - -* Underscore before everything else; -* Letters alphabetical, and before numbers; -* Lowercase letters before uppercase; -* Zero-padded numbers before equivalent non-padded (or less padded) number ('01' is < '1'); -* If a token contains a combination of numbers and letters, it is internally split into groups -containing only numbers or only letters, and the resulting list is compared using the same rules -as above. - -The following table shows some example version token comparisons: - -smaller token | larger token ---------------|------------- -0 | 1 -a | b -a | A -a | 3 -_5 | 2 -ham | hamster -alpha | beta -alpha | bob -02 | 2 -002 | 02 -13 | 043 -3 | 3a -beta3 | 3beta - -Versions are compared by their token lists. The token delimiter (usually dot, but can also be dash) -is ignored for comparison purposes - thus the versions '1.0.0' and '1-0.0' are equivalent. If two -versions share the same token list prefix, the longer version is greater - thus '1.0.0' is a higher -version than '1.0'. - -Note that no special importance is given to specific characters or letters in Rez version numbers - -the terms 'alpha' and 'beta' for example have no special meaning. Similarly, the number of tokens in -a version number doesn't matter, you can have as many as you like. While you are encouraged to use -semantic versioning (see here), it is not enforced. - -## Packages - -A *package* is a versioned piece of software, that may have dependencies on other packages. Packages -are self-contained - they have a single package definition file (typically *package.py*), which -describes everything we need to know about the package in order to use it. Rez manages any kind of -package, whether it be a python package, compiled package, or simply build code or configuration -data. - -Here is an example package definition file (see [here](Package-Definition-Guide) for further details -of each attribute): - - name = "foo" - - version = "1.0.0" - - description = "Something that does foo-like things." - - requires = [ - "python-2.6", - "utils-1.1+<2" - ] - - tools = [ - "fooify" - ] - - def commands(): - env.PYTHONPATH.append("{root}/python") - env.PATH.append("{root}/bin") - -The *requires* section defines the requirements of the package. The *commands* section describes -what happens when this package is added to an environment. Here, the *bin* directory in the package -installation is appended to *PATH*, and similarly the *python* subdirectory is appended to -*PYTHONPATH*. - -## Package Repositories - -Packages are installed into package repositories. A package repository is a directory on disk, with -packages and their versions laid out in a known structure underneath. Going on with our (foo, bah, -eek) example, here is how the package repository might look: - - /packages/inhouse/foo/1.1 - /1.2 - /1.3 - /packages/inhouse/bah/2 - /3 - /4 - /packages/inhouse/eek/2.5 - /2.6 - /2.7 - - # more detailed example of foo-1.1 - /packages/inhouse/foo/1.1/package.py - /python/ - /bin/ - -Here we have a package repository under the directory */packages/inhouse*. The actual package content -(files, executables etc) is installed into each leaf-node version directory, as shown for *foo-1.1*. -The package definition file, in this case *package.py*, is always stored at the root of the package - -right under the version directory for that package. - -Rez only requires that the package's *package.py* file is at the root of the package installation. The -layout of the rest of the package - for example, the *python* and *bin* directories - is completely -up to the package's own build to determine. You should expect to see a package's *commands* section -match up with its installation though. For example, notice how the path for foo's python files and -binaries match what its package commands specified from earlier - "{root}/python" and "{root}/bin" -will expand to these paths respectively. - -## Package Search Path - -Rez finds packages using a search path in much the same way that python finds python modules using -PYTHONPATH. You can find out what the search path is, using the rez command line tool rez-config -(which you can also use to find any other rez setting): - - ]$ rez-config packages_path - - /home/ajohns/packages - - /packages/inhouse - - /packages/vendor - -If the same package appears in two or more repositories on the search path, the earlier package is -used in preference. This happens at the version level - an earlier package "foo-1.0.0" will hide a -later package "foo-1.0.0", but not "foo-1.2.0". - -The example search path shown is a typical setting. There are some central repositories later in the -search path, where packages are released to so everyone can use them. But there is also a local -package path at the front of the search path. This is where packages go that are being locally -developed by a user. Having this at the start of the searchpath allows developers to resolve -environments that pull in test packages in preference to released ones, so they can test a package -before releasing it for general use. - -You can change the packages search path in several ways. A common way is to set the REZ_PACKAGES_PATH -environment variable; see [Configuring Rez](Configuring-Rez) for more configuration options. - -## Package Commands - -The *commands* section of the package definition determines how the environment is configured in -order to use it. It is a python function, but note that if any imports are used, they must appear -within the body of this function. - -Consider this commands example: - - def commands(): - env.PYTHONPATH.append("{root}/python") - env.PATH.append("{root}/bin") - -This is a typical example, where a package adds its source path to *PYTHONPATH*, and its tools to -*PATH*. See [here](Package-Commands) for details on what can be done within the *commands* section, -as well as details on what order package commands are executed in. - -## Package Requests - -A *package request* is a string with a special syntax which matches a number of possible package -versions. You use package requests in the requires section of a package definition file, and also -when creating your own configured environment directly using tools such as *rez-env*. - -For example, here is a request (using the *rez-env* tool) to create an environment containing -*python* version 2.6 or greater, and *my_py_utils* version 5.4 or greater, but less than 6: - - ]$ rez-env 'python-2.6+' 'my_py_utils-5.4+<6' - -Here are some example package requests: - -package request | description | example versions within request -----------------|-------------------------------------|-------------------------------- -foo | Any version of foo. | foo-1, foo-0.4, foo-5.0, foo-2.0.alpha -foo-1 | Any version of foo-1[.x.x...x]. | foo-1, foo-1.0, foo-1.2.3 -foo-1+ | foo-1 or greater. | foo-1, foo-1.0, foo-1.2.3, foo-7.0.0 -foo-1.2+<2 | foo-1.2 or greater, but less than 2 | foo-1.2.0, foo-1.6.4, foo-1.99 -foo<2 | Any version of foo less than 2 | foo-1, foo-1.0.4 -foo==2.0.0 | Only version 2.0.0 exactly | foo-2.0.0 -foo-1.3\|5+ | OR'd requests | foo-1.3.0, foo-6.0.0 - -### The Conflict Operator - -The '!' operator is called the *conflict* operator, and is used to define an incompatibility -between packages, or to specify that you do *not* want a package version present. For example, -consider the command: - - ]$ rez-env maya_utils '!maya-2015.6' - -This specifies that you require any version of *maya_utils*, but that any version of *maya* within -2015.6 (and this includes 2015.6.1 and so on) is not acceptable. - -### Weak References - -The '~' operator is called the *weak reference* operator. It forces a package version to be within -the specified range if present, but does not actually require the package. For example, consider -the command: - - ]$ rez-env foo '~nuke-9.rc2' - -This request may or may not pull in the *nuke* package, depending on the requirements of *foo*; -however, if nuke *is* present, it must be within the version 9.rc2. - -Weak references are useful in certain cases. For example, applications such as *nuke* and *maya* -sometimes ship with their own version of *python*. Their rez packages don't have a requirement on -*python* (they have their own embedded version already). However often other python libraries are -used both inside and outside of these applications, and those packages *do* have a python -requirement. So, to make sure that they're using a compatible python version when used within the -app, the app may define a *weak package reference* to their relevant python version, like so: - - # in maya's package.py - requires = [ - "~python-2.7.3" - ] - -This example ensures that any package that uses python, will use the version compatible with maya -when maya is present in the environment. - -## Implicit Packages - -The *implicit packages* are a list of package requests that are automatically added to every rez -request (for example, when you use *rez-env*). They are set by the configuration setting -*implicit_packages*. The default setting looks like so: - - implicit_packages = [ - "~platform=={system.platform}", - "~arch=={system.arch}", - "~os=={system.os}", - ] - -Rez models the current system - the platform, architecture and operating systems - as packages -themselves. The default implicits are a set of *weak requirements* on each of *platform*, *arch* and -*os*. This ensures that if any platform-dependent package is requested, the platform, architecture -and/or operating system it depends on, matches the current system. - -The list of implicits that were used in a request are printed by *rez-env* when you enter the newly -configured subshell, and are also printed by the *rez-context* tool. - -## Dependency Resolving - -Rez contains a solving algorithm that takes a *request* - a list of package requests - and produces -a *resolve* - a final list of packages that satisfy the request. The algorithm avoids version -conflicts - two or more different versions of the same package at once. - -When you submit a request to rez, it finds a solution for that request that aims to give you the -latest possible version of each package. If this is not possible, it will give you the next latest -version, and so on. - -Consider the following example (the arrows indicate dependencies): - -

- - -

- -Here we have three packages - 'foo', 'bah' and 'eek', where both foo and bah have dependencies on -eek. For example, package "bah-4" might have a package definition file that looks something like -this (some entries skipped for succinctness): - - name = "bah" - - version = "4" - - requires = [ - "eek-2.6" - ] - -A request for "foo-1.3" is going to result in the resolve ("foo-1.3", "eek-2.7"). A request for -"foo" will give the same result - we are asking for "any version of foo", but rez will prefer the -latest. However, if we request ("foo", "bah"), we are not going to get the latest of both - they -depend on different versions of eek, and that would cause a version conflict. Instead, our resolve -is going to be ("foo-1.2", "bah-4", "eek-2.6"). Rez has given you the latest possible versions of -packages, that do not cause a conflict. - -Sometimes your request is impossible to fulfill. For example, the request ("foo-1.3", "bah-4") is -not possible. In this case, the resolve will fail, and rez will inform you of the conflict. - -## Resolving An Environment - -A user can create a resolved environment using the command line tool *rez-env* (also via the API - -practically everything in rez can be done in python). When you create the environment, the current -environment is not changed - you are placed into a sub-shell instead. Here is an example of using -rez-env, assuming that the package repository is from our earlier (foo, bah, eek) example: - - ]$ rez-env foo bah - - You are now in a rez-configured environment. - - resolved by ajohns@14jun01.methodstudios.com, on Wed Oct 22 12:44:00 2014, - using Rez v2.0.rc1.10 - - requested packages: - foo - bah - - resolved packages: - eek-2.6 /packages/inhouse/eek/2.6 - foo-1.2 /packages/inhouse/foo/1.2 - bah-4 /packages/inhouse/bah/4 - - > ]$ █ - -The output of rez-env shows the original request, along with the matching resolve. It's the resolve -that tells you what actual package versions are present in the newly resolved environment. Notice -the '**>**' character in the prompt - this is a visual cue telling you that you have been placed -into a rez-resolved environment. - -### Putting It All Together - -Let's go through what happens when an environment is resolved, using a new (and slightly more -realistic) example. -Let us assume that the following packages are available: - -* maya-2014.sp2; -* nuke-8.0v3; -* 3 versions of a maya plugin "mplugin"; -* 2 versions of a nuke plugin "nplugin"; -* 3 versions of a common base library "lib". - -The following diagram shows what happens when the command *"rez-env mplugin-1.3.0"* is run: - - -

- - -

- -The digram shows the following operations occurring: - -* Rez takes the user's request, and runs it through the dependency solver. The solver reads packages - from the package repositories in order to complete the solve; -* This results in a list of resolved packages. These are the packages that are used in the - configured environment; -* The commands from each package are concatenated together; -* This total list of commands is then translated into the target shell language (in this example - that is *bash*); -* A sub-shell is created and the translated command code is sourced within this environment, - creating the final configured environment. - -The order of package command execution depends on package dependencies, and the order that packages -were requested in. See -[here](Package-Commands#order-of-command-execution) for more details. diff --git a/wiki/pages/Building-Packages.md b/wiki/pages/Building-Packages.md deleted file mode 100644 index 3bf365659..000000000 --- a/wiki/pages/Building-Packages.md +++ /dev/null @@ -1,255 +0,0 @@ -## Overview - -Rez packages can be built and locally installed using the *rez-build* tool. This -tool performs the following actions: - -* Iterates over a package's [variants](Variants); -* Constructs the build environment; -* Runs the build system within this environment. - -Each build occurs within a *build path* - this is typically either a *build* -subdirectory, or a variant-specific subdirectory under *build*. For example, a -package with two python-based variants might look like this: - - +- package.py - +- CMakeLists.txt (or other build file) - +-build - +-python-2.6 # build dir for python-2.6 variant - +-python-2.7 # build dir for python-2.6 variant - -The current working directory is set to the *build path* during a build. - -## The Build Environment - -The build environment is a rez resolved environment. Its requirement list is -constructed like so: - -* First, the package's [requires](Package-Definition-Guide#requires) list is used; -* Then, the package's [build_requires](Package-Definition-Guide#build_requires) is - appended. This is transitive - the *build_requires* of all other packages in the - environment are also used; -* Then, the package's [private_build_requires](Package-Definition-Guide#private-build_requires) - is appended (unlike *build_requires*, it is not transitive). -* Finally, if the package has variants, the current variant's requirements are - appended. - -A standard list of environment variables is also set (all prefixed with *REZ_BUILD_*) - -you can see the full list [here](Environment-Variables#resolved-build-environment-variables). - -The build system is then invoked within this environment, for each variant. - -## Build Time Dependencies - -Sometimes it is desirable for a package to depend on another package only for the purposes -of building its code, or perhaps generating documentation. Let’s use documentation as an -example - a C++ project may need to builds its docs using doxygen, but once the docs are -generated, doxygen is no longer needed. - -This is achieved by listing build-time dependencies under a -[build_requires](Package-Definition-Guide#build_requires) or -[private_build_requires](Package-Definition-Guide#private-build_requires) -section in the *package.py*. The requirements in *private_build_requires* are only used -from the package being built; requirements from *build_requires* however are transitive - build -requirements from all packages in the build environment are included. - -Some example *private_build_requires* use cases include: - -* Documentation generators such as *doxygen* or *sphinx*; -* Build utilities. For example, you may have a package called *pyqt_cmake_utils*, which - provides cmake macros for converting *ui* files to *py*; -* Statically linked libraries (since the library is linked at build time, the package - is not needed at runtime). - -An example use case of *build_requires* is a header-only (hpp) C++ library - if your own -C++ package includes this library in its own headers, other packages will also need this -library at build time (since they may include your headers, which in turn include the -hpp headers). - -## Package Communication - -Let's say I have two C++ packages, *maya_utils* and the well-known *boost* library. How -does *maya_utils* find *boost*'s header files, or library files? - -The short answer is, that is entirely up to you. Rez is not actually a build system - -it supports various build systems (as the next section describes), and it configures the -build environment, but the details of the build itself are left open for the user. -Having said that, *cmake* has been supported by rez for some time, and rez comes with a -decent amount of utility code to manage cmake builds. - -When a rez environment is configured, each required package's -[commands](Package-Definition-Guide#commands) section configures the environment for the building -package to use. When a build is occurring, a special variable -[building](Package-Commands#building) is set to *True*. Your required packages should use this -variable to communicate build information to the package being built. - -For example, our *boost* package's commands might look like so: - - def commands(): - if building: - # there is a 'FindBoost.cmake' file in this dir.. - env.CMAKE_MODULE_PATH.append("{root}/cmake") - -> [[media/icons/warning.png]] Note that _commands_ is never executed for the package actually -being built - -> if you want to run commands in that case, you can use -[pre_build_commands](Package-Commands#pre-build-commands) instead. - -A (very simple) *FindBoost.cmake* file might look like this: - - set(Boost_INCLUDE_DIRS $ENV{REZ_BOOST_ROOT}/include) - set(Boost_LIBRARY_DIRS $ENV{REZ_BOOST_ROOT}/lib) - set(Boost_LIBRARIES boost-python) - -Then, our *maya_utils* package might have a *CMakeLists.txt* file (cmake's build script) -containing: - - find_package(Boost) - include_directories(${Boost_INCLUDE_DIRS}) - link_directories(${Boost_LIBRARY_DIRS}) - target_link_libraries(maya_utils ${Boost_LIBRARIES}) - -As it happens, the [find_package](https://cmake.org/cmake/help/v3.0/command/find_package.html) -cmake macro searches the paths listed in the *CMAKE_MODULE_PATH* environment variable, -and looks for a file called *FindXXX.cmake*, where *XXX* is the name of the package (in this -case, *Boost*), which it then includes. - -## The Build System - -Rez supports multiple build systems, and new ones can be added as plugins. When a -build is invoked, the build system is detected automatically. For example, if a -*CMakeLists.txt* file is found in the package's root directory, the *cmake* build -system is used. - -### Argument Passing - -There are two ways to pass arguments to the build system. - -First, some build system plugins add extra options to the *rez-build* command directly. -For example, if you are in a CMake-based package, and you run *rez-build -h*, you will -see cmake-specific options listed, such as *--build-target*. - -Second, you can pass arguments directly to the build system - either using the -*rez-build* *--build-args* option; or listing the build system arguments after *--*. - -For example, here we explicitly define a variable in a cmake build: - - ]$ rez-build -- -DMYVAR=YES - -### Custom Build Commands - -As well as detecting the build system from build files, a package can explicitly -specify its own build commands, using the -[build_command](Package-Definition-Guide#build_command) package attribute. If present, -this takes precedence over other detected build systems. - -For example, consider the following *package.py* snippet: - - name = "nuke_utils" - - version = "1.2.3" - - build_command = "bash {root}/build.sh {install}" - -When *rez-build* is run on this package, the given *build.sh* script will be executed -with *bash*. The *{root}* string expands to the root path of the package (the same -directory containing *package.py*). The *{install}* string expands to "*install*" if -an install is occurring, or the empty string otherwise. This is useful for passing the -install target directly to the command (for example, when using *make*) rather than -relying on a build script checking the *REZ_BUILD_INSTALL* environment variable. - -> [[media/icons/warning.png]] The current working directory during a build is set -> to the *build path*, **not** to the package root directory. For this reason, you -> will typically use the *{root}* string to refer to a build script in the package's -> root directory. - -#### Passing Arguments - -You can add arguments for your build script to the *rez-build* command directly, by -providing a *parse_build_args.py* source file in the package root. Here is an example: - - # in parse_build_args.py - parser.add_argument("--foo", action="store_true", help="do some foo") - -Now if you run *rez-build -h* on this package, you will see the option listed: - - $ rez-build -h - usage: rez build [-h] [-c] [-i] [-p PATH] [--fail-graph] [-s] [--view-pre] - [--process {remote,local}] [--foo] - [--variants INDEX [INDEX ...]] [--ba ARGS] [--cba ARGS] [-v] - - Build a package from source. - - optional arguments: - ... - --foo do some foo - -The added arguments are stored into environment variables so that your build script -can access them. They are prefixed with `__PARSE_ARG_`; in our example above, the -variable `__PARSE_ARG_FOO` will be set. Booleans will be set to 0/1, and lists are -space separated, with quotes where necessary. - -#### Make Example - -Following is a very simple C++ example, showing how to use a custom build command to -build and install via *make*: - - # in package.py - build_command = "make -f {root}/Makefile {install}" - - - # in Makefile - hai: ${REZ_BUILD_SOURCE_PATH}/lib/main.cpp - g++ -o hai ${REZ_BUILD_SOURCE_PATH}/lib/main.cpp - - .PHONY: install - install: hai - mkdir -p ${REZ_BUILD_INSTALL_PATH}/bin - cp $< ${REZ_BUILD_INSTALL_PATH}/bin/hai - -## Local Package Installs - -After you've made some code changes, you presumably want to test them. You do this -by *locally installing* the package, then resolving an environment with *rez-env* -to test the package in. The cycle goes like this: - -* Make code changes; -* Run *rez-build --install* to install as a local package; -* Run *rez-env mypackage* in a separate shell. This will pick up your local package, - and your package requirements; -* Test the package. - -A local install builds and installs the package to the *local package repository*, -which is typically the directory *~/packages* (see [here](Configuring-Rez#local_packages_path)). -This directory is listed at the start of the -[package search path](Basic-Concepts#package-search-path), so when you resolve an -environment to test with, the locally installed package will be picked up first. Your -package will typically be installed to *~/packages/packagename/version*, for example -*~/packages/maya_utils/1.0.5*. If you have variants, they will be installed into subdirectories -within this install path (see [here](Variants#disk-structure) for more details). - -> [[media/icons/info.png]] You don't need to run *rez-env* after every install. If your -> package's requirements haven't changed, you can keep using the existing test environment. - -You can make sure you've picked up your local package by checking the output of the -*rez-env* call: - - ]$ rez-env sequence - - You are now in a rez-configured environment. - - resolved by ajohns@turtle, on Thu Mar 09 11:41:06 2017, using Rez v2.7.0 - - requested packages: - sequence - ~platform==linux (implicit) - ~arch==x86_64 (implicit) - ~os==Ubuntu-16.04 (implicit) - - resolved packages: - arch-x86_64 /sw/packages/arch/x86_64 - os-Ubuntu-16.04 /sw/packages/os/Ubuntu-16.04 - platform-linux /sw/packages/platform/linux - python-2.7.12 /sw/packages/python/2.7.12 - sequence-2.1.2 /home/ajohns/packages/sequence/2.1.2 (local) - -Note here that the *sequence* package is a local install, denoted by the *(local)* label. diff --git a/wiki/pages/Bundles.md b/wiki/pages/Bundles.md deleted file mode 100644 index 83f589c92..000000000 --- a/wiki/pages/Bundles.md +++ /dev/null @@ -1,81 +0,0 @@ -## Overview - -A "context bundle" is a directory containing a context (an rxt file), and a -package repository. All packages in the context are stored in the repository, -making the bundle relocatable and standalone. You can copy a bundle onto a -server for example, or into a container, and there are no external references -to shared package repositories. This is in contrast to a typical context, which -contains absolute references to one or more package repositories that are -typically on shared disk storage. - -To create a bundle via command line: - -``` -]$ rez-env foo -o foo.rxt -]$ rez-bundle foo.rxt ./mybundle - -]$ # example of running a command from the bundled context -]$ rez-env -i ./mybundle/context.rxt -- foo-tool -``` - -To create a bundle via API: - -``` ->>> from rez.bundle_context import bundle_context ->>> from rez.resolved_context import ResolvedContext ->>> ->>> c = ResolvedContext(["python-3+", "foo-1.2+<2"]) ->>> bundle_context(c, "./mybundle") -``` - - -## Structure - -A bundle directory looks like this: - -``` -.../mybundle/ - ./context.rxt - ./packages/ - -``` - -Package references in the rxt file are relative (unlike in a standard context, -where they're absolute), and this makes the bundle relocatable. - - -## Patching Libraries - -Depending on how compiled libraries and executables within a rez package were -built, it's possible that the dynamic linker will attempt to resolve them to -libraries found outside of the bundle. For example, this is possible in linux -if an elf contains an absolute searchpath in its rpath/runpath header to a -library in another package. - -Rez bundling performs a library patching step that applies various fixes to -solve this issue (use `--no-lib-patch` if you want to skip this step). This step -is platform-specific and is covered in the following sections. Note that in all -cases, references to libraries outside of the bundle will remain intact, if there -is no equivalent path found within the bundle (for example, if the reference is -to a system library not provided by a rez package). - -### Linux - -On linux, rpath/runpath headers are altered if paths are found that map to a -subdirectory within another package in the bundle. To illustrate what happens, -consider the following example, where packages from `/sw/packages` have been -bundled into the local directory `./mybundle`: - -``` -]$ # a lib in an original non-bundled package -]$ patchelf --print-rpath /sw/packages/foo/1.0.0/bin/foo -/sw/packages/bah/2.1.1/lib -]$ -]$ # the same lib in our bundle. We assume that package 'bah' is in the bundle -]$ # also, since foo links to one of its libs -]$ patchelf --print-rpath ./mybundle/packages/foo/1.0.0/bin/foo -$ORIGIN/../../../bah/2.1.1/lib -``` - -Remapped rpaths make use of the special `$ORIGIN` variable, which refers to -the directory containing the current file. diff --git a/wiki/pages/Command-Line-Tools.md b/wiki/pages/Command-Line-Tools.md deleted file mode 100644 index 434525e91..000000000 --- a/wiki/pages/Command-Line-Tools.md +++ /dev/null @@ -1 +0,0 @@ -__GENERATED_MD__ diff --git a/wiki/pages/Configuring-Rez.md b/wiki/pages/Configuring-Rez.md deleted file mode 100644 index baaa2a719..000000000 --- a/wiki/pages/Configuring-Rez.md +++ /dev/null @@ -1,118 +0,0 @@ -## Overview - -Rez has a good number of configurable settings. The default settings, and -documentation for every setting, can be found -[here](https://github.com/__GITHUB_REPO__/blob/__GITHUB_BRANCH__/src/rez/rezconfig.py). - -Settings are determined in the following way: - -- The setting is first read from the file *rezconfig.py* in the rez installation; -- The setting is then overridden if it is present in another settings file pointed at by the - *REZ_CONFIG_FILE* environment variable. This can also be a path-like variable, to read from - multiple configuration files; -- The setting is further overriden if it is present in *$HOME/.rezconfig*; -- The setting is overridden again if the environment variable *REZ_XXX* is present, where *XXX* is - the uppercase version of the setting key. For example, "image_viewer" will be overriden by - *REZ_IMAGE_VIEWER*. -- This is a special case applied only during a package build or release. In this case, if the - package definition file contains a "config" section, settings in this section will override all - others. See [here](#package-overrides). - -It is fairly typical to provide your site-specific rez settings in a file that the environment -variable *REZ_CONFIG_FILE* is then set to for all your users. Note that you do not need to provide -a copy of all settings in this file - just provide those that are changed from the defaults. - -## Settings Merge Rules - -When multiple configuration sources are present, the settings are merged together - -one config file does not replace the previous one, it overrides it. By default, the -following rules apply: - -* Dicts are recursively merged together; -* Non-dicts override the previous value. - -However, it is also possible to append and/or prepend list-based settings. For example, the -following config entry will append to the `release_hooks` setting value defined by the -previous configuration sources (you can also supply a *prepend* argument): - - release_hooks = ModifyList(append=["custom_release_notify"]) - -## Package Overrides - -Packages themselves can override configuration settings. To show how this is useful, -consider the following example: - - # in package.py - with scope("config") as c: - c.release_packages_path = "/svr/packages/internal" - -Here a package is overriding the default release path - perhaps you're releasing -internally- and externally-developed packages to different locations, for example. - -These config overrides are only applicable during building and releasing of the package. -As such, even though any setting can be overridden, it's only useful to do so for -those that have any effect during the build/install process. These include: - -* Settings that determine where packages are found, such as *packages_path*, - *local_packages_path* and *release_packages_path*; -* Settings in the *build_system*, *release_hook* and *release_vcs* plugin types; -* *package_definition_python_path*; -* *package_filter*. - -## String Expansions - -The following string expansions occur on all configuration settings: - -* Any environment variable reference, in the form *${HOME}*; -* Any property of the *system* object, eg *{system.platform}*. - -The *system* object has the following attributes: - -* platform: The platform, eg 'linux'; -* arch: The architecture, eg 'x86_64'; -* os: The operating system, eg 'Ubuntu-12.04'; -* user: The current user's username; -* home: Current user's home directory; -* fqdn: Fully qualified domain name, eg 'somesvr.somestudio.com'; -* hostname: Host name, eg 'somesvr'; -* domain: Domain name, eg 'somestudio.com'; -* rez_version: Version of rez, eg '2.0.1'. - -## Delay Load - -It is possible to store a config setting in a separate file, which will be loaded -only when that setting is referenced. This can be useful if you have a large value -(such as a dict) that you don't want to pollute the main config with. YAML and -JSON formats are supported: - - # in rezconfig - default_relocatable_per_package = DelayLoad('/svr/configs/rez_relocs.yaml') - -## Commandline Tool - -You can use the *rez-config* command line tool to see what the current configured settings are. -Called with no arguments, it prints all settings; if you specify an argument, it prints out just -that setting: - - ]$ rez-config packages_path - - /home/sclaus/packages - - /home/sclaus/.rez/packages/int - - /home/sclaus/.rez/packages/ext - -Here is an example showing how to override settings using your own configuration file: - - ]$ echo 'packages_path = ["~/packages", "/packages"]' > myrezconfig.py - ]$ export REZ_CONFIG_FILE=${PWD}/myrezconfig.py - ]$ rez-config packages_path - - /home/sclaus/packages - - /packages - -## Configuration Settings - -Following is an alphabetical list of rez settings. - -> [[media/icons/info.png]] Note that this list has been generated automatically -> from the [rez-config.py](https://github.com/__GITHUB_REPO__/blob/main/src/rez/rezconfig.py) -> file in the rez source, so you can also refer to that file for the same information. - -__REZCONFIG_MD__ diff --git a/wiki/pages/Contexts.md b/wiki/pages/Contexts.md deleted file mode 100644 index b960353a6..000000000 --- a/wiki/pages/Contexts.md +++ /dev/null @@ -1,97 +0,0 @@ -## Overview - -When you use *rez-env* to create a resolved environment, you are actually -creating something called a *context*. A context is a store of information -including: - -* The initial [package request](Basic-Concepts#package-requests) list; -* The *resolve* - the list of variants that were chosen; -* A graph which shows the resolve visually. - -The context does not store copies of the packages it resolved to; rather, it -stores a kind of handle for each, which gives enough information to know where -to fetch the full package definition and contents from. - -Contexts themselves are quite small, and are stored in JSON format in a file -with the extension *rxt*. When you use *rez-env*, it actually creates a temporary -context file on disk, which is removed when the shell is exited: - - ]$ rez-env foo bah - - You are now in a rez-configured environment. - - resolved by ajohns@14jun01.methodstudios.com, on Wed Oct 22 12:44:00 2014, - using Rez v2.0.rc1.10 - - requested packages: - foo - bah - - resolved packages: - eek-2.6 /packages/inhouse/eek/2.6 - foo-1.2 /packages/inhouse/foo/1.2 - bah-4 /packages/inhouse/bah/4 - - > ]$ echo $REZ_RXT_FILE - /tmp/rez_context_0tMS4U/context.rxt - -## Baking Resolves - -You can use the *rez-env* flag *--output* to write a resolved context directly -to file, rather than invoking a subshell: - - ]$ rez-env foo bah --output test.rxt - -Later, you can read the context back again, to reconstruct the same environment: - - ]$ rez-env --input test.rxt - - You are now in a rez-configured environment. - - resolved by ajohns@14jun01.methodstudios.com, on Wed Oct 22 12:44:00 2014, - using Rez v2.0.rc1.10 - - requested packages: - foo - bah - - resolved packages: - eek-2.6 /packages/inhouse/eek/2.6 - foo-1.2 /packages/inhouse/foo/1.2 - bah-4 /packages/inhouse/bah/4 - - > ]$ █ - -Contexts do not store a copy of the environment that is configured (that is, the -environment variables exported, for example). A context just stores the resolved -list of packages that need to be applied in order to configure the environment. -When you load a context via *rez-env --input*, each of the packages' *commands* -sections are interpreted once more. - -You can think of package *commands* like fragments of a wrapper script which -configures an environment. By creating a context, you are creating a list of -script fragments which, when run in serial, produce the target environment. So, -if your package added a path to *$PATH* which included a reference to *$USER* -for example, this would work correctly even if Joe created the rxt file, and -Jill read it - the commands are reinterpreted when Jill loads the context. - -## The rez-context Tool - -The *rez-context* tool inspects context files. When you're within a resolved -subshell, *rez-context* inspects the current context, unless one is specified -explicitly. For example, we can inspect the context created in the previous -example, without actually being within it: - - ]$ rez-context test.rxt - - resolved by ajohns@14jun01.methodstudios.com, on Wed Oct 22 12:44:00 2014, - using Rez v2.0.rc1.10 - - requested packages: - foo - bah - - resolved packages: - eek-2.6 /packages/inhouse/eek/2.6 - foo-1.2 /packages/inhouse/foo/1.2 - bah-4 /packages/inhouse/bah/4 diff --git a/wiki/pages/Credits.md b/wiki/pages/Credits.md deleted file mode 100644 index 90494b0cf..000000000 --- a/wiki/pages/Credits.md +++ /dev/null @@ -1,15 +0,0 @@ -## Contributors - -

-__CONTRIBUTORS_MD__ -

- -## Other - -

-[[media/icons/info.png]]
-[[media/icons/warning.png]]
-[[media/icons/under_construction.png]]
-[[media/icons/new.png]] pixel buddha
-Icons made by Freepik. -

diff --git a/wiki/pages/Environment-Variables.md b/wiki/pages/Environment-Variables.md deleted file mode 100644 index 59327f3c4..000000000 --- a/wiki/pages/Environment-Variables.md +++ /dev/null @@ -1,102 +0,0 @@ -This chapter lists the environment variables that rez generates in certain -circumstances, as well as environment variables that you can set which affect -the operation of rez. - -## Context Environment Variables - -These are variables that rez generates within a resolved environment (a "context"). - -* **REZ_RXT_FILE** - Filepath of the current context (an rxt file). -* **REZ_USED** - Path to rez installation that was used to resolve this environment. -* **REZ_USED_IMPLICIT_PACKAGES** - The list of implicit packages used in the resolve. -* **REZ_USED_PACKAGES_PATH** - The package searchpath used for this resolve. -* **REZ_USED_RESOLVE** - The list of resolved packages, eg *"platform-linux utils-1.2.3"*. -* **REZ_USED_EPH_RESOLVE** - The list of resolved ephemerals, eg *".foo.cli-1 .debugging-0"*. -* **REZ_USED_LOCAL_RESOLVE** - The list of resolved local packages, eg *"utils-1.2.3 maya_utils-1.3+"*. - Packages listed here will always be a subset of the packages in *REZ_USED_RESOLVE*. -* **REZ_USED_REQUEST** - The environment request string, eg *"maya-2017 maya_utils-1.3+"*. - Does not include implicit packages. -* **REZ_USED_REQUESTED_TIMESTAMP** - The epoch time of this resolved environment, - explicitly set by the user with (for example) the rez-env '\-\-time' flag; zero otherwise. -* **REZ_USED_TIMESTAMP** - The epoch time when this environment was resolved; OR, - the value of *REZ_USED_REQUESTED_TIMESTAMP*, if non-zero. -* **REZ_USED_VERSION** - The version of rez used to resolve this environment. -* **REZ_SHELL_INIT_TIMESTAMP** - The epoch time when the current shell was instantiated. -* **REZ_SHELL_INTERACTIVE** - Will be '1' if the shell is interactive, and '0' otherwise - (ie, when a command is specified, like `rez-env foo -- mycommand`). - -Specifically, per-package, the following variables are generated. Note that for a given -package name, *"(PKG)"* in the variables below is the uppercased package name, with any -dots replaced with underscore: - -* **REZ_(PKG)_BASE** - The base directory of the package installation, eg - *"/packages/utils/1.0.0"*. -* **REZ_(PKG)_ROOT** - The root directory of the package installation (actually, - the variant), eg *"/packages/utils/1.0.0/python-2.7"*. -* **REZ_(PKG)_VERSION** - The version of the package. -* **REZ_(PKG)_MAJOR_VERSION** - The major version of the package, or ''. -* **REZ_(PKG)_MINOR_VERSION** - The minor version of the package, or ''. -* **REZ_(PKG)_PATCH_VERSION** - The patch version of the package, or ''. - -For every _ephemeral_ package request, the following variables are generated. Note -that for a given ephemeral package name, *"(PKG)"* in the variables below is the -uppercased package name, with dots replaced by underscore, and **the leading dot -removed**: - -* **REZ_EPH_(PKG)_REQUEST** - The resolved ephemeral package request. - -## Build Environment Variables - -These are variables that rez generates within a build environment, in addition -to those listed [here](#context-environment-variables). - -* **REZ_BUILD_ENV** - Always present in a build, has value 1. -* **REZ_BUILD_INSTALL** - Has a value of 1 if an installation is taking place - (either a *rez-build -i* or *rez-release*), otherwise 0. -* **REZ_BUILD_INSTALL_PATH** - Installation path, if an install is taking place. -* **REZ_BUILD_PATH** - Path where build output goes. -* **REZ_BUILD_PROJECT_DESCRIPTION** - Equal to the *description* attribute of the - package being built. -* **REZ_BUILD_PROJECT_FILE** - The filepath of the package being built (typically - a *package.py* file). -* **REZ_BUILD_PROJECT_NAME** - Name of the package being built. -* **REZ_BUILD_PROJECT_VERSION** - Version of the package being built. -* **REZ_BUILD_REQUIRES** - Space-separated list of requirements for the build - - comes from the current package's *requires*, *build_requires* and - *private_build_requires* attributes, including the current variant's requirements. -* **REZ_BUILD_REQUIRES_UNVERSIONED** - Equivalent but unversioned list to - *REZ_BUILD_REQUIRES*. -* **REZ_BUILD_SOURCE_PATH** - Path containing the package.py file. -* **REZ_BUILD_THREAD_COUNT** - Number of threads being used for the build. -* **REZ_BUILD_TYPE** - One of *local* or *central*. Value is *central* if a - release is occurring. -* **REZ_BUILD_VARIANT_INDEX** - Zero-based index of the variant currently being - built. For non-varianted packages, this is "0". -* **REZ_BUILD_VARIANT_REQUIRES** - Space-separated list of runtime requirements - of the current variant. This does not include the common requirements as found - in *REZ_BUILD_REQUIRES*. For non-varianted builds, this is an empty string. -* **REZ_BUILD_VARIANT_SUBPATH** - Subdirectory containing the current variant. - For non-varianted builds, this is an empty string. - -## Runtime Environment Variables - -These are environment variables that the user can set, which affect the -operation of rez. - -* **REZ_(CONFIG_ENTRY)** - For any given rez config entry (see *rezconfig.py*), - you can override the setting with an environment variable, for convenience. Here, - *(CONFIG_ENTRY)* is the uppercased equivalent of the setting name. For example, - a setting commonly overriden this way is *packages_path*, whos equivalent - variable is *REZ_PACKAGES_PATH*. -* **REZ_(CONFIG_ENTRY)_JSON** - Same as the previous env-var, except that the format - is a JSON string. This means that some more complex settings can be overridden, - that aren't supported in the non-JSON case (*package_filter* is an example). -* **REZ_DISABLE_HOME_CONFIG** - If 1/t/true, The default `~/.rezconfig.py` config - file is skipped. -* **EDITOR** - On Linux and OSX systems, this will set the default editor to use - if and when rez requires one (an example is on release if the *prompt_release_message* - config setting is true). -* **REZ_KEEP_TMPDIRS** - If set to a non-empty string, this prevents rez from - cleaning up any temporary directories. This is for debugging purposes. -* **REZ_SIGUSR1_ACTION** - If you set this to *print_stack*, rez will prints its - current stacktrace to stdout if sent a USR1 signal. This is for debugging purposes. diff --git a/wiki/pages/Ephemeral-Packages.md b/wiki/pages/Ephemeral-Packages.md deleted file mode 100644 index d794fd57d..000000000 --- a/wiki/pages/Ephemeral-Packages.md +++ /dev/null @@ -1,156 +0,0 @@ -## Overview - -_Introduced in version 2.71.0_ - -Ephemeral packages (or simply 'ephemerals') are requests for packages that do not -exist. Ephemeral package names always begin with a dot (`.`). Like all package -requests, ephemerals can be requested as part of packages' requires or variants -lists, or directly by the user (via `rez-env` for eg). - -Example: - - ]$ rez-env .foo-1 - You are now in a rez-configured environment. - - resolved by ajohns@turtle, on Tue Dec 22 08:17:00 2020, using Rez v2.70.0 - - requested packages: - .foo-1 (ephemeral) - ~platform==linux (implicit) - ~arch==x86_64 (implicit) - ~os==Ubuntu-16.04 (implicit) - - resolved packages: - .foo-1 (ephemeral) - -Ephemerals will act like real packages during a resolve - ie, their request ranges -will intersect, and conflicts can occur - but they never actually correlate to a -real package, nor do they perform any configuration on the runtime (not directly -in any case). - -Example showing range intersection: - - ]$ rez-env .foo-1 '.foo-1.5+' - - You are now in a rez-configured environment. - - resolved by ajohns@turtle, on Tue Dec 22 08:21:04 2020, using Rez v2.70.0 - - requested packages: - .foo-1 (ephemeral) - .foo-1.5+ (ephemeral) - ~platform==linux (implicit) - ~arch==x86_64 (implicit) - ~os==Ubuntu-16.04 (implicit) - - resolved packages: - .foo-1.5+<1_ (ephemeral) - -Example of conflicting request: - - ]$ rez-env .foo-1 .foo-2 - The context failed to resolve: - The following package conflicts occurred: (.foo-1 <--!--> .foo-2) - -## Environment Variables - -Ephemerals do not affect the runtime in the way that packages can (via their -`commands` section), however some environment variables are set: - -* `REZ_USED_EPH_RESOLVE` lists all resolved ephemeral requests; -* `REZ_EPH_(PKG)_REQUEST` is set for every resolved ephemeral. Here, `(PKG)` is - the ephemeral name, in uppercase, with dots replaced by underscores and with - **the leading dot removed**. - -The following example illustrates: - - ]$ rez-env python .foo-1 .bah-2 - ... - ]$ echo $REZ_EPH_FOO_REQUEST - 1 - ]$ echo $REZ_USED_EPH_RESOLVE - .foo-1 .bah-2 - -## Introspection - -In order for a package to inspect the ephemerals that are present in a runtime, -there is an [ephemerals](Package-Commands#ephemerals) object provided, similar -to the [resolve](Package-Commands#resolve) object. You would typically use the -[intersects](Package-Commands#intersects) function to inspect it, like so: - - # in package.py - def commands() - if intersects(ephemerals.get_range('enable_tracking', '0'), '1'): - env.TRACKING_ENABLED = 1 - -In this example, the given package would set the `TRACKING_ENABLED` environment -variable if an ephemeral such as `.enable_tracking-1` (or `.enable_tracking-1.2+` -etc) is present in the resolve. Note that the leading `.` is implied and not -included when querying the `ephemerals` object. - -> [[media/icons/warning.png]] Since `ephemerals` is a dict-like object, so it has -> a `get` function which will return a full request string if key exists. Hence, -> the default value should also be a full request string, not just a version range -> string like `'0'` in `get_range`. Or `intersects` may not work as expect. - -## Ephemeral Use Cases - -Why would you want to request packages that don't exist? There are two main use -cases. - -### Passing Information to Packages - -Ephemerals can be used as a kind of 'package option', or a way to pass information -to packages in a resolve. For example, consider the following package definition: - - name = 'bah' - - def commands(): - if intersects(ephemerals.get_range('bah.cli', '1'), '1'): - env.PATH.append('{root}/bin') - -This package will disable its command line tools if an ephemeral like `.bah.cli-0` -is present in the runtime. - -> [[media/icons/info.png]] Ephemerals are standard package requests and so can -> have any range, such as `1.2.3`, `2.5+` and so on. However, they're often used -> as boolean package options, as in the example above. In this case, it is -> recommended to use the conventional ranges `1` and `0` to designate true and -> false. - -Since ephemerals can be pretty much anything, you might also decide to use them -as a global package option. Here's another take on our example, but in this case -we introduce a `.cli` ephemeral that acts as a global whitelist: - - name = 'bah' - - def commands(): - if intersects(ephemerals.get_range('cli', ''), 'bah'): - env.PATH.append('{root}/bin') - -Here, all packages' cli will be enabled if `.cli` is not specified, but if it is -specified then it acts as a whitelist: - - # turn on cli for foo and bah only - ]$ rez-env foo-1 bah==2.3.1 eek-2.4 '.cli-foo|bah' - -### Abstract Package Representation - -Sometimes it makes sense for a package to require some form of abstract object or -capability, rather than an actual package. For example, perhaps your package (or -one of its variants) requires a GPU to be present on the host machine. To support -this, you might have something setup that includes a `.gpu-1` ephemeral in the -[implicits](Basic-Concepts#implicit-packages) list on all GPU-enabled hosts. -Then, your package could look like this: - - name = 'pixxelator' - - variants = [ - ['.gpu-0'], # renders via CPU - ['.gpu-1'] # renders via GPU - ] - -> [[media/icons/warning.png]] Be aware that on hosts that do _not_ have a gpu -> implicit, either variant could be selected. You would want to either guarantee -> that every host has the gpu implicit set to 0 or 1, or that the user always -> explicitly specifies `.gpu-0` or `.gpu-1` in their request. diff --git a/wiki/pages/FAQ.md b/wiki/pages/FAQ.md deleted file mode 100644 index d6360a1c1..000000000 --- a/wiki/pages/FAQ.md +++ /dev/null @@ -1 +0,0 @@ -FAQ diff --git a/wiki/pages/Getting-Started.md b/wiki/pages/Getting-Started.md deleted file mode 100644 index 8ac4ae7d0..000000000 --- a/wiki/pages/Getting-Started.md +++ /dev/null @@ -1,158 +0,0 @@ -## Essential Packages - -After installation, you need to create some essential Rez packages. The *rez-bind* -tool creates Rez packages that reference software already installed on your system. -Use the *--quickstart* argument to bind a set of standard packages (note that you -may require administrative privileges for some of them): - - ]$ rez-bind --quickstart - Binding platform into /home/ajohns/packages... - Binding arch into /home/ajohns/packages... - Binding os into /home/ajohns/packages... - Binding python into /home/ajohns/packages... - Binding rez into /home/ajohns/packages... - Binding rezgui into /home/ajohns/packages... - Binding setuptools into /home/ajohns/packages... - Binding pip into /home/ajohns/packages... - - Successfully converted the following software found on the current system into Rez packages: - - PACKAGE URI - ------- --- - arch /home/ajohns/packages/arch/x86_64/package.py - os /home/ajohns/packages/os/osx-10.11.5/package.py - pip /home/ajohns/packages/pip/8.0.2/package.py - platform /home/ajohns/packages/platform/osx/package.py - python /home/ajohns/packages/python/2.7.11/package.py - rez /home/ajohns/packages/rez/2.0.rc1.44/package.py - rezgui /home/ajohns/packages/rezgui/2.0.rc1.44/package.py - setuptools /home/ajohns/packages/setuptools/19.4/package.py - -Now you should be able to create an environment containing Python. Try this: - - ]$ rez-env python -- which python - /home/ajohns/packages/python-2.7.8/platform-linux/arch-x86_64/os-Ubuntu-12.04/bin/python - - -## Building Your First Package - -Before building your first rez package, ensure that: - -* The directory *$HOME/packages* exists and is writable; -* The [cmake](https://cmake.org/) tool is available. - -The *rez-build* tool is used to build packages and install them locally (typically -to *$HOME/packages*). Once you've done that, you can use them via *rez-env*, just -like any other package: - - ]$ cd example_packages/hello_world - ]$ rez-build --install - - -------------------------------------------------------------------------------- - Building hello_world-1.0.0... - -------------------------------------------------------------------------------- - Resolving build environment: python - resolved by ajohns@workstation.local, on Sun Jul 31 14:39:33 2016, using Rez v2.0.rc1.44 - - requested packages: - python - ~platform==osx (implicit) - ~arch==x86_64 (implicit) - ~os==osx-10.11.5 (implicit) - - resolved packages: - arch-x86_64 /home/ajohns/packages/arch/x86_64 (local) - os-osx-10.11.5 /home/ajohns/packages/os/osx-10.11.5 (local) - platform-osx /home/ajohns/packages/platform/osx (local) - python-2.7.11 /home/ajohns/packages/python/2.7.11/platform-osx/arch-x86_64/os-osx-10.11.5 (local) - - Invoking cmake build system... - Executing: /usr/local/bin/cmake -d /home/ajohns/workspace/rez/example_packages/hello_world -Wno-dev -DCMAKE_ECLIPSE_GENERATE_SOURCE_PROJECT=TRUE -D_ECLIPSE_VERSION=4.3 --no-warn-unused-cli -DCMAKE_INSTALL_PREFIX=/home/ajohns/packages/hello_world/1.0.0 -DCMAKE_MODULE_PATH=${CMAKE_MODULE_PATH} -DCMAKE_BUILD_TYPE=Release -DREZ_BUILD_TYPE=local -DREZ_BUILD_INSTALL=1 -G Unix Makefiles - Not searching for unused variables given on the command line. - -- Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE) - -- Configuring done - -- Generating done - -- Build files have been written to: /home/ajohns/workspace/rez/example_packages/hello_world/build - - Executing: make -j4 - [100%] Built target py - - Executing: make -j4 install - [100%] Built target py - Install the project... - -- Install configuration: "Release" - -- Installing: /home/ajohns/packages/hello_world/1.0.0/./python/hello_world.py - -- Installing: /home/ajohns/packages/hello_world/1.0.0/./python/hello_world.pyc - -- Installing: /home/ajohns/packages/hello_world/1.0.0/./bin/hello - - All 1 build(s) were successful. - -You have just built your first package, and installed it to the *local package path*, which defaults -to (and is usually kept as) *$HOME/packages*. - - -## Testing Your Package - -You can use the *rez-env* tool to request a configured environment containing your package: - - ]$ rez-env hello_world - - You are now in a rez-configured environment. - - resolved by ajohns@workstation.local, on Sun Jul 31 14:43:54 2016, using Rez v2.0.rc1.44 - - requested packages: - hello_world - ~platform==osx (implicit) - ~arch==x86_64 (implicit) - ~os==osx-10.11.5 (implicit) - - resolved packages: - arch-x86_64 /home/ajohns/packages/arch/x86_64 (local) - hello_world-1.0.0 /home/ajohns/packages/hello_world/1.0.0 (local) - os-osx-10.11.5 /home/ajohns/packages/os/osx-10.11.5 (local) - platform-osx /home/ajohns/packages/platform/osx (local) - python-2.7.11 /home/ajohns/packages/python/2.7.11/platform-osx/arch-x86_64/os-osx-10.11.5 (local) - - > ]$ █ - -Now you are within the configured environment. The caret (>) prefixed to your prompt is a visual cue -telling you that you're within a rez-configured subshell. Rez does not update the currect environment, -instead it configures a subshell and puts you within it. - -Now you can run the *hello* tool in our *hello_world* package: - - > ]$ hello - Hello world! - -If you're within a rez shell, and you forget what packages are currently available or want to see the -list again, you can use the *rez-context* tool. It prints the same information you see when you -initially created the environment: - - > ]$ rez-context - resolved by ajohns@workstation.local, on Sun Jul 31 14:43:54 2016, using Rez v2.0.rc1.44 - - requested packages: - hello_world - ~platform==osx (implicit) - ~arch==x86_64 (implicit) - ~os==osx-10.11.5 (implicit) - - resolved packages: - arch-x86_64 /home/ajohns/packages/arch/x86_64 (local) - hello_world-1.0.0 /home/ajohns/packages/hello_world/1.0.0 (local) - os-osx-10.11.5 /home/ajohns/packages/os/osx-10.11.5 (local) - platform-osx /home/ajohns/packages/platform/osx (local) - python-2.7.11 /home/ajohns/packages/python/2.7.11/platform-osx/arch-x86_64/os-osx-10.11.5 (local) - -To exit the configured environment, simply exist the shell using the *exit* command: - - > ]$ exit - ]$ █ - -You can also create a configured environment and run a command inside of it, with a single command. -When you use this form, the shell is immediately exited after the command runs: - - ]$ rez-env hello_world -- hello - Hello world! - ]$ █ diff --git a/wiki/pages/Glossary.md b/wiki/pages/Glossary.md deleted file mode 100644 index c1cf05651..000000000 --- a/wiki/pages/Glossary.md +++ /dev/null @@ -1,50 +0,0 @@ - - -## context -An object containing everything about an environment resolve. Contexts can be stored in - .rxt files, and used to reconstruct the same environment at a later date. - -## local package path -Path to package repository where locally-developed packages are installed to. This path typically -appears at the front of the packages search path. It is typically `~/packages`. - -## package -A versioned piece of software - the things that rez manages. - -## package commands -A block of python code in the package definition file that determines how the package updates the -environment it is used in. - -## package definition file -A file, such as 'package.py', that defines everything we want to know about a package, including -its dependencies. Every package has one. - -## package repository -A place where packages are stored - usually a directory on disk. - -## package request -A string describing a request for a package, such as "python-2.6+", "foo==1.0.0". - -## package search path -Search path that rez uses to find packages. - -## request -A list of package requests, such as ("python-2.6+", "foo-1", "bah==2.3.3"). - -## resolve -A list of packages resulting from resolving a request with the dependency solver. - -## rez-config -A command line tool that shows the current rez configuration settings. - -## rez-env -A command line tool that places the user into a newly resolved environment. - -## version -A version number, such as "1", "2.0", "1.5.3alpha". - -## version conflict -Two requests for the same package that do not overlap. For example, ("python-2.5", "python-2.7"). - -## version range -A string describing a range of possible versions, such as "4+", "<2.1", "3.0", "1.1+<2", "==4.2.2". diff --git a/wiki/pages/Home.md b/wiki/pages/Home.md deleted file mode 100644 index 25517e4c8..000000000 --- a/wiki/pages/Home.md +++ /dev/null @@ -1,105 +0,0 @@ -## What Is Rez? - -Rez is a cross-platform package manager with a difference. Using Rez you can create -standalone environments configured for a given set of packages. However, unlike many -other package managers, packages are not installed into these standalone environments. -Instead, all package versions are installed into a central repository, and standalone -environments reference these existing packages. This means that configured environments -are lightweight, and very fast to create, often taking just a few seconds to configure -despite containing hundreds of packages. - -

- - -
Typical package managers install packages into an environment -

- -
-

- - -
Rez installs packages once, and configures environments dynamically -

- -
-Rez takes a list of package requests, and constructs the target environment, resolving -all the necessary package dependencies. Any type of software package is supported - -compiled, python, applications and libraries. - - -## The Basics - -Packages are stored in repositories on disk. Each package has a single concise -definition file (*package.py*) that defines its dependencies, its commands (how it -configures the environment containing it), and other metadata. For example, the -following is the package definition file for the popular *requests* python module: - - name = "requests" - - version = "2.8.1" - - authors = ["Kenneth Reitz"] - - requires = [ - "python-2.7+" - ] - - def commands(): - env.PYTHONPATH.append("{root}/python") - -This package requires python-2.7 or greater. When used, the 'python' subdirectory -within its install location is appended to the PYTHONPATH environment variable. - -When an environment is created with the rez API or *rez-env* tool, a dependency -resolution algorithm tracks package requirements and resolves to a list of needed -packages. The commands from these packages are concatenated and evaluated, resulting -in a configured environment. Rez is able to configure environments containing -hundreds of packages, often within a few seconds. Resolves can also be saved to file, -and when re-evaluated later will reconstruct the same environment once more. - - -## Examples - -This example places the user into a resolved shell containing the requested packages, -using the [rez-env](Command-Line-Tools#rez-env) tool: - - ]$ rez-env requests-2.2+ python-2.6 'pymongo-0+<2.7' - - You are now in a rez-configured environment. - - resolved by ajohns@nn188.somewhere.com, on Wed Feb 26 15:56:20 2014, using Rez v2.0.0 - - requested packages: - requests-2.2+ - python-2.6 - pymongo-0+<2.7 - - resolved packages: - python-2.6.8 /software/ext/python/2.6.8 - platform-linux /software/ext/platform/linux - requests-2.2.1 /software/ext/requests/2.2.1/python-2.6 - pymongo-2.6.3 /software/ext/pymongo/2.6.3 - arch-x86_64 /software/ext/arch/x86_64 - - > ]$ _ - -This example creates an environment containing the package 'houdini' version 12.5 -or greater, and runs the command 'hescape -h' inside that environment: - - ]$ rez-env houdini-12.5+ -- hescape -h - Usage: hescape [-foreground] [-s editor] [filename ...] - -h: output this usage message - -s: specify starting desktop by name - -foreground: starts process in foreground - -Resolved environments can also be created via the API: - - >>> import subprocess - >>> from rez.resolved_context import ResolvedContext - >>> - >>> r = ResolvedContext(["houdini-12.5+", "houdini-0+<13", "java", "!java-1.8+"]) - >>> p = r.execute_shell(command='which hescape', stdout=subprocess.PIPE) - >>> out, err = p.communicate() - >>> - >>> print out - '/software/ext/houdini/12.5.562/bin/hescape' diff --git a/wiki/pages/Installation.md b/wiki/pages/Installation.md deleted file mode 100644 index ebbfd3c82..000000000 --- a/wiki/pages/Installation.md +++ /dev/null @@ -1,89 +0,0 @@ -## Installation Script - -> [[media/icons/warning.png]] Only Python greater or eual to 3.7 is supported -> when installing via the install script! - -To install rez, download the source. Then from the root directory, run: - -``` -]$ python ./install.py -``` - -This installs rez to `/opt/rez`. See `install.py -h` for how to install to a -different location. - -Once the installation is complete, a message tells you how to run it: - -``` -SUCCESS! To activate Rez, add the following path to $PATH: -/opt/rez/bin/rez - -You may also want to source the completion script (for bash): -source /opt/rez/completion/complete.sh -``` - -> [[media/icons/warning.png]] Do _not_ move the installation - re-install to a new -> location if you want to change the install path. If you want to install rez for -> multiple operating systems, perform separate installs for each of those systems. - - -## Installation Via Pip - -It is possible to install rez with pip, like so: - -``` -]$ pip install rez -``` - -> [[media/icons/info.png]] While the `install.py` script only supoprts Python 3, -> rez is still usable with Python when installed via pip. In other words, the API -> is fully compatible with Python 2 and 3. - -However, this comes with a caveat - rez command line tools _are not guaranteed -to work correctly_ once inside a rez environment (ie after using the `rez-env` -command). The reasons are given in the next section. - -Pip installation is adequate however, if all you require is the rez API, or you -don't require its command line tools to be available within a resolved environment. - -Note that running pip-installed rez command line tools will print a warning like so: - -``` -Pip-based rez installation detected. Please be aware that rez command line tools -are not guaranteed to function correctly in this case. See -https://github.com/AcademySoftwareFoundation/rez/wiki/Installation#why-not-pip-for-production -for futher details. -``` - - -## Why Not Pip For Production? - -Rez is not a normal python package. Although it can successfully be installed -using standard mechanisms such as pip, this comes with a number of caveats. -Specifically: - -* When within a rez environment (ie after using the `rez-env` command), the rez - command line tools are not guaranteed to function correctly; -* When within a rez environment, other packages' tools (that were also installed - with pip) remain visible, but are not guaranteed to work. - -When you enter a rez environment, the rez packages in the resolve configure -that environment as they see fit. For example, it is not uncommon for a python -package to append to PYTHONPATH. Environment variables such as PYTHONPATH -affect the behaviour of tools, including rez itself, and this can cause it to -crash or behave abnormally. - -When you use the `install.py` script to install rez, some extra steps are taken -to avoid this problem. Specifically: - -* Rez is installed into a virtualenv so that it operates standalone; -* The rez tools are shebanged with `python -E`, in order to protect them from - environment variables that affect python's behaviour; -* The rez tools are stored in their own directory, so that other unrelated tools - are not visible. - -Due to the way standard wheel-based python installations work, it simply is not -possible to perform these extra steps without using a custom installation script. -Wheels do not give the opportunity to run post-installation code; neither do -they provide functionality for specifying interpreter arguments to be added for -any given entry point. diff --git a/wiki/pages/Managing-Packages.md b/wiki/pages/Managing-Packages.md deleted file mode 100644 index 3a2d071c0..000000000 --- a/wiki/pages/Managing-Packages.md +++ /dev/null @@ -1,360 +0,0 @@ -## Ignoring Packages - -Packages can be ignored. When this happens, the package is still present in its -repository, but it will not be visible to the rez API nor to any newly resolved -runtimes. Any runtimes that are currently using an ignored package are unaffected, -since the package's payload has not been removed. - -To ignore a package via comandline: - -``` -]$ # you need to specify the repo, but you'll be shown a list if you don't -]$ rez-pkg-ignore foo-1.2.3 -No action taken. Run again, and set PATH to one of: -filesystem@/home/ajohns/packages - -]$ rez-pkg-ignore foo-1.2.3 filesystem@/home/ajohns/packages -Package is now ignored and will not be visible to resolves -``` - -Via API: - -``` ->>> from rez.package_repository import package_repository_manager ->>> ->>> repo_path = "filesystem@/home/ajohns/packages" ->>> repo = package_repository_manager.get_repository(repo_path) ->>> repo.ignore_package("foo", "1.2.3") -1 # -1: pkg not found; 0: pkg already ignored; 1: pkg ignored -``` - -Both of these options generate a *.ignore\* file (e.g. -*.ignore3.1.2*) next to the package version directory. - -You can also do the reverse (ie unignore a package). Use the `-u` option of -`rez-pkg-ignore`, or the `unignore_package` function on the package repository -object. - - -## Copying Packages - -Packages can be copied from one [package repository](Basic-Concepts#package-repositories) -to another, like so: - -Via commandline: - -``` -]$ rez-cp --dest-path /svr/packages2 my_pkg-1.2.3 -``` - -Via API: - -``` ->>> from rez.package_copy import copy_package ->>> from rez.packages import get_latest_package ->>> ->>> p = get_latest_package("python") ->>> p -Package(FileSystemPackageResource({'location': '/home/ajohns/packages', 'name': 'python', 'repository_type': 'filesystem', 'version': '3.7.4'})) - ->>> r = copy_package(p, "./repo2") ->>> ->>> print(pprint.pformat(r)) -{ - 'copied': [ - ( - Variant(FileSystemVariantResource({'location': '/home/ajohns/packages', 'name': 'python', 'repository_type': 'filesystem', 'index': 0, 'version': '3.7.4'})), - Variant(FileSystemVariantResource({'location': '/home/ajohns/repo2', 'name': 'python', 'repository_type': 'filesystem', 'index': 0, 'version': '3.7.4'})) - ) - ], - 'skipped': [] -} -``` - -Copying packages is actually done one variant at a time, and you can copy some -variants of a package if you want, rather than the entire package. The API call's -return value shows what variants were copied - The 2-tuple in `copied` lists the -source (the variant that was copied from) and destination (the variant that was -created) respectively. - -> [[media/icons/warning.png]] Do not simply copy package directories on disk - -> you should always use `rez-cp`. Copying directly on disk is bypassing rez and -> this can cause problems such as a stale resolve cache. Using `rez-cp` gives -> you more control anyway. - -### Enabling Package Copying - -Copying packages is enabled by default, however you're also able to specify which -packages are and are not _relocatable_, for much the same reasons as given -[here](Managing-Packages#enabling-package-caching). - -You can mark a package as non-relocatable by setting [relocatable](Package-Definition-Guide#relocatable) -to False in its package definition file. There are also config settings that affect relocatability -in the event that relocatable is not defined in a package's definition. For example, -see [default_relocatable](Configuring-Rez#default_relocatable), -[default_relocatable_per_package](Configuring-Rez#default_relocatable_per_package) -and [default_relocatable_per_repository](Configuring-Rez#default_relocatable_per_repository). - -Attempting to copy a non-relocatable package will raise a `PackageCopyError`. -However, note that there is a `force` option that will override this - use at -your own risk. - - -## Moving Packages - -Packages can be moved from one [package repository](Basic-Concepts#package-repositories) -to another. Be aware that moving a package does not actually delete the source -package however. Instead, the source package is hidden (ignored) - it is up to -you to delete it at some later date. - -To move a package via commandline: - -``` -]$ rez-mv --dest-path /packages2 python-3.7.4 /packages -``` - -Via API: - -``` ->>> from rez.package_move import move_package ->>> from rez.packages import get_package_from_repository ->>> ->>> p = get_package_from_repository("python", "3.7.4", "/packages") ->>> p -Package(FileSystemPackageResource({'location': '/packages', 'name': 'python', 'repository_type': 'filesystem', 'version': '3.7.4'})) ->>> ->>> new_p = move_package(p, "/packages2") ->>> new_p -Package(FileSystemPackageResource({'location': '/packages2', 'name': 'python', 'repository_type': 'filesystem', 'version': '3.7.4'})) ->>> ->>> p = get_package_from_repository("python", "3.7.4", "/packages") ->>> p -None -``` - -Be aware that a non-relocatable package is also not movable (see -[here](Package-Definition-Guide#relocatable) for more details). Like package -copying, there is a `force` option to move it regardless. - -A typical reason you might want to move a package is to archive packages that are -no longer in use. In this scenario, you would move the package to some archival -package repository. In case an old runtime needs to be resurrected, you would add -this archival repository to the packages path before performing the resolve. Note -that you will probably want to use the `--keep-timestamp` option when doing this, -otherwise rez will think the package did not exist prior to its archival date. - - -## Removing Packages - -Packages can be removed. This is different from ignoring - the package and its -payload is deleted from storage, whereas ignoring just hides it. It is not -possible to un-remove a package. - -To remove a package via commandline: - -``` -]$ rez-rm --package python-3.7.4 /packages -``` - -Via API: - -``` ->>> from rez.package_remove import remove_package ->>> ->>> remove_package("python", "3.7.4", "/packages") -``` - -During the removal process, package versions will first be ignored so that -partially-deleted versions are not visible. - -It can be useful to ignore packages that you don't want to use anymore, and -actually remove them at a later date. This gives you a safety buffer in case -current runtimes are using the package - they won't be affected if the package is -ignored, but could break if it is removed. - -To facilitate this workflow, `rez-rm` lets you remove all packages that have -been ignored for longer than N days (using the timestamp of the -*.ignore\* file). Here we remove all packages that have been ignored -for 30 days or longer: - -``` -]$ rez-rm --ignored-since=30 -v -14:47:09 INFO Searching filesystem@/home/ajohns/packages... -14:47:09 INFO Removed python-3.7.4 from filesystem@/home/ajohns/packages -1 packages were removed. -``` - -Via API: - -``` ->>> from rez.package_remove import remove_packages_ignored_since ->>> ->>> remove_packages_ignored_since(days=30) -1 -``` - - -## Package Caching - -Package caching is a feature that copies package payloads onto local disk in -order to speed up runtime environments. For example, if your released packages -reside on shared storage (which is common), then running say, a Python process, -will fetch all source from the shared storage across your network. The point of -the cache is to copy that content locally instead, and avoid the network cost. - -> [[media/icons/info.png]] Please note: Package caching does _NOT_ cache package -> definitions - only their payloads (ie, the package root directory). - -### Enabling Package Caching - -Package caching is not enabled by default. To enable it, you need to configure -[cache_packages_path](Configuring-Rez#cache_packages_path) to specify a path to -store the cache in. - -You also have granular control over whether an individual package will or will -not be cached. To make a package cachable, you can set [cachable](Package-Definition-Guide#cachable) - to False in its package definition file. Reasons you may _not_ want to do this include -packages that are large, or that aren't relocatable because other compiled packages are -linked to them in a way that doesn't support library relocation. - -There are also config settings that affect cachability in the event that `cachable` -is not defined in a package's definition. For example, see -[default_cachable](Configuring-Rez#default_cachable), -[default_cachable_per_package](Configuring-Rez#default_cachable_per_package) -and [default_cachable_per_repository](Configuring-Rez#default_cachable_per_repository). - -Note that you can also disable package caching on the command line, using -`rez-env --no-pkg-caching`. - -### Verifying - -When you resolve an environment, you can see which variants have been cached by -noting the `cached` label in the righthand column of the `rez-context` output, -as shown below: - -``` -]$ rez-env Flask - -You are now in a rez-configured environment. - -requested packages: -Flask -~platform==linux (implicit) -~arch==x86_64 (implicit) -~os==Ubuntu-16.04 (implicit) - -resolved packages: -Flask-1.1.2 /home/ajohns/package_cache/Flask/1.1.2/d998/a (cached) -Jinja2-2.11.2 /home/ajohns/package_cache/Jinja2/2.11.2/6087/a (cached) -MarkupSafe-1.1.1 /svr/packages/MarkupSafe/1.1.1/d9e9d80193dcd9578844ec4c2c22c9366ef0b88a -Werkzeug-1.0.1 /home/ajohns/package_cache/Werkzeug/1.0.1/fe76/a (cached) -arch-x86_64 /home/ajohns/package_cache/arch/x86_64/6450/a (cached) -click-7.1.2 /home/ajohns/package_cache/click/7.1.2/0da2/a (cached) -itsdangerous-1.1.0 /home/ajohns/package_cache/itsdangerous/1.1.0/b23f/a (cached) -platform-linux /home/ajohns/package_cache/platform/linux/9d4d/a (cached) -python-3.7.4 /home/ajohns/package_cache/python/3.7.4/ce1c/a (cached) -``` - -For reference, cached packages also have their original payload location stored to -an environment variable like so: - -``` -]$ echo $REZ_FLASK_ORIG_ROOT -/svr/packages/Flask/1.1.2/88a70aca30cb79a278872594adf043dc6c40af99 -``` - -### How it Works - -Package caching actually caches _variants_, not entire packages. When you perform -a resolve, or source an existing context, the variants required are copied to -local disk asynchronously (if they are cachable), in a separate process called -`rez-pkg-cache`. This means that a resolve will not necessarily use the cached -variants that it should, the first time around. Package caching is intended to have -a cumulative effect, so that more cached variants will be used over time. This is -a tradeoff to avoid blocking resolves while variant payloads are copied across -your network (and that can be a slow process). - -Note that a package cache is **not** a package repository. It is simply a store -of variant payloads, structured in such a way as to be able to store variants from -any package repository, into the one shared cache. - -Variants that are cached are assumed to be immutable - no check is done to see if -a variant's payload has changed, and needs to replace an existing cache entry. So -you should _not_ enable caching on package repositories where packages may get -overwritten. It is for this reason that caching is disabled for local packages by -default (see [package_cache_local](Configuring-Rez#package_cache_local)). - -### Commandline Tool - -#### Inspection - -Use the `rez-pkg-cache` tool to view the state of the cache, and to perform -warming and deletion operations. Example output follows: - -``` -]$ rez-pkg-cache -Package cache at /home/ajohns/package_cache: - -status package variant uri cache path ------- ------- ----------- ---------- -cached Flask-1.1.2 /svr/packages/Flask/1.1.2/package.py[0] /home/ajohns/package_cache/Flask/1.1.2/d998/a -cached Jinja2-2.11.2 /svr/packages/Jinja2/2.11.2/package.py[0] /home/ajohns/package_cache/Jinja2/2.11.2/6087/a -cached Werkzeug-1.0.1 /svr/packages/Werkzeug/1.0.1/package.py[0] /home/ajohns/package_cache/Werkzeug/1.0.1/fe76/a -cached arch-x86_64 /svr/packages/arch/x86_64/package.py[] /home/ajohns/package_cache/arch/x86_64/6450/a -cached click-7.1.2 /svr/packages/click/7.1.2/package.py[0] /home/ajohns/package_cache/click/7.1.2/0da2/a -cached itsdangerous-1.1.0 /svr/packages/itsdangerous/1.1.0/package.py[0] /home/ajohns/package_cache/itsdangerous/1.1.0/b23f/a -cached platform-linux /svr/packages/platform/linux/package.py[] /home/ajohns/package_cache/platform/linux/9d4d/a -copying python-3.7.4 /svr/packages/python/3.7.4/package.py[0] /home/ajohns/package_cache/python/3.7.4/ce1c/a -stalled MarkupSafe-1.1.1 /svr/packages/MarkupSafe/1.1.1/package.py[1] /home/ajohns/package_cache/MarkupSafe/1.1.1/724c/a - -``` - -Each variant is stored into a directory based on a partial hash of that variant's -unique identifier (its "handle"). The package cache is thread- and multiprocess- -proof, and uses a file lock to control access where necessary. - -Cached variants have one of the following statuses at any given time: - -* **copying**: The variant is in the process of being copied into the cache, and is not - yet available for use; -* **cached**: The variant has been cached and is ready for use; -* **stalled**: The variant was getting copied, but something went wrong and there is - now a partial copy present (but unused) in the cache. - -#### Logging - -Caching operations are stored into logfiles within the cache directory. To view: - -``` -]$ rez-pkg-cache --logs -rez-pkg-cache 2020-05-23 16:17:45,194 PID-29827 INFO Started daemon -rez-pkg-cache 2020-05-23 16:17:45,201 PID-29827 INFO Started caching of variant /home/ajohns/packages/Werkzeug/1.0.1/package.py[0]... -rez-pkg-cache 2020-05-23 16:17:45,404 PID-29827 INFO Cached variant to /home/ajohns/package_cache/Werkzeug/1.0.1/fe76/a in 0.202576 seconds -rez-pkg-cache 2020-05-23 16:17:45,404 PID-29827 INFO Started caching of variant /home/ajohns/packages/python/3.7.4/package.py[0]... -rez-pkg-cache 2020-05-23 16:17:46,006 PID-29827 INFO Cached variant to /home/ajohns/package_cache/python/3.7.4/ce1c/a in 0.602037 seconds -``` - -#### Cleaning The Cache - -Cleaning the cache refers to deleting variants that are stalled or no longer in use. -It isn't really possible to know whether a variant is in use, so there is a -configurable [package_cache_max_variant_days](Configuring-Rez#package_cache_max_variant_days) -setting, that will delete variants that have not been used (ie that have not appeared -in a created or sourced context) for more than N days. - -You can also manually remove variants from the cache using `rez-pkg-cache -r`. -Note that when you do this, the variant is no longer available in the cache, -however it is still stored on disk. You must perform a clean (`rez-pkg-cache --clean`) -to purge unused cache files from disk. - -You can use the [package_cache_clean_limit](Configuring-Rez#package_cache_clean_limit) -setting to asynchronously perform some cleanup every time the cache is updated. If -you do not use this setting, it is recommended that you set up a cron or other form -of execution scheduler, to run `rez-pkg-cache --clean` periodically. Otherwise, -your cache will grow indefinitely. - -Lastly, note that a stalled variant will not attempt to be re-cached until it is -removed by a clean operation. Using `package_cache_clean_limit` will not clean -stalled variants either, as that could result in a problematic variant getting -cached, then stalled, then deleted, then cached again and so on. You must run -`rez-pkg-cache --clean` to delete stalled variants. diff --git a/wiki/pages/Notes.md b/wiki/pages/Notes.md deleted file mode 100644 index 5efbbdc0a..000000000 --- a/wiki/pages/Notes.md +++ /dev/null @@ -1,4 +0,0 @@ - - -## Bez Deprecation, awaiting PR merge - diff --git a/wiki/pages/Package-Commands.md b/wiki/pages/Package-Commands.md deleted file mode 100644 index 914a2f1b3..000000000 --- a/wiki/pages/Package-Commands.md +++ /dev/null @@ -1,612 +0,0 @@ -## Overview - -Package definition files (*package.py*) usually define a *commands* section. This is a python -function that determines how the environment is configured in order to include the package. - -Consider the simple example: - - def commands(): - env.PYTHONPATH.append("{root}/python") - env.PATH.append("{root}/bin") - -This is a typical case, where a package adds its source path to PYTHONPATH, and its tools to -PATH. The "{root}" string expands to the installation directory of the package. - -When a rez environment is configured, every package in the resolve list has its *commands* section -interpreted and converted into shell code (the language - bash or other - depends on the platform -and is extensible). The resulting shell code is sourced, and this configures the environment. -Within a configured environment, the variable *REZ_CONTEXT_FILE* points at this shell code, and the -command *rez-context --interpet* prints it. - -The python API that you use in the *commands* section is called *rex* (Rez EXecution language). It -is an API for performing shell operations in a shell-agnostic way. Some common operations you would -perform with this API include setting environment variables, and appending/prepending path-like -environment variables. - -> [[media/icons/info.png]] By default, environment variables that are not referenced by any package -> are left unaltered. There will typically be many system variables that are left unchanged. - -> [[media/icons/warning.png]] If you need to import any python modules to use in a *commands* -> section, the import statements **must** appear inline to that function. - -## Order Of Command Execution - -The order in which package commands are interpreted depends on two factors - the order in which -the packages were requested, and dependencies between packages. This order can be defined as: - -* If package *A* was requested before package *B*, then *A*'s commands are interpreted before *B*'s; -* Unless package *A* requires (depends on) *B*, in which case *B* will be interpreted before *A*. - -Consider a package *maya_anim_tool*. Let us say this is a maya plugin. Naturally it has a dependency -on *maya*, therefore *maya*'s commands will be interpreted first. This is because the maya plugin -may depend on certain environment variables that *maya* sets. For example, *maya* might initialize -the *MAYA_PLUG_IN_PATH* environment variable, and *maya_anim_tool* may then append to this -variable. - -For example, consider the request: - - ]$ rez-env maya_anim_tool-1.3+ PyYAML-3.10 maya-2015 - -Assuming that *PyYAML* depends on *python*, and *maya_anim_tool* depends on *maya*, then the -resulting *commands* execution order would be: - -* maya; -* maya_anim_tool; -* python; -* PyYAML. - -## Variable Appending And Prepending - -Path-like environment variables can be appended and prepended like so: - - env.PATH.append("{root}/bin") - -However, the first append/prepend operation on any given variable actually **overwrites** the -variable, rather than appending. Why does this happen? Consider *PYTHONPATH* - if an initial -overwrite did not happen, then any modules visible on *PYTHONPATH* before the rez environment was -configured would still be there. This would mean you may not have a properly configured -environment. If your system *PyQt* were on *PYTHONPATH* for example, and you used *rez-env* to set -a different *PyQt* version, an attempt to import it within the configured environment would still, -incorrectly, import the system version. - -> [[media/icons/info.png]] *PATH* is a special case. It is not simply overwritten, because if that -> happened you would lose important system paths and thus utilities like *ls* and *cd*. In this -> case the system paths are appended back to *PATH* after all commands are interpreted. The system -> paths are defined as the default value of *PATH* in a non-interactive shell. - -> [[media/icons/under_construction.png]] Better control over environment variable initialization is -> coming. Specifically, you will be able to specify various modes for variables. For example, one -> mode will append the original (pre-rez) value back to the resulting value. - -## String Expansion - -### Object Expansion - -Any of the objects available to you in a *commands* section can be referred to in formatted strings -that are passed to rex functions such as *setenv* and so on. For example, consider the code: - - appendenv("PATH", "{root}/bin") - -Here, "{root}" will expand out to the value of [root](#root), which is the installation path of the -package ("this.root" could also have been used). - -You don't *have* to use this feature; it is provided as a convenience. For example, the following -code is equivalent to the previous example, and is just as valid (but more verbose): - - import os.path - appendenv("PATH", os.path.join(root, "bin")) - -Object string expansion is also supported when setting an environment variable via the *env* object: - - env.FOO_LIC = "{this.root}/lic" - -### Environment Variable Expansion - -Environment variable expansion is also supported when passed to rex functions. The syntaxes *$FOO* -and *${FOO}* are supported, regardless of the syntax supported by the target shell. - -### Literal Strings - -You can use the [literal](#literal) function to inhibit object- and environment variable- string -expansion. For example, the following code will set the environment variable to the literal string: - - env.TEST = literal("this {root} will not expand") - -There is also an expandable function, which matches the default behavior. You wouldn't typically -use this function; however, you can define a string containing literal and expandable parts by -chaining together *literal* and *expandable*: - - env.DESC = literal("the value of {root} is").expandable("{root}") - -### Explicit String Expansion - -Object string expansion usually occurs **only** when a string is passed to a rex function, or to -the *env* object. For example the simple statement *var = "{root}/bin"* would not expand "{root}" -into *var*. However, you can use the [expandvars](#expandvars) function to enable this behavior -explicitly: - - var = expandvars("{root}/bin") - -The *expandvars* and *expandable* functions are slightly different - *expandable* will generate a -shell variable assignment that will expand out; *expandvars* will expand the value immediately. - -This table illustrates the difference between *literal*, *expandable* and *expandvars*: - -package command | equivalent bash command ---------------------------------|------------------------ -env.FOO = literal("${USER}") | export FOO='${USER}' -env.FOO = expandable("${USER}") | export FOO="${USER}" -env.FOO = expandvars("${USER}") | export FOO="jbloggs" - -## Filepaths - -Rez expects POSIX-style filepath syntax in package commands, regardless of the shell or platform. -Thus, even if you're on Windows, you should do this: - - def commands(): - env.PATH.append("{root}/bin") # note the forward slash - -Where necessary, filepaths will be automatically normalized for you - that is, converted into -the syntax expected by the shell. In order for this to work correctly however, rez needs to know -what environment variables are actually paths. You determine this with the -[pathed_env_vars](Configuring-Rez#pathed_env_vars) config setting. By default, any environment -variable ending in `PATH` will be treated as a filepath or list of filepaths, and any -set/append/prepend operation on it will cause those values to be path-normalized automatically. - -> [[media/icons/warning.png]] Avoid using `os.pathsep` or hardcoded lists of paths such as -> `{root}/foo:{root}/bah`. Doing so can cause your package to be incompatible with some shells or -> platforms. Even the seemingly innocuous `os.pathsep` is an issue, because there are some cases -> (eg Git for Windows, aka git-bash) where the shell's path separator does not match the underlying -> system's. - -## Pre And Post Commands - -Occasionally it's useful for a package to run commands either before or after all other packages, -regardless of the command execution order rules. This can be achieved by defining a *pre_commands* -or *post_commands* function. A package can have any, all or none of *pre_commands*, *commands* and -*post_commands* defined, although it is very common for a package to define just *commands*. - -The order of command execution is: - -* All package *pre_commands* are executed, in standard execution order; -* Then, all package *commands* are executed, in standard execution order; -* Then, all package *post_commands* are executed, in standard execution order. - -## Pre Build Commands - -If a package is being built, that package's commands are not run, simply because that package is -not present in its own build environment! However, sometimes there is a need to run commands -specifically for the package being built. For example, you may wish to set some environment -variables to pass information along to the build system. - -The *pre_build_commands* function does just this. It is called prior to the build. Note that info -about the current build (such as the installation path) is available in a -[build](#build) object (other commands functions do not have this object visible). - -## Pre Test Commands - -Sometimes it's useful to perform some extra configuration in the environment that a package's test -will run in. You can define the *pre_test_commands* function to do this. It will be invoked just -before the test is run. As well as the standard [this](#this) object, a [test](#test) object is also -provided to distinguish which test is about to run. - -## A Largish Example - -Here is an example of a package definition with a fairly lengthy *commands* section: - - name = "foo" - - version = "1.0.0" - - requires = [ - "python-2.7", - "~maya-2015" - ] - - def commands(): - import os.path # imports MUST be inline to the function - - # add python module, executables - env.PYTHONPATH.append("{this.root}/python") - env.PATH.append("{this.root}/bin") - - # show include path if a build is occurring - if building: - env.FOO_INCLUDE_PATH = "{this.root}/include" - - # debug support to point at local config - if defined("DEBUG_FOO"): - conf_file = os.path.expanduser("~/.foo/config") - else: - conf_file = "{this.root}/config" - env.FOO_CONFIG_FILE = conf_file - - # if maya is in use then include the maya plugin part of this package - if "maya" in resolve: - env.MAYA_PLUG_IN_PATH.append("{this.root}/maya/plugins") - - if resolve.maya.version.minor == "sp3": - error("known issue with GL renderer in service pack 3, beware") - - # license file per major version - env.FOO_LIC = "/lic/foo_{this.version.major}.lic" - -## Objects - -Various objects and functions are available to use in the *commands* function (as well as -*pre_commands* and *post_commands*). For example, *env* is a dict-like object that represents all -the environment variables being constructed in the target environment. - -Following is a list of the objects and functions available. - -### alias -*Function* - - alias("nukex", "Nuke -x") - -Create a command alias. - -> [[media/icons/info.png]] In *bash*, aliases are implemented as bash functions. - -### base -*String* - -See [this.base](#thisbase). - -### build -*Dict-like object* - - if build.install: - info("An installation is taking place") - -This object is only available in the [pre_build_commands](#pre-build-commands) -function. It has the following fields: - -#### build.build_type -*String* - -One of 'local', 'central'. The type is _central_ if a package _release_ is occurring, and _local_ -otherwise. - -#### build.install -*Boolean* - -True if an installation is taking place, False otherwise. - -#### build.build_path -*String* - -Path to the build directory (not the installation path). This will typically reside somewhere -within the `./build` subdirectory of the package being built. - -#### build.install_path -Installation directory. Note that this will be set, even if an installation is _not_ taking place. -Do not check this variable to detect if an installation is occurring - see `build.install` instead. - -### building -*Boolean* - - if building: - env.FOO_INCLUDE_PATH = "{root}/include" - -This boolean variable is *True* if a build is occurring (typically done via the *rez-build* tool), -and *False* otherwise, however, the `commands` block is only executed when the package is brought -into a resolved environment, so this is not used when the package itself is building. Typically a -package will use this variable to set environment variables that are only useful during when other -packages are being built - C++ header include paths are a good example. - -### command -*Function* - - command("rm -rf ~/.foo_plugin") - -Run an arbitrary shell command. Note that you cannot return a value from this function call, because -*the command has not yet run*. All of the packages in a resolve only have their commands executed -after all packages have been interpreted and converted to the target shell language. Therefore any -value returned from the command, or any side effect the command has, is not visible to any package. - -You should prefer to perform simple operations (such as file manipulations and so on) in python -where possible instead. Not only does that take effect immediately, but it's also more cross -platform. For example, instead of running the command above, we could have done this: - - def commands(): - import shutil - import os.path - path = os.path.expanduser("~/.foo_plugin") - if os.path.exists(path): - shutil.rmtree(path) - -### comment -*Function* - - if "nuke" in resolve: - comment("note: taking over 'nuke' binary!") - alias("nuke", "foo_nuke_replacer") - -Creates a comment line in the converted shell script code. This is only visible if the user views -the current shell's code using the command *"rez-context --interpret"* or looks at the file -referenced by the environment variable *REZ_CONTEXT_FILE*. You would create a comment for debugging -purposes. - -### defined -*Function* - - if defined("REZ_MAYA_VERSION"): - env.FOO_MAYA = 1 - -Use this boolean function to determine whether or not an environment variable is set. - -### env -*Dict-like object* - - env.FOO_DEBUG = 1 - env["BAH_LICENSE"] = "/lic/bah.lic" - -The *env* object represents the environment dict of the configured environment. Note that this is -different from the standard python *os.environ* dict, which represents the current environment, -not the one being configured. If a prior package's *commands* set a variable via the *env* object, -it will be visible only via *env*, not *os*. The *os* dict hasn't been updated because the target -configured environment does not yet exist! - -The *env* object also provides the following functions: - -#### env.append -*Function* - - env.PATH.append("{root}/bin") - -Appends a value to an environment variable. By default this will use the *os.pathsep* delimiter -between list items, but this can be overridden using the config setting *env_var_separators*. See -[here](#variable-appending-and-prepending) for further information on the behavior of this function. - -#### env.prepend -*Function* - - env.PYTHONPATH.prepend("{root}/python") - -like *env.append*, but prepends the environment variable instead. - -### ephemerals -*Dict-like object* - - if "foo.cli" in ephemerals: - info("Foo cli option is being specified!") - -A dict representing the list of ephemerals in the resolved environment. Each item is a -string (the full request, eg `.foo.cli-1`), keyed by the ephemeral package name. Note -that you do **not** include the leading `.` when getting items from the `ephemerals` -object. - -Use `get_range` to test with the [intersects](Package-Commands#intersects) function. -Here, we enable foo's commandline tools by default, unless explicitly disabled via -a request for `.foo.cli-0`: - - if intersects(ephemerals.get_range("foo.cli", "1"), "1"): - info("Enabling foo cli tools") - env.PATH.append("{root}/bin") - -### error -*Function* - - if "PyQt" in resolve: - error("The floob package has problems running in combo with PyQt") - -Prints to standard error. - -> [[media/icons/info.png]] This function just prints the error, it does not prevent the target -> environment from being constructed (use the [stop](#stop) command for that). - -### getenv -*Function* - - if getenv("REZ_MAYA_VERSION") == "2016.sp1": - pass - -Gets the value of an environment variable; raises *RexUndefinedVariableError* if not set. - -### implicits -*Dict-like object* - - if "platform" in implicits: - pass - -This is similar to the [request](#request) object, but it contains only the package requests as -defined by the [implicit_packages](Configuring-Rez#implicit_packages) configuration setting. - -### info -*Function* - - info("floob version is %s" % resolve.floob.version) - -Prints to standard out. - -### intersects -*Function* - - if intersects(resolve.maya, "2019+"): - info("Maya 2019 or greater is present") - -A boolean function that returns True if the version or version range of the given -object, intersects with the given version range. Valid objects to query include: - -* A resolved package, eg `resolve.maya`; -* A package request, eg `request.foo`; -* A version of a resolved package, eg `resolve.maya.version`; -* A resolved ephemeral, eg `ephemerals.foo`; -* A version range object, eg `ephemerals.get_range('foo.cli', '1')` - -> [[media/icons/warning.png]] Do **not** do this: -> `if intersects(ephemerals.get("foo.cli", "0"), "1"): ...` -> If 'foo.cli' is not present, this will unexpectedly compare the unversioned -> package named "0" against the version range "1", which will succeed! Use -> `get_range` when testing intersections on the _request_ and _ephemerals_ -> objects instead: -> `if intersects(ephemerals.get_range("foo.cli", "0"), "1"): ...` - -### literal -*Function* - - env.FOO = literal("this {root} will not expand") - -Inhibits expansion of object and environment variable references. You can also chain together -*literal* and *expandable* functions like so: - - env.FOO = literal("the value of {root} is").expandable("{root}") - -### optionvars -*Function* - -A `dict.get` like function for package accessing arbitrary data from `optionvars` in rez config. See the [optionvars](Configuring-Rez#optionvars) configuration setting for more details. - -### request -*Dict-like object* - - if "maya" in request: - info("maya was asked for!") - -A dict representing the list of package requests. Each item is a request string keyed by the -package name. For example, consider the package request: - - ]$ rez-env maya-2015 maya_utils-1.2+<2 !corelib-1.4.4 - -This request would yield the following *request* object: - - { - "maya": "maya-2015", - "maya_utils": "maya_utils-1.2+<2", - "corelib": "!corelib-1.4.4" - } - -Use `get_range` to test with the [intersects](Package-Commands#intersects) function: - - if intersects(request.get_range("maya", "0"), "2019"): - info("maya 2019.* was asked for!") - -> [[media/icons/info.png]] If multiple requests are present that refer to the same package, the -> request is combined ahead of time. In other words, if requests *foo-4+* and *foo-<6* were both -> present, the single request *foo-4+<6* would be present in the *request* object. - -### resolve -*Dict-like object* - - if "maya" in resolve: - info("Maya version is %s", resolve.maya.version) - # ..or resolve["maya"].version - -A dict representing the list of packages in the resolved environment. Each item is a -[Package](Package-Definition-Guide) object, keyed by the package name. - -### root -*String* - -See [this.root](#thisroot). - -### setenv -*Function* - - setenv("FOO_PLUGIN_PATH", "{root}/plugins") - -This function sets an environment variable to the given value. It is equivalent to setting a -variable via the *env* object (eg, "env.FOO = 'BAH'"). - -### source -*Function* - - source("{root}/scripts/init.sh") - -Source a shell script. Note that, similarly to *commands*, this function cannot return a value, and -any side effects that the script sourcing has is not visible to any packages. For example, if the -*init.sh* script above contained *"export FOO=BAH"*, a subsequent test for this variable on the -*env* object would yield nothing. - -### stop -*Function* - - stop("The value should be %s", expected_value) - -Raises an exception and stops a resolve from completing. You should use this when an unrecoverable -error is detected and it is not possible to configure a valid environment. - -### system -*System object* - - if system.platform == "windows": - ... - -This object provided system information, such as current platform, arch and os. See -[the source](https://github.com/__GITHUB_REPO__/blob/__GITHUB_BRANCH__/src/rez/system.py) for more info. - -### test -*Dict-like object* - - if test.name == "unit": - info("My unit test is about to run yay") - -This object is only available in the [pre_test_commands](#pre-test-commands) function. It has the -following fields: - -#### test.name -*String* - -Name of the test about to run. - -### this -*Package object* - - import os.path - env.PATH.append(os.path.join(this.root, "bin")) - -The *this* object represents the current package. The following attributes are most commonly used -in a *commands* section (though you have access to all package attributes - see -[here](Package-Definition-Guide)): - -#### this.base -*String* - -Similar to *this.root*, but does not include the variant subpath, if there is one. Different -variants of the same package share the same *base* directory. See [here](Variants) for more -information on package structure in relation to variants. - -#### this.name -*String* - -The name of the package, eg 'houdini'. - -#### this.root -*String* - -The installation directory of the package. If the package contains variants, this path will include -the variant subpath. This is the directory that contains the installed package payload. See -[here](Variants) for more information on package structure in relation to variants. - -#### this.version -*Version object* - -The package version. It can be used as a string, however you can also access specific tokens in the -version (such as major version number and so on), as this code snippet demonstrates: - - env.FOO_MAJOR = this.version.major # or, this.version[0] - -The available token references are *this.version.major*, *this.version.minor* and -*this.version.patch*, but you can also use a standard list index to reference any version token. - -### undefined -*Function* - - if undefined("REZ_MAYA_VERSION"): - info("maya is not present") - -Use this boolean function to determine whether or not an environment variable is set. This is the -opposite of [defined](#defined). - -### unsetenv -*Function* - - unsetenv("FOO_LIC_SERVER") - -Unsets an environment variable. This function does nothing if the environment variable was not set. - -### version -*Version object* - -See [this.version](#thisversion). diff --git a/wiki/pages/Package-Definition-Guide.md b/wiki/pages/Package-Definition-Guide.md deleted file mode 100644 index b39ea66b0..000000000 --- a/wiki/pages/Package-Definition-Guide.md +++ /dev/null @@ -1,925 +0,0 @@ -## Overview - -Packages are defined by a *package definition file*. This is typically a file named *package.py* -that is located in the root directory of each package install. For example, given package -repository location */packages/inhouse*, the package definition file for package "foo-1.0.0" would -be */packages/inhouse/foo/1.0.0/package.py*. - -Here is an example package definition file: - - name = 'sequence' - - version = '2.1.2' - - description = 'Sequence detection library.' - - authors = ['ajohns'] - - tools = [ - 'lsq', - 'cpq' - ] - - requires = [ - 'python-2.6+<3', - 'argparse' - ] - - def commands(): - env.PATH.append("{root}/bin") - env.PYTHONPATH.append("{root}/python") - - uuid = '6c43d533-92bb-4f8b-b812-7020bf54d3f1' - -## Package Attributes - -Every variable defined in the package definition file becomes an attribute on the built or -installed package. This includes attributes that are not in the -[standard list](#standard-package-attributes) - you can add any custom attribute to a package. - -Some variables are not, however, added as package attributes. Consider the following package -definition snippet: - - import sys - - description = "This package was built on %s" % sys.platform - -Here we do not want *sys* to become a package attribute, because providing a python module as a -package attribute is nonsensical. - -Python variables that do **not** become package attributes include: - -* Python modules; -* Functions, not include early- and late- binding functions (see next), and not including the - *commands* and related functions; -* Any variable with a leading double underscore; -* Any variable that is a [build-time package attribute](#build-time-package-attributes). - -### Function Attributes - -Package attributes can be implemented as functions - the return value of the function becomes -the attribute value. There are two types of attribute functions - *early binding* functions, -and *late binding* functions - and these are decorated using *@early* and *@late* respectively. - -> [[media/icons/warning.png]] The *commands* functions are an exception to the rule. They are -> late bound, but are not the same as a standard function attribute, and are *never* decorated -> with the early or late decorators. - -#### Early Binding Functions - -Early binding functions use the *@early* decorator. They are evaluated at *build time*, hence the -'early' in 'early binding'. Any package attribute can be implemented as an early binding function. - -Here is an example of an *authors* attribute that is automatically set to the contributors of the -package's git project: - - @early() - def authors(): - import subprocess - p = subprocess.Popen("git shortlog -sn | cut -f2", - shell=True, stdout=subprocess.PIPE) - out, _ = p.communicate() - return out.strip().split('\n') - -> [[media/icons/info.png]] You can assume that during evaluation of early binding functions, the -> current working directory is the root directory containing your *package.py*. - -An early bound function can also have access to other package attributes. To do this, use the -implicit *this* object: - - @early() - def description(): - # a not very useful description - return "%s version %s" % (this.name, this.version) - -> [[media/icons/warning.png]] Do not reference other early bound or late bound attributes in -> your early bound function - an error will be raised if you do. - -Early binding functions are a convenience - you can always use an arbitrary function instead, like so: - - def _description(): - return "%s version %s" % (this.name, this.version) - - description = _description() - -However, using early binding results in a package definition that is cleaner and more explicit - it -is clear that an attribute is intended to be evaluated at build time, and you avoid the need to -define an arbitrary function earlier in the python source. You can always use a combination of the -two as well - an early binding function can call an arbitrary function defined at the bottom of -your definition file. - -##### Available Objects - -Following is the list of objects that are available during early evaluation. - -* *building* - see [building](Package-Commands#building); -* *build_variant_index* - the index of the variant currently being built. This is only relevant if - `building` is True. -* *build_variant_requires* - the subset of package requirements specific to the variant - currently being built. This is a list of `PackageRequest` objects. This is only relevant if - `building` is True. -* *this* - the current package, as described previously. - -Be aware that early-bound functions are actually evaluated multiple times during a build - once -pre-build, and once per variant, during its build. This is necessary in order for early-bound -functions to change their return value based on variables like `build_variant_index`. Note that the -*pre-build* evaluated value is the one set into the installed package, and in this case, `building` -is False. - -An example of where you'd need to be aware of this is if you wanted the `requires` field to include -a certain package at runtime only (ie, not present during the package build). In this case, `requires` -might look like so: - - @early() - def requires(): - if building: - return ["python-2"] - else: - return ["runtimeonly-1.2", "python-2"] - -> [[media/icons/warning.png]] You **must** ensure that your early-bound function returns the value -> you want to see in the installed package, when `building` is False. - -#### Late Binding Functions - -Late binding functions stay as functions in the installed package definition, and are only evaluated -lazily, when the attribute is accessed for the first time (the return value is then cached). - -Not any attribute can be implemented as a late binding function. The allowed attributes are: - -* requires -* build_requires -* private_build_requires -* tools -* help -* any arbitrary attribute - -Here is an example of a late binding *tools* attribute: - - @late() - def tools(): - import os - - # get everything in bin dir - binpath = os.path.join(this.root, "bin") - result = os.listdir(binpath) - - # we don't want artists to see the admin tools - if os.getenv("_USER_ROLE") != "superuser": - result = set(result) - set(["delete-all", "mod-things"]) - - return list(result) - -> [[media/icons/warning.png]] Late binding function attributes *must* perform any necessary imports -> *within* the function, not at the top of the *package.py* file. - -Note that, if this function just returned the binaries found in the bin dir, it would have made -more sense to implement this as an *early binding* function - no code evaluation has to happen at -runtime then, so it's cheaper. However here a modification is made based on the value of the -*_USER_ROLE* environment variable, which isn't known at build time. - -If some information for an attribute could be calculated once at build time, you can reduce the -runtime cost by storing that part into an early binding arbitrary attribute. For example, we could -reimplement the above example like so: - - @late() - def tools(): - import os - result = this._tools - - # we don't want artists to see the admin tools - if os.getenv("_USER_ROLE") != "superuser": - result = set(result) - set(["delete-all", "mod-things"]) - - return list(result) - - @early() - def _tools(): - import os - return os.listdir("./bin") - -Note how in the *_tools* function we're referring to a relative path. Remember that early binding -functions are evaluated at build time - the package hasn't actually been built or installed yet, -so attributes such as *this.root* don't exist. - -##### The *in_context* Function - -When late binding functions are evaluated, a boolean function *in_context* is present, which -returns True if the package is part of a resolved context, or False otherwise. For example, -if you just use the rez API to iterate over packages (as the *rez-search* tool does), these -packages do not belong to a context; however if you create a *ResolvedContext* object (as -the *rez-env* tool does) and iterate over its resolved packages, these belong to a context. - -The in-context or not-in-context distinction is important, because often the package attribute -will need information from the context to give desired behavior. For example, consider the -late binding *tool* attribute below: - - @late() - def tools(): - result = ["edit"] - - if in_context() and "maya" in request: - result.append("maya-edit") - - return result - -Here the *request* object is being checked to see if the *maya* package was requested in the -current env; if it was, a maya-specific tool *maya-edit* is added to the tool list. - -> [[media/icons/warning.png]] Always ensure your late binding function returns a sensible -> value regardless of whether *in_context* is True or False. Otherwise, simply trying to -> query the package attributes (using *rez-search* for example) may cause errors. - -##### Available Objects - -Following is the list of objects that are available during late evaluation, if *in_context* -is *True*: - -* *context* - the *ResolvedContext* instance this package belongs to; -* *system* - see [system](Package-Commands#system); -* *building* - see [building](Package-Commands#building); -* *request* - see [request](Package-Commands#request); -* *implicits* - see [implicits](Package-Commands#implicits). - -The following objects are available in *all* cases: - -* *this* - the current package/variant (see note below); -* *in_context* - the *in_context* function itself. - -> [[media/icons/warning.png]] The *this* object may be either a package or a variant, -> depending on the situation. For example, if *in_context* is True, then *this* is a -> variant, because variants are the objects present in a resolved context. On the other -> hand, if a package is accessed via API (for example, by using the *rez-search* tool), -> then *this* may be a package. The difference matters, because variants have some -> attributes that packages don't - notably, *root* and *index*. Use the properties -> `this.is_package` and `this.is_variant` to distinguish the case if needed. - -##### Example - Late Bound build_requires - -Here is an example of a package.py with a late-bound `build_requires` field: - - name = "maya_thing" - - version = "1.0.0" - - variants = [ - ["maya-2017"], - ["maya-2018"] - ] - - @late() - def build_requires(): - if this.is_package: - return [] - elif this.index == 0: - return ["maya_2017_build_utils"] - else: - return ["maya_2018_build_utils"] - -Note the check for `this.is_package`. This is necessary, otherwise the evaluation would -fail in some circumstances. Specifically, if someone ran the following command, the `this` -field would actually be a `Package` instance, which doesn't have an `index`: - - ]$ rez-search maya_thing --type package --format '{build_requires}' - -In this case, `build_requires` is somewhat nonsensical (there is no common build requirement -for both variants here), but something needs to be returned nonetheless. - -## Sharing Code Across Package Definition Files - -It is possible to share common code across package definition function attributes, but the -mechanism that is used is different depending on whether a function is early binding or late -binding. This is to avoid installed packages being dependent on external code that may change -at any time; builds being dependent on external code is not problematic however. - -### Sharing Code During A Build - -Functions in a *package.py* file which are evaluated at build time include: - -* The *preprocess* function; -* Any package attribute implemented as a function using the *@early* decorator. - -You expose common code to these functions by using the -[package_definition_build_python_paths](Configuring-Rez#package_definition_build_python_paths) -config setting. - -### Sharing Code Across Installed Packages - -Functions that are evaluated in installed packages' definition files include: - -* The various *commands* functions; -* Any package attribute implemented as a function using the *@late* decorator. - -You expose common code to these functions by using the *@include* decorator, which relies on the -[package_definition_python_path](Configuring-Rez#package_definition_python_path) config setting. -The module source files are actually copied into each package's install payload, so the package -stays self-contained, and will not break or change behavior if the original modules' source -files are changed. The downside though, is that these modules are not imported, and they themselves -cannot import other modules managed in the same way. - -Here is an example of a package's *commands* using a shared module: - - # in package.py - @include("utils") - def commands(): - utils.set_common_env_vars(this, env) - -## Requirements Expansion - -Often a package may be compatible with a broader range of its dependencies at build time than it is -at runtime. For example, a C++ package may build against any version of *boost-1*, but may -then need to link to the specific minor version that it was built against, say *boost-1.55*. - -You can describe this in your package's *requires* attribute (or any of the related attributes, -such as *build_requires*) by using wildcards as shown here: - - requires = [ - "boost-1.*" - ] - -If you check the *package.py* of the built package, you will see that the boost reference in the -requires list will be expanded to the latest found within the given range ("boost-1.55" for example). - -There is also a special wilcard available - *"**"*. This expands to the full package version. For -example, the requirement *boost-1.*** might expand to *boost-1.55.1*. - -You can also achieve requirements expansion by implementing *requires* as an early binding -function (and you may want to use some variation of this to generate *variants* for example), and -using the rez *expand_requires* function: - - @early() - def requires(): - from rez.package_py_utils import expand_requires - return expand_requires(["boost-1.*"]) - -## Package Preprocessing - -You can define a *preprocessing* function either globally or in a *package.py*. This can be used to -validate a package, or even change some of its attributes, before it is built. To set a global -preprocessing function, see the -[package_preprocess_function](Configuring-Rez#package_preprocess_function) config setting. - -Consider the following preprocessing function, defined in a *package.py*: - - def preprocess(this, data): - from rez.package_py_utils import InvalidPackageError - import re - - if not re.match("[a-z]+$", this.name): - raise InvalidPackageError("Invalid name, only lowercase letters allowed") - - if not this.authors: - from preprocess_utils import get_git_committers - data["authors"] = get_git_committers() - -This preprocessor checks the package name against a regex; and sets the package authors list to its -git committers, if not already supplied in the *package.py*. To update package attributes, you have -to update the given *data* dict, *not* the *package* instance. - -To halt a build because a package is not valid, you must raise an *InvalidPackageError* as shown -above. - -> [[media/icons/info.png]] To see the preprocessed contents of a package.py, run the command -> *"rez-build --view-pre"* in the source root directory. This will just print the preprocessed -> package to standard out, then exit. - -### Overriding Config Settings In Preprocessing - -It is not uncommon to override config settings such as the release path in a package, like so: - - # in package.py - with scope("config") as c: - c.release_packages_path = "/software/packages/external" - -Let's say we have a scenario where we want to install third party packages to a specific install -path, and that we set the arbitrary attribute *external* to True for these packages. We could do -this with a global preprocessing function like this: - - def preprocess(this, data): - if not data.get("external"): - return - - try: - _ = data["config"]["release_packages_path"] - return # already explicitly specified by package - except KeyError: - pass - - data["config"] = data.get("config", {}) - data["config"]["release_packages_path"] = "/software/packages/external" - -The *"with scope(...)"* statement is just a fancy way of defining a dict, so you can do the same -thing in the preprocess function simply by updating the *config* dict within *data*. - -## Example Package - -Here is an example package definition, demonstrating several features. This is an example of a -python package which, instead of actually installing python, detects the existing system python -installation instead, and binds that into a rez package. - - name = "python" - - @early() - def version(): - return this.__version + "-detected" - - authors = [ - "Guido van Rossum" - ] - - description = \ - """ - The Python programming language. - """ - - @early() - def variants(): - from rez.package_py_utils import expand_requires - requires = ["platform-**", "arch-**", "os-**"] - return [expand_requires(*requires)] - - @early() - def tools(): - version_parts = this.__version.split('.') - - return [ - "2to3", - "pydoc", - "python", - "python%s" % (version_parts[0]), - "python%s.%s" % (version_parts[0], version_parts[1]) - ] - - uuid = "recipes.python" - - def commands(): - env.PATH.append("{this._bin_path}") - - if building: - env.CMAKE_MODULE_PATH.append("{root}/cmake") - - # --- internals - - def _exec_python(attr, src): - import subprocess - - p = subprocess.Popen( - ["python", "-c", src], - stdout=subprocess.PIPE, stderr=subprocess.PIPE) - out, err = p.communicate() - - if p.returncode: - from rez.exceptions import InvalidPackageError - raise InvalidPackageError( - "Error determining package attribute '%s':\n%s" % (attr, err)) - - return out.strip() - - @early() - def _bin_path(): - return this._exec_python( - "_bin_path", - "import sys, os.path; print(os.path.dirname(sys.executable))") - - def _version(): - return _exec_python( - "version", - "import sys; print(sys.version.split()[0])") - - __version = _version() - -Note the following: - -* *variants* is implemented as an early bound attribute, and uses *requirements expansion* to - dynamically define the variant requirements. Even though only the *requires* and related attributes - natively expand wildcards, you can still use the *expand_requirement* function yourself, as - illustrated here. -* A *_version* function has been defined, and its return value stored into the *__version* variable. - This is done because two other early binding attributes - *version* and *tools* - use this value, - and we avoid calling the function twice. Both *_version* and *__version* are later stripped from - the package, because one is a normal function, and the other has double leading underscores. -* An arbitrary attribute *_bin_path* has been defined, and implemented as an early bound attribute. - The *commands* function then uses this value. In this example, it was far better to take this - approach than the alternative - running the python subprocess in the *commands* function. Doing that - would have been very costly, since commands are executed every time a new environment is created - (and launching a subprocess is slow). Instead, here we take this cost at build time, and cache the - result into the package attribute. -* Common code was provided in the normal function *_exec_python*, which will be stripped from the - installed package. - -## Standard Package Attributes - -Following is a list, in alphabetical order, of every standard attribute that a user can define in a -package definition file (you can also define your own arbitrary attributes). Each entry specifies -the data type, and includes a code snippet. - -### authors -*List of string* - - authors = ["jchrist", "sclaus"] - -Package authors. Should be in order, starting with the major contributor. - -### build_requires -*List of string* - - build_requires = [ - "cmake-2.8", - "doxygen" - ] - -This is the same as *requires*, except that these dependencies are only included during a build -(typically invoked using the *rez-build* tool). - -### cachable -*Boolean* - - cachable = True - -Determines whether a package can be cached when [package caching](Package-Caching#Overview) is -enabled. If not provided, this is determined from the global config setting [default_cachable](Configuring-Rez#default_cachable) and related `default_cachable_*` settings. - -### commands -*Function* - - def commands(): - env.PYTHONPATH.append("{root}/python") - env.PATH.append("{root}/bin") - -This is a block of python code which tells rez how to update an environment so that this package -can be used. It is executed when the package is brought into a rez environment, either by explicit -request or by another package's requirements. There is a python API provided (see -[here](Package-Commands) for more details) that lets you do things such as: - -* set, unset, prepend and append environment variables; -* create aliases; -* source scripts; -* print messages. - -In this example, the 'foo' package is appending a path to *PYTHONPATH*, and appending a path to -*PATH*. The special string "{root}" will expand out to the install location of the package. This is -a fairly typical example. - -### config - - with scope("config"): - release_packages_path = "/software/packages/apps" - -Packages are able to override rez configuration settings. This is useful in some cases; for example, -we may want a package to release to a different directory than the default (as this example shows). -See [here](Configuring-Rez) for more details. - -### description -*String* - - description = "Library for communicating with the dead." - -This is a general description of the package. It should not mention details about a particular -version of the package, just about the package in general. - -### has_plugins -*Boolean* - - has_plugins = True - -Indicates that the package is an application that may have plugins. These plugins are often made -available as rez packages also. - -### hashed_variants -*Boolean* - - hashed_variants = True - -Instructs the package to install variants into a subdirectory based on a hash of the variant's -contents (its requirements in other words). This is useful for variants with a high number of -requirements, or with requirements that do not translate well to directories on the filesystem -(such as conflict requirements). - -### help -*String or List of String* - - help = "https://github.com/__GITHUB_REPO__/wiki" - -URL for package webpage, or, if a string containing spaces, a command to run. You can show the help -for a package using the *rez-help* command line tool. If this value is a list, then this represents -multiple help entries, and you can specify the entry you want to see using the `SECTION` argument. - -### name -*String (mandatory)* - - name = "maya_utils" - -This is the name of the package. Alphanumerics and underscores are allowed. Name is case sensitive. - -### plugin_for -*String* - - plugin_for = "maya" - -Provided if this package is a plugin of another package. For example, this might be a maya plugin. - -### post_commands -*Function* - - def post_commands(): - env.FOO_PLUGIN_PATH.append("@") - -Similar to *pre_commands*, but runs in a final phase rather than the first. See that attribute for -further details. - -### pre_commands -*Function* - - def pre_commands(): - import os.path - env.FOO_PLUGIN_PATH = os.path.join(this.root, "plugins") - -This is the same as *commands*, except that all packages' *pre_commands* are executed in a first -pass; then, all *commands* are run in a second; and lastly, *post_commands* are all run in a third -phase. It is sometimes useful to ensure that some of a package's commands are run before, or after -all others, and using pre/post_commands is a way of doing that. - -### pre_test_commands -*Function* - - def pre_test_commands(): - if test.name == "unit": - env.IS_UNIT_TEST = 1 - -This is similar to *commands*, except that it is run prior to each test defined in -[tests](#tests) -See [here](Package-Commands#pre-test-commands) for more details. - -### relocatable -*Boolean* - - relocatable = True - -Determines whether a package can be copied to another package repository (using the `rez-cp` tool for -example). If not provided, this is determined from the global config setting [default_relocatable](Configuring-Rez#default_relocatable) and related `default_relocatable_*` settings. - -### requires -*List of string* - - requires = [ - "python-2", - "maya-2016", - "maya_utils-3.4+<4" - ] - -This is a list of other packages that this package depends on. A rez package should list all the -packages it needs - someone should be able to use your package without needing to know about how it -works internally - and this includes needing to know its dependencies. - -Rez has a syntax for these package requests. For example, "python-2.6" is a package request which -covers the range of all python packages starting with 2.6 - for example, "python-2.6.0", -"python-2.6.4" (it is not simply a prefix - "python-2.65" is not within the request). When you -request a package, you are asking rez for any version within this request, although rez will aim to -give you the latest possible version. - -For more details on request syntax, see [here](Basic-Concepts#package-requests). - -### tests -*Dict* - - tests = { - "unit": "python -m unittest discover -s {root}/python/tests", - "lint": { - "command": "pylint mymodule", - "requires": ["pylint"], - "run_on": ["default", "pre_release"] - }, - "maya_CI": { - "command": "python {root}/ci_tests/maya.py", - "on_variants": { - "type": "requires", - "value": ["maya"] - }, - "run_on": "explicit" - } - } - -This is a dict of tests that can be run on the package using the *rez-test* tool. For example, to -run a linter on the package *maya_utils* with the *test* attribute above, you would simply run: - - ]$ rez-test maya_utils lint - -If a test entry is a string or list of strings, this is interpreted as the command to run. Command -strings will expand any references to package attributes, such as *{root}*. - -If you provide a nested dict, you can specify extra fields per test, as follows: - -* **requires**: Extra package requirements to include in the test's runtime env. -* **run_on**: When to run this test. Valid values are: - * `default` (the default): Run when `rez-test` is run with no `TEST` args specified. - * `pre_install`: Run before an install (ie `rez-build -i`), and abort the install on fail. - * `pre_release`: Run before a release, and abort the release on fail. - * `explicit`: Only run if specified as `TEST` when `rez-test` is run. -* **on_variants**: Which variants the test should be run on. Valid values are: - * True: Run the test on all variants. - * False (the default): Run the test only on one variant (ie the variant you get by - default when the test env is resolved). This is useful for tests like linting, - where variants may be irrelevant. - * A dict. This is a variant selection mechanism. In the example above, the "maya_CI" test will - run only on those variants that directly require `maya` (or a package within this range, eg - `maya-2019`). Note that "requires" is the only filter type currently available. - -### tools -*List of string* - - tools = [ - "houdini", - "hescape", - "hython" - ] - -This is a list of tools that the package provides. This entry is important later on when we talk -about [suites](Suites#suite-tools). - -### uuid -*String* - - uuid = "489ad32867494baab7e5be3e462473c6" - -This string should uniquely identify this *package family* - in other words, all the versions of a -particular package, such as 'maya'. It is used to detect the case where two unrelated packages that -happen to have the same name are attempted to be released. If rez detects a uuid mismatch, it will -abort the release. - -You should set the uuid on a new package once, and not change it from then on. The format of the -string doesn't actually matter, but you'd typically use a true UUID, and you can generate one -like so: - - ]$ python -c 'import uuid; print(uuid.uuid4().hex)' - -### variants -*List of list of string* - - variants = [ - ["maya-2015.3"], - ["maya-2016.1"], - ["maya-2016.7"] - ] - -A package can contain *variants* - think of them as different flavors of the same package version, -but with differing dependencies. See the [variants chapter](Variants) for further details. - -### version -*String* - - version = "1.0.0" - -This is the version of the package. See [here](Basic-Concepts#versions) for further details on valid -package versions. - - -## Build Time Package Attributes - -The following package attributes only appear in packages to be built; they are stripped from the -package once installed because they are only used at build time. - -### build_command -*String or False* - - build_command = "bash {root}/build.sh {install}" - -Package build command. If present, this is used as the build command when *rez-build* is run, -rather than detecting the build system from present build scripts (such as *CMakeLists.txt*). If -*False*, this indicates that no build step is necessary (the package definition will still be -installed, and this is enough to define the package). - -The *{root}* string expands to the root directory of the package (where the package.py is -contained). Note that, like all builds, the working directory is set to the *build path*, which -is typically somewhere under a *build* subdirectory, and is where build outputs should go. - -The *{install}* string expands to "*install*" if an installation is occurring, or the empty string -otherwise. This is useful for passing the install target directly to the command (for example, when -using *make*) rather than relying on a build script checking the *REZ_BUILD_INSTALL* environment -variable. - -The full set of variables that can be referenced in the build command are: - -* *root*: (see above); -* *install*: (see above) -* *build_path*: The build path (this will also be the current working directory); -* *install_path*: Full path to install destination; -* *name*: Name of the package getting built; -* *variant_index*: Index of the current variant getting built, or an empty - string ('') if no variants are present. -* *version*: Package version currently getting built. - -### build_system -*String* - - build_system = "cmake" - -Specify the build system used to build this package. If not set, it is detected automatically when -a build occurs (or the user specifies it using the `--build-system` option). - -### pre_build_commands -*Function* - - def pre_build_commands(): - env.FOO_BUILT_BY_REZ = 1 - -This is similar to *commands*, except that it is run _prior to the current package being built_. -See [here](Package-Commands#pre-build-commands) for more details. - -### preprocess -*Function* - -See [Package Preprocessing](#package-preprocessing) - -### private_build_requires -*List of string* - - private_build_requires = [ - "cmake-2.8", - "doxygen" - ] - -This is the same as *build_requires*, except that these dependencies are only included if this -package is being built. Contrast this with *build_requires*, whose dependencies are included if a -build is occurring - regardless of whether this package specifically is being built, or whether -this package is a dependency of the package being built. - -### requires_rez_version -*String* - - requires_rez_version = "2.10" - -This defines the minimum version of rez needed to build this package. New package features have -been added over time, so older rez versions cannot necessarily build newer packages. - -## Release Time Package Attributes - -The following package attributes are created for you by Rez when your package is released via the -*rez-release* tool. If you look at the released *package.py* file you will notice that some or all -of these attributes have been added. - -### changelog -*String* - - changelog = \ - """ - commit 22abe31541ceebced8d4e209e3f6c44d8d0bea1c - Author: allan johns <> - Date: Sun May 15 15:39:10 2016 -0700 - - first commit - """ - -Change log containing all commits since the last released package. If the previous release was from -a different branch, the changelog given will go back to the last common commit ancestor. The syntax -of this changelog depends on the version control system; the example here is from a *git*-based -package. - -### previous_revision -*Type varies* - -Revision information of the previously released package, if any (see *revision* for code example - -the code for this attribute is the same). - -### previous_version -*String* - - previous_version = "1.0.1" - -The version of the package previously released, if any. - -### release_message -*String* - - release_message = "Fixed the flickering thingo" - -The package release message. This is supplied either via the *rez-release* tool's *--message* -option, or was entered in a text editor on release if rez is configured to do this (see the config -setting 'TODO_ADD_THIS'). A package may not have a release message. - -### revision -*Type varies* - - revision = \ - {'branch': 'main', - 'commit': '22abe31541ceebced8d4e209e3f6c44d8d0bea1c', - 'fetch_url': 'git@github.com:foo/dummy.git', - 'push_url': 'git@github.com:foo/dummy.git', - 'tracking_branch': 'origin/main'} - -Information about the source control revision containing the source code that was released. The -data type is determined by the version control system plugin that was used. The example code shown -here is the revision dict from a *git*-based package. - -### timestamp -*Integer* - - timestamp = 1463350552 - -Epoch time at which the package was released. - -### vcs -*String* - - vcs = "git" - -Name of the version control system this package was released from. diff --git a/wiki/pages/Pip.md b/wiki/pages/Pip.md deleted file mode 100644 index 060fb928d..000000000 --- a/wiki/pages/Pip.md +++ /dev/null @@ -1,132 +0,0 @@ -## Overview - -Rez is language agnostic. -But since python is so much used in the VFX industry (and outside), -it knows how to convert/translate it into a rez package. -To do so, it provides a `rez-pip` command. - -> [[media/icons/warning.png]] It doesn't know how to translate its own packages -into pip packages. - -### Usage - -``` -usage: rez pip [-h] [--python-version VERSION] [--pip-version VERSION] [-i] - [-s] [-r] [-v] - PACKAGE - -Install a pip-compatible python package, and its dependencies, as rez -packages. - -positional arguments: - PACKAGE package to install or archive/url to install from - -optional arguments: - -h, --help show this help message and exit - --python-version VERSION - python version (rez package) to use, default is - latest. Note that the pip package(s) will be installed - with a dependency on python-MAJOR.MINOR. - --pip-version VERSION - pip version (rez package) to use, default is latest. - This option is deprecated and will be removed in the - future. - -i, --install install the package - -s, --search search for the package on PyPi - -r, --release install as released package; if not set, package is - installed locally only - -p PATH, --prefix PATH - install to a custom package repository path. - -v, --verbose verbose mode, repeat for more verbosity -``` - -The `rez-pip` command allows you to do two main things. - -1. Install or release a pip package as a rez package. -2. Search for a package on PyPI - -### Which pip will be used? - -`rez-pip` uses a fallback mechanism to find which pip will be used to run the commands. -The logic is as follow: - -1. Search for pip in the rezified `python` package specified with `--python-version`, or -the latest version if not specified; -2. If found, use it; -3. If not found, search for pip in the rezified `pip` package specified with `--pip-version`, -or latest version if not specified. - - **Note that this is deprecated and will be removed in the future**; - -4. If rezified `pip` is found, use it; -5. If not found, fall back to pip installed in rez own virtualenv. - -> [[media/icons/info.png]] In all of these options, we also check if the version of pip is greater -or equal than 19.0. This is a hard requirement of `rez-pip`. - -Note that rez-pip output should tell you what it tries and which pip it will use. - -It is extremely important to know which pip is used and understand why it is used. Pip packages -define which python version they are compatible with. -When you install a pip package, pip checks which python version it is -currently running under to determine if a package can be downloaded and installed. -Not only this but it also checks which python implementation is used (CPython, PyPy, -IronPython, etc), the architecture python was built with, and other variables. So the thing you -really need to know first is which python you want to use and from there you should know -which pip is used. Knowing the pip version comes in second place. - -At some point, we supported the `--pip-version` argument, but considering what was just said -above, we decided to deprecate it (but not yet removed) just for backward compatibility reasons. -Pip is too much (read tightly) coupled to the python version/interpreter it is installed with -for us to support having pip as a rez package. We just can't garantee that pip can be -install once in a central way and work with multiple different python version, and potentially -different implementations. - -### How should I install pip? - -Following the [Which pip will be used?](#which-pip-will-be-used) section, we recommend to install -pip inside your python packages. For Python 2, this can be done when you compile it with the -`--with-ensurepip` flag of the `configure` script. This will install a version older than 19.0 -though, so you will need to upgrade it. For Python 3, it is already installed by default. -Though your milleage may vary for the version installed, depending on which Python version you -installed. So check the pip version and update it if necessary. We also encourage you -to install `wheel` and possibly update `setuptools`. `pip`, `setuptools` and `wheel` -are perfectly fine when installed in the interpreter directly as they are pretty core -packages and all have no dependencies (and that's what `virtualenv` does by default too). - -Tip: When installing something in an interpreter, make sure you really install in this interpreter. -That means using somehting like: - -``` -/path/to/python -E -s -m pip install -``` - -`-E` will render any `PYTHON*` environment variable to not be used and `-s` will -remove your [user site](https://docs.python.org/3.7/library/site.html) from the equation. - -#### Install/release - -You have two options when you want to convert a pip package to a rez package. You can -install it, or release it. Install means that it will install in your -[local_packages_path](Configuring-Rez#local_packages_path), while -release means it will be installed in your [release_packages_path](Configuring-Rez#release_packages_path). -You can also specify a custom installation location using `--prefix` (or `-p`). - - -You can (and we recommend) use the `--python-version` to choose for which python -version you want to install a given package. This will make `rez-pip` to resolve -the given version of the `python` rez package and use it to run the `pip install`. -See [Which pip will be used?](#which-pip-will-be-used) for more details. -If the pip package is not pure (so contains `.so` for example), you will need to -call `rez-pip` for each python version you want to install the pip package for. - -> [[media/icons/warning.png]] `--pip-version` is deprecated and will be removed in the future. -> See [How should I install pip?](#how-should-i-install-pip) on how we recommend -> to install pip from now on. - - -#### Search - -With `rez-pip --search `, you can search for a package in PyPI. The main -advantage of using it over using `pip search ` is that `rez-pip --search` -uses the same logic as `rez-pip --install` and `--release` to find which pip to use. diff --git a/wiki/pages/Suites.md b/wiki/pages/Suites.md deleted file mode 100644 index fee142441..000000000 --- a/wiki/pages/Suites.md +++ /dev/null @@ -1,150 +0,0 @@ -## Overview - -Let us say that you wish to provide a number of different tools to your users, -even though these tools may require being run in different environments. For -example, you may want artists at your studio to be able to run *maya* and *nuke* -from the command line, without needing to know that they execute within different -environments. - -Let's say that in order to do this, you create two *contexts* - *maya.rxt* and -*nuke.rxt* (see [here](Contexts#baking-resolves) for more information). In -order to run maya, you would do this: - - ]$ rez-env --input maya.rxt -- maya - -You may then decide to wrap this command within a wrapper script which is also -called *maya*, and that might look something like this: - - #!/bin/bash - rez-env --input maya.rxt -- maya $* - -Now, if you put this somewhere on *$PATH*, and do the same for *nuke*, then -voila, your artists can run these applications from the command line, without -needing to know what's happening under the hood. - -This, in a nutshell, is what a *suite* does. A suite is simply a directory -containing a set of contexts, and wrapper scripts which run tools within those -contexts. - -## The rez-suite Tool - -Let's go through the same example, this time using the *rez-suite* tool. - -First, we create the suite. This creates a directory called *mysuite* in the -current working directory: - - ]$ rez-suite --create mysuite - -Now we need to add contexts to our suite. First we create the contexts: - - ]$ rez-env maya-2016.2 --output maya.rxt - ]$ rez-env nuke --output nuke.rxt - -Then, we add these contexts to the suite (note that the *--context* arg just -gives each context a label - you would typically have this match the context -filename as shown). - - ]$ rez-suite --add maya.rxt --context maya mysuite - ]$ rez-suite --add nuke.rxt --context nuke mysuite - -The suite is created! Now all we need to do is to activate it, and that's as -simple as adding its *bin* path to *$PATH*: - - ]$ export PATH=$(pwd)/mysuite/bin:$PATH - -You should now see your tools coming from the suite: - - ]$ which maya - ./mysuite/bin/maya - - ]$ ls ./mysuite/bin - maya - nuke - -## Suite Tools - -The tools in a context which are exposed by the suite is determined by the -[tools](Package-Definition-Guide#tools) package attribute. For example, the -*maya* package might have a *tools* definition like so: - - # in maya package.py - tools = [ - "maya", - "mayapy", - "fcheck" - ] - -All these tools would be made available in the suite (although you can explicitly -hide tools - see the *rez-suite* *--hide* option). - -> [[media/icons/warning.png]] Only packages listed in the context *requests*, -> that are not weak or conflict requests, have their tools exposed - packages -> pulled in as dependencies do not. If you need to control the version of a package -> not in the request, without adding its command line tools, just add it as a weak -> reference to the request list. - -### Tool Aliasing - -Tools can be aliased to different names, either explicitly (on a per-tool basis), -or by applying a common prefix or suffix to all tools in a context. - -Prefixing/suffixing is particularly useful when you want to expose the same -package's tools, but in two or more contexts. For example, you may want to run a -stable version of maya, but also a newer beta version. These would run in -different contexts, and the beta context might prefix all tools with *_beta*, -hence making available tools such as *maya_beta*. - -For example, here we create a context with a newer version of maya, add it to -the suite, then add a suffix to all its tools: - - ]$ rez-env maya-2017 --output maya2017.rxt - ]$ rez-suite --add maya2017.rxt --context maya2017 mysuite - ]$ rez-suite --suffix _beta --context maya2017 mysuite - -### Control Arguments - -When using suite tools, any arguments passed to the wrappers are passed through -to the underlying tool, as expected. However, there is an exception to the case - -rez provides a set of *control* arguments, which are prefixed with `+`/`++` -rather than the typical `-`/`--`. These are suite-aware arguments that pass -directly to rez. You can get a listing of them using `+h`/`++help`, like so: - -``` -]$ maya ++help -usage: maya [+h] [+a] [+i] [+p [PKG [PKG ...]]] [++versions] - [++command COMMAND [ARG ...]] [++stdin] [++strict] [++nl] - [++peek] [++verbose] [++quiet] [++no-rez-args] - -optional arguments: - +h, ++help show this help message and exit - +a, ++about print information about the tool - +i, ++interactive launch an interactive shell within the tool's - configured environment - +p [PKG [PKG ...]], ++patch [PKG [PKG ...]] - run the tool in a patched environment - ++versions list versions of package providing this tool - ++command COMMAND [ARG ...] - read commands from string, rather than executing the - tool - ++stdin read commands from standard input, rather than - executing the tool - ++strict strict patching. Ignored if ++patch is not present - ++nl, ++no-local don't load local packages when patching - ++peek diff against the tool's context and a re-resolved copy - - this shows how 'stale' the context is - ++verbose verbose mode, repeat for more verbosity - ++quiet hide welcome message when entering interactive mode - ++no-rez-args pass all args to the tool, even if they start with '+' -``` - -For example, to see information about the suite wrapper: - - ]$ maya ++about - Tool: maya - Path: ./mysuite/bin/maya - Suite: ./mysuite - Context: ./mysuite/contexts/maya2016.rxt ('maya2016') - -> [[media/icons/info.png]] If the target tool also uses `+` for some of its -> own arguments, you can change the prefix character that rez uses for its -> control arguments. See the *rez-suite* *--prefix-char* option. diff --git a/wiki/pages/Variants.md b/wiki/pages/Variants.md deleted file mode 100644 index 7c2708708..000000000 --- a/wiki/pages/Variants.md +++ /dev/null @@ -1,226 +0,0 @@ -## Overview - -Packages in rez can contain different *variants*. Think of these as different -flavors of the same package version. Each variant has one or more package -dependencies that differ to the other variants in the same package. - -Use of variants is best illustrated with an example. Consider a maya plugin, -*my_maya_plugin*. Let us assume that there are two active versions of maya -currently in use at your studio - *2016.sp2* and *2017*. If your plugin is compiled, -you may need to build it separately for each maya version, even though the source -is no different. - -You would use *variants* to create a version of your plugin that will work with -both maya versions. Consider the following package definition for our plugin: - - name = "my_maya_plugin" - - version = "1.0.0" - - requires = [ - "openexr-2.2" - ] - - variants = [ - ["maya-2016.sp2"], - ["maya-2017"] - ] - -When you build and install this package, two separate builds will occur - one -using *maya-2016.sp2*, and the other *maya-2017*. When an environment is resolved -that includes *my_maya_plugin*, the correct variant will be selected depending on -the version of maya present. Only one variant of a package is ever used in a -given configured environment. - -Each variant entry is a list of dependencies, no different to the packages listed -in the *requires* field. These dependencies are appended to the *requires* list -for each variant. Thus the first variant requires *openexr-2.2* and *maya-2016.sp1*, -and the second variant requires *openexr-2.2* and *maya-2017*. - -## Disk Structure - -Package variants are stored within the package, under subdirectories that match -the variant requirements. For example, continuing on with our *my_maya_plugin* -package, the installation of that package would look like so: - - /rez/packages/my_maya_plugin/1.0.0/maya-2016.sp2/ - /maya-2017/ - -The anatomy of a package with variants is illustrated in the following diagram: - -

- - -

- -The *root* of a package is the root directory of its current variant (the one -the current environment is configured to use); the *base* of a package is the -directory containing its variants. In a package that does not have variants, -*base* and *root* are the same. - -## Hashed Variants - -There are two problems with the variant subpath as illustrated above: -* The variant install path can become long if there are many requirements; -* If some variant requirements contain characters such as `!` and `<`, they - can cause escaping problems that affect build systems; and, depending on the - platform, may not be a valid filesystem path. - -You can avoid these issues by using _hashed variants_. This sets the variant -subpath to a hash of its requirements, rather than the requirements themselves. -The resulting subdirectly is somewhat unwieldy (example: -`83e0c415db1b602f9d59cee028da6ac785e9bacc`). However, another feature - -_variant shortlinks_ - deals with this. A shortlink is a symlink to each variant, -created in a separate subdirectory (default `_v`). - -Here is an example hashed variant path: - - /rez/packages/my_maya_plugin/1.0.0/83e0c415db1b602f9d59cee028da6ac785e9bacc - -Here is the matching _shortlink_, which is what will be used in a resolved -environment: - - /rez/packages/my_maya_plugin/1.0.0/_v/a - -Hashed variants must be enabled explicitly for a package. To do this, simply set -this in your package definition: - - hashed_variants = True - - -## Platform As Variant - -It is not uncommon to see the platform, architecture and/or operating system -packages in package variants (recall that rez represents these as packages). For -example, you might see variants like this: - - # in package.py - variants = [ - ["platform-linux", "arch-x86_64", "os-Ubuntu-12.04"], - ["platform-linux", "arch-x86_64", "os-Ubuntu-16.04"] - ] - -This indicates that the package has been built for multiple platforms. The correct -variant will be selected for you, because you probably have the relevant -[implicit packages](Basic-Concepts#implicit-packages) set to limit packages to -the current platform. - -## Single Variants - -You may often see packages with just one variant. There are two reasons for this: - -* *Future proofing*. Let's say you have a compiled package that links against python. - It may currently support *python-2.7*, however it's conceivable that support for - newer python versions may be added later. It is not possible to add new variants - to a package that does not have any; so by adding the *python-2.7* variant now, - you can add variants later without needing to move to a newer version. -* *Installation path*. People often expect to see platform, architecture and/or - operating system information in the installation path of a piece of software (and - may also expect the same of python version, or other core packages). By putting - these dependencies into a variant, we ensure that they appear in the installation - path of the package. - -## Variant Selection - -As mentioned, rez will automatically select the correct variant of a package -depending on the environment being resolved. For example, consider: - - ]$ rez-env my_maya_plugin maya-2017 -- echo '$REZ_MY_MAYA_PLUGIN_ROOT' - /rez/packages/my_maya_plugin/1.0.0/maya-2017 - ]$ rez-env my_maya_plugin maya-2016 -- echo '$REZ_MY_MAYA_PLUGIN_ROOT' - /rez/packages/my_maya_plugin/1.0.0/maya-2016.sp2 - -You can see how the correct variant (ie the one that does not conflict with other -packages in the request) has been selected. But what if both variants are valid -for the given request? Consider: - - ]$ rez-env my_maya_plugin -- echo '$REZ_MY_MAYA_PLUGIN_ROOT' - /rez/packages/my_maya_plugin/1.0.0/maya-2017 - -Here *maya* was not in the request. Either variant of *my_maya_plugin* would have -satisfied the request, since we have not specified which version of maya we actually -want. - -By default, rez will prefer the variant with the higher-versioned packages, which -is why the *maya-2017* variant was selected in this example. If there are -multiple packages in the variant, priority is given to those that were in the request -list, if any; following that, priority is given to packages listed earlier in the -variant. For example, consider: - - name = "foo" - - variants = [ - ["python-2.6", "maya-2017"], - ["python-2.7", "maya-2016"] - ] - -If I run "rez-env foo", which variant will I get? The answer is not clear. In this -case it will be the second variant, since *python* is given priority (it is the -first listed package in the variant), and the second variant has the higher version -of python. However, if I ran "rez-env foo maya", I would get the *first* variant - -because priority is now given to *maya*, because it's listed in my request, and the -first variant has the higher version of maya. - -The rez setting -[variant_select_mode](Configuring-Rez#variant_select_mode) affects this selection behavior. -The default mode just described is *version_priority*, but there is another mode - -*intersection_priority*. In this mode, variants are preferred that have *the most -number of packages present in the request*; version priority is secondary. - -### Mutual Exclusivity - -In all the examples we've seen so far, a package's variants have been mutually -exclusive. For example, you cannot have both *python-2.6* and *python-2.7* in the -same environment, so when we request "foo python-2.6" we can be sure of which -variant we will get. - -Variants, however, do not need to be mutually exclusive. In fact, you may use -variants in order to provide support for different DCCs for your package. Consider -a package with the following variants: - - name = "geocache" - - variants = [ - ["maya-2016"], - ["houdini-14"] - ] - -Which variant will I get if I run "rez-env geocache"? Behavior in this case is -undefined - rez gives no guarantees as to which variant will be selected. We cannot -meaningfully compare version numbers across packages, so maya will not have preference -simply because 2016 > 14. However, *version_priority* mode does give priority to -packages listed in the request. So if we ran "rez-env geocache maya", we will get -the first variant... probably. - -#### Probably? - -The operative word here is *preference*. Because the variants are not mutually -exclusive, we can't make guarantees. A resolve is still deterministic - you aren't -going to get differing results when requesting the same environment - but predicting -which variant you'll get can be tricky. - -Consider the following request: - - ]$ rez-env geocache maya animtools-1.4 - -We would expect to get the *maya-2016* variant of *geocache*. However, what if -*animtools* requires *maya-2017*? This makes the first *geocache* variant impossible -to select, since a conflict would occur, and so the *houdini* variant of *geocache* -will be selected - quite possibly not what you expected. - -> [[media/icons/under_construction.png]] I plan on adding a new package request -> syntax, that is able to explicitly select package variants. This will avoid the -> ambiguity in cases like the one described here. - -## Why Use Variants? - -Variants are a powerful mechanism in rez. They avoid the need to maintain separate -branches of a package in order to support varying dependencies. You may have had -problems in the past where a common library depends on, say, boost, and is used in -various DCCs (maya, nuke etc), and depended on by many other packages in your -pipeline. When a DCC moves to a new version of boost (or python, or OIIO, etc) -you now have to branch this library, which potentially affects many other packages. -The problem gets worse if you have multiple dependencies with varying versions. -Variants solve that problem - you simply add another boost variant to your library, -and other dependent packages are not affected. Rez will correctly select the -package variant that does not conflict with the resolved environment. diff --git a/wiki/pages/_Footer.md b/wiki/pages/_Footer.md deleted file mode 100644 index 94ede22fb..000000000 --- a/wiki/pages/_Footer.md +++ /dev/null @@ -1,4 +0,0 @@ -[![Google Group](https://img.shields.io/badge/rez--config-Google%20Group-blue?style=flat-square&logo=google)](https://groups.google.com/forum/#!forum/rez-config) -[![Contributing Guidelines](https://img.shields.io/badge/rez-Contributing%20Guidelines-0b610e?style=flat-square&logo=github)](https://github.com/__GITHUB_REPO__/blob/main/CONTRIBUTING.md) -[![Report A Bug](https://img.shields.io/badge/rez-Report%20A%20Bug-critical?style=flat-square&logo=github)](https://github.com/__GITHUB_REPO__/issues/new) -[![Slack](https://img.shields.io/badge/slack-rez--talk-7a6800?style=flat-square&logo=slack)](https://rez-talk.slack.com/) diff --git a/wiki/pages/_Sidebar.md b/wiki/pages/_Sidebar.md deleted file mode 100644 index 1c9499085..000000000 --- a/wiki/pages/_Sidebar.md +++ /dev/null @@ -1,47 +0,0 @@ -- [[Home]] - -:beginner: Introduction: - -- [[Installation]] and [[Pip]] -- [[Getting Started]] -- [[Basic Concepts]] -- [[Configuring Rez]] - -:memo: `package.py`: - -- [[Package Commands]] -- [[Package Definition Guide]] -- [[Variants]] -- [[Ephemeral Packages]] - -:rocket: rez: - -- [[Contexts]] -- [[Bundles]] -- [[Suites]] -- [[Building Packages]] -- [[Managing Packages]] -- [[Environment Variables]] -- [[Command Line Tools]] - -:information_source: Others: - -- [API Reference](https://__GITHUB_USER__.github.io/__REPO_NAME__/) -- [[Glossary]] -- [[FAQ]] -- [[Notes]] -- [[Credits]] - -
:construction: Unwritten pages: - -- [[Advanced Topics]] -- [[Caching]] -- [[Releasing Packages]] -- [[Package Filters]] -- [[Testing Packages]] -- [[Rez GUI]] -- [[The Resolve Graph]] -- [[Timestamping]] -- [[Troubleshooting]] - -