diff --git a/CHANGELOG.md b/CHANGELOG.md index 311ca765..2c6ce0c6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,8 @@ and this project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html) ### Changed +- Refactored our development guide to clarify development environment setup and how to run tests + ([@jhkennedy](https://github.com/jhkennedy)) - Use built-in `assert` statements instead of `unittest` assertions in integration tests ([#743](https://github.com/nsidc/earthaccess/issues/743)) ([@chuckwondo](https://github.com/chuckwondo)) diff --git a/docs/contributing/development.md b/docs/contributing/development.md index 1bd52211..aa97c41a 100644 --- a/docs/contributing/development.md +++ b/docs/contributing/development.md @@ -7,26 +7,105 @@ ```bash git clone git@github.com:{my-username}/earthaccess ``` +1. We recommend creating a feature branch when developing new features, fixing bugs, updating docs, etc. It's best to give the branch +a descriptive name For example, if you're updating the contributing docs, you might create a branch called `update-contributing-docs`: + ```bash + git switch -c update-contributing-docs + ``` -In order to develop new features or fix bugs etc. we need to set up a virtual +Next, In order to develop new features or fix bugs etc., you'll need to set up a virtual environment and install the library locally. -## Quickstart development +## Development environment setup -The fastest way to start with development is to use nox. If you don't have nox, -you can use `pipx run nox` to run it without installing, or `pipx install nox`. -If you don't have pipx (pip for applications), then you can install with -`pip install pipx` (the only case were installing an application with regular -pip is reasonable). If you use macOS, then pipx and nox are both in brew, use -`brew install pipx nox`. -To use, run `nox` without any arguments. This will run the type check and unit -test "sessions" (tasks) using your local (and active) Python version. -Nox handles everything for you, including setting up a temporary virtual -environment for each run. +There are a few options for setting up a development environment; you can use Python's `venv`, use `conda`/`mamba`, or _not_ +manage one yourself and use `pipx` to run tests and build docs with `nox`. -You can see all available sessions with `nox --list`: +* If you're a Windows user, you'll likely want to set up an environment yourself with `conda`/`mamba`. +* If you're working in a JupyterHub, you'll likely want to set up an environment yourself with `conda`/`mamba`. +* If you're using an IDE like VS Code or PyCharm, you'll likely want to set up an environment yourself with `venv` or `conda`/`mamba`. +* If you're using a plain text editor and don't know how to or want to manage a virtual environment, you'll likely want to start with `pipx`. + +=== "`venv`" + + `venv` is a virtual environment manager that's built into Python. + + Create and activate the development environment with: + + ```bash + python -m venv .venv + source .venv/bin/activate + ``` + + Then install `earthaccess` into the environment in editable mode with the optional development dependencies: + + ```bash + python -m pip install --editable ".[dev,test,docs]" + ``` + +=== "`conda`/`mamba`" + + `conda` and `mamba` are open-source package and environment managers that are language and platform agnostic. + `mamba` is a newer and faster re-implementation of `conda` -- you can use either `conda` or `mamba` + in the commands below. + + After you have followed [the `conda`/`mamba` installation instructions](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html) (we recommend Miniforge), you can create and activate the development environment with: + + ```bash + mamba env update -f environment.yml + mamba activate earthaccess + ``` + + This will ensure the `earthaccess` environment exists and is up-to-date, and active it. The `earthaccess` package will + be installed into the environment in editable mode with the optional development dependencies. + + ??? note "2024-09-23: Conda environment name changed from `earthaccess-dev` -> `earthaccess`" + + On Sept. 23, 2024, the name of the conda environment changed from `earthaccess-dev` to `earthacess` to align with + community best practices. If you have an `earthaccess-dev` environment, we recommend deleting it and creating a new one. + From the repository root, you can do that with these commands: + + ```bash + mamba env update -f environment.yml + mamba activate earthaccess + mamba env remove -n earthaccess-dev + ``` + +=== "`pipx`+`nox`" + + `pipx` is a tool to help you install and run end-user applications written in Python and `nox` is Python application + that automates testing in multiple Python environments. By using `pipx` to install `nox` and using `nox` to run common development tasks, some users can avoid the need to set up and manage a full development environment. + Once you have [installed `pipx` following the official instructions](https://github.com/pypa/pipx?tab=readme-ov-file#install-pipx), you can either run `nox` without installing it via: + + ```bash + pipx run nox [NOX_ARGS] + ``` + or install `nox` into an isolated environment and run it with: + + ```bash + pipx install nox + nox [NOX_ARGS] + ``` + + `nox` handles everything for you, including setting up a temporary virtual environment for each task (e.g. running tests, building docs, etc.) you need to run. + +Now that your development environment is set up, you can run the tests. + +## Running tests + +We recommend using `nox` to run the various "sessions" (`nox`'s term for tasks) provided by `earthaccess`. To use, run `nox` without +any arguments: + +```bash +nox +``` + +This will run the type check and unit test sessions using your local (and active) Python +version. `nox` handles everything for you, including setting up a temporary virtual environment for each run. + +You can see all available sessions with `nox --list`: ``` $ nox --list Sessions defined in earthaccess/noxfile.py: @@ -41,8 +120,7 @@ Sessions defined in earthaccess/noxfile.py: sessions marked with * are selected, sessions marked with - are skipped. ``` -You can also run individual tasks (_sessions_ in `nox` parlance, hence the `-s` -option below), like so: +You can also run individual sessions like so: ```bash nox -s integration-tests @@ -54,57 +132,74 @@ and pass options to the underlying session like: nox -s integration-tests -- [ARGS] ``` -!!! tip - - In order to run integration tests locally, you must set the - environment variables `EARTHDATA_USERNAME` and `EARTHDATA_PASSWORD` to your - username and password, respectively, of your - [NASA Earthdata](https://urs.earthdata.nasa.gov/) account (registration is - free). - - +!!! info "Important" -## Manual development environment setup + In order to run integration tests locally, you must set the environment variables `EARTHDATA_USERNAME` and + `EARTHDATA_PASSWORD` to the username and password of your [NASA Earthdata](https://urs.earthdata.nasa.gov/) + account, respectively (registration is free). -While `nox` is the fastest way to get started, you will likely need a full -development environment for making code contributions, for example to test in a -REPL, or to resolve references in your favorite IDE. This development -environment also includes `nox`. You can create it with `venv`, `conda`, or `mamba`. +### IDE setup -=== "`venv`" +Integrated development environments (IDEs) like VS Code and PyCharm provide powerful refactoring, testing, and +debugging integrations, but they typically don't understand "task runners" like `nox` and won't know how to launch +debugging or testing sessions connected to the provided integrations. - `venv` is a virtual environment manager that's built into Python. +Fortunately, if you've set up a development environment you should be able to call the underlying testing tools +(e.g., `mypy` and `pytest`) directly, or run them via your IDE integrations. To understand how `nox` is running the +underlying tools in each test session you can read the `noxfile.py` in the repository root, or, run all the test directly +in your development environment like: - Create and activate the development environment with: +```bash +nox -fb none --no-install +``` - ```bash - python -m venv .venv - source .venv/bin/activate - ``` +This will force `nox` to not use an environment backend (will just use the active environment) and not attempt to install +any packages. When `nox` runs, it will describe the commands it executes: - Then install `earthaccess` into the environment in editable mode with the optional development dependencies: +``` +$ nox -fb none --no-install +nox > Running session typecheck +nox > mypy +Success: no issues found in 35 source files +nox > Session typecheck was successful. +nox > Running session tests +nox > pytest tests/unit -rxXs +========================================== test session starts ========================================== +... +==================================== 43 passed, 1 xfailed in 24.01s ===================================== +nox > Session tests was successful. +nox > Ran multiple sessions: +nox > * typecheck: success +nox > * tests: success +``` - ```bash - python -m pip install --editable ".[dev,test,docs]" - ``` +Note these lines in particular: +``` +nox > Running session typecheck +nox > mypy +nox > Running session tests +nox > pytest tests/unit -rxXs +``` -=== "`conda`/`mamba`" +So to reproduce the typecheck session all you have to do is run `mypy` in your development environment. Similarly, reproducing +the unit tests is running `pytest test/unit -rxXs`. - `conda` and `mamba` are open-source package and environment managers that are language and platform agnostic. - `mamba` is a newer and faster re-implementation of `conda` -- you can use either `conda` or `mamba` - in the commands below. +Since we're not doing any complicated configuration or setting complicated arguments to pytest, simply hitting the "play" button +for a pytest in your IDE should work once you've configured it to use your development environment. - Create and activate the development environment with: +!!! info "Important" + Currently, our integration tests are *flakey* and a small number of random failures are expected. When the integration + test suite runs, it may return a status code of 99 if the failure rate was less than an "acceptable" threshold. Since + any non-zero status code is considered an error, your console and/or IDE will consider this a failure by default. + `nox`, however, knows about this special status code and will report a success. To get pytest or your IDE to match + this behavior, you can modify the special status code to be zero with the `EARTHACCESS_ALLOWABLE_FAILURE_STATUS_CODE` + evnironment variable: ```bash - mamba env update -f environment.yml - mamba activate earthaccess + export EARTHACCESS_ALLOWABLE_FAILURE_STATUS_CODE=0 ``` - This will update (or create if missing) the `earthaccess` environment and active it. The `earthaccess` package will - be installed into the environment in editable mode with the optional development dependencies. - ## Managing Dependencies If you need to add a new dependency, edit `pyproject.toml` and insert the @@ -134,14 +229,14 @@ Since `python-cmr` exposes the `cmr` package, the stubs appear under `stubs/cmr` To work on documentation locally, we provide a script that will automatically re-render the docs when you make changes: ``` -nox -s serve_docs +nox -s serve-docs ``` MkDocs does not support incremental rebuilds and will execute every Jupyter Notebook every time it builds a new version of the site, which can be quite slow. To speed up the build, you can pass MkDocs these options: ``` -nox -s serve_docs -- --dirty --no-strict +nox -s serve-docs -- --dirty --no-strict ``` !!! warning diff --git a/docs/css/styles.css b/docs/css/styles.css index 43aa0abf..dbb65baa 100644 --- a/docs/css/styles.css +++ b/docs/css/styles.css @@ -1,2 +1,9 @@ .details .open { } + +.tabbed-set { + padding: 5px; + border: 1px lightgrey; + border-radius: 2px; + border-style: solid; +}