Removing run-tests.py file. (#4180)

* Removing run-test.py file Signed-off-by: Pradyot Ranjan <[email protected]> * Removing redundent files Signed-off-by: Pradyot Ranjan <[email protected]> * Removing redundent files Signed-off-by: Pradyot Ranjan <[email protected]> * style: pre-commit fixes * Update test_examples.py * Update noxfile.py Co-authored-by: Agriya Khetarpal <[email protected]> * Removing run-test.py file Signed-off-by: Pradyot Ranjan <[email protected]> * Removing redundent files Signed-off-by: Pradyot Ranjan <[email protected]> * Removing redundent files Signed-off-by: Pradyot Ranjan <[email protected]> * Removing example marker Signed-off-by: Pradyot Ranjan <[email protected]> * adding conftest.py file Signed-off-by: Pradyot Ranjan <[email protected]> * adding back changes Signed-off-by: Pradyot Ranjan <[email protected]> * style: pre-commit fixes * Update test_examples.py * style: pre-commit fixes * Removing run-test.py file Signed-off-by: Pradyot Ranjan <[email protected]> * Removing redundent files Signed-off-by: Pradyot Ranjan <[email protected]> * Removing redundent files Signed-off-by: Pradyot Ranjan <[email protected]> * style: pre-commit fixes * Update test_examples.py * Update noxfile.py Co-authored-by: Agriya Khetarpal <[email protected]> * adding back changes Signed-off-by: Pradyot Ranjan <[email protected]> * removing loop for file discovery Signed-off-by: Pradyot Ranjan <[email protected]> * removing loop for file discovery Signed-off-by: Pradyot Ranjan <[email protected]> * Adding more flags to pytest Signed-off-by: Pradyot Ranjan <[email protected]> * style: pre-commit fixes * Adding markers for unit and integration tests Signed-off-by: Pradyot Ranjan <[email protected]> * Adding markers for unit and integration tests Signed-off-by: Pradyot Ranjan <[email protected]> * style: pre-commit fixes * Added back assertDomainEqual Signed-off-by: Pradyot Ranjan <[email protected]> * style: pre-commit fixes * Added fixture for debug mode Signed-off-by: Pradyot Ranjan <[email protected]> * style: pre-commit fixes * Removed style failures Signed-off-by: Pradyot Ranjan <[email protected]> * Removed style failures Signed-off-by: Pradyot Ranjan <[email protected]> * Update conftest.py Co-authored-by: Agriya Khetarpal <[email protected]> * style: pre-commit fixes * Update conftest.py Co-authored-by: Agriya Khetarpal <[email protected]> * Update noxfile.py Co-authored-by: Agriya Khetarpal <[email protected]> * Update pyproject.toml Co-authored-by: Agriya Khetarpal <[email protected]> * Update tests/testcase.py Co-authored-by: Agriya Khetarpal <[email protected]> * Update tests/test_scripts.py Co-authored-by: Agriya Khetarpal <[email protected]> * Update tests/test_scripts.py Co-authored-by: Agriya Khetarpal <[email protected]> * style: pre-commit fixes * using -m instead of args Signed-off-by: Pradyot Ranjan <[email protected]> * removed redundent files Signed-off-by: Pradyot Ranjan <[email protected]> * style: pre-commit fixes * fixing style failure Signed-off-by: Pradyot Ranjan <[email protected]> * style: pre-commit fixes * adding documentation changes Signed-off-by: Pradyot Ranjan <[email protected]> * Update CONTRIBUTING.md Co-authored-by: Agriya Khetarpal <[email protected]> * Update CONTRIBUTING.md Co-authored-by: Agriya Khetarpal <[email protected]> * adding suggestions Signed-off-by: Pradyot Ranjan <[email protected]> * removing -s flag and getting file names in scripts log Signed-off-by: Pradyot Ranjan <[email protected]> * Update pyproject.toml Co-authored-by: Agriya Khetarpal <[email protected]> * Update CONTRIBUTING.md Co-authored-by: Agriya Khetarpal <[email protected]> --------- Signed-off-by: Pradyot Ranjan <[email protected]> Co-authored-by: Pradyot Ranjan <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Agriya Khetarpal <[email protected]> Co-authored-by: Arjun Verma <[email protected]> Co-authored-by: Eric G. Kratz <[email protected]>
pybamm-team · Jul 26, 2024 · 48b4e48 · 48b4e48
1 parent 91c9744
commit 48b4e48
Show file tree

Hide file tree

Showing 7 changed files with 103 additions and 338 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -149,9 +149,13 @@ The `test_optional_dependencies` function extracts `pybamm` mandatory distributi
 
 ## Testing
 
-All code requires testing. We use the [unittest](https://docs.python.org/3.3/library/unittest.html) package for our tests. (These tests typically just check that the code runs without error, and so, are more _debugging_ than _testing_ in a strict sense. Nevertheless, they are very useful to have!)
+All code requires testing. We use the [pytest](https://docs.pytest.org/en/stable/) package for our tests. (These tests typically just check that the code runs without error, and so, are more _debugging_ than _testing_ in a strict sense. Nevertheless, they are very useful to have!)
 
-We also use [pytest](https://docs.pytest.org/en/latest/) along with the [nbmake](https://github.com/treebeardtech/nbmake) and the [pytest-xdist](https://pypi.org/project/pytest-xdist/) plugins to test the example notebooks.
+We use following plugins for various needs:
+
+[nbmake](https://github.com/treebeardtech/nbmake&#41) : plugins to test the example notebooks.
+
+[pytest-xdist](https://pypi.org/project/pytest-xdist/&#41) : plugins to run tests in parallel.
 
 If you have `nox` installed, to run unit tests, type
 
@@ -162,14 +166,14 @@ nox -s unit
 else, type
 
 ```bash
-python run-tests.py --unit
+pytest -m unit
 ```
 
 ### Writing tests
 
 Every new feature should have its own test. To create ones, have a look at the `test` directory and see if there's a test for a similar method. Copy-pasting this is a good way to start.
 
-Next, add some simple (and speedy!) tests of your main features. If these run without exceptions that's a good start! Next, check the output of your methods using any of these [assert methods](https://docs.python.org/3.3/library/unittest.html#assert-methods).
+Next, add some simple (and speedy!) tests of your main features. If these run without exceptions that's a good start! Next, check the output of your methods using [assert statements](https://docs.pytest.org/en/7.1.x/how-to/assert.html).
 
 ### Running more tests
 
@@ -193,7 +197,7 @@ nox -s examples
 Alternatively, you may use `pytest` directly with the `--nbmake` flag:
 
 ```bash
-pytest --nbmake
+pytest --nbmake docs/source/examples/
 ```
 
 which runs all the notebooks in the `docs/source/examples/notebooks/` folder in parallel by default, using the `pytest-xdist` plugin.
@@ -245,19 +249,19 @@ This also means that, if you can't fix the bug yourself, it will be much easier
 1. Run individual test scripts instead of the whole test suite:
 
    ```bash
-   python tests/unit/path/to/test
+   pytest tests/unit/path/to/test
    ```
 
    You can also run an individual test from a particular script, e.g.
 
    ```bash
-   python tests/unit/test_quick_plot.py TestQuickPlot.test_failure
+   pytest tests/unit/test_plotting/test_quick_plot.py::TestQuickPlot::test_simple_ode_model
    ```
 
    If you want to run several, but not all, the tests from a script, you can restrict which tests are run from a particular script by using the skipping decorator:
 
    ```python
-   @unittest.skip("")
+   @pytest.mark.skip("")
    def test_bit_of_code(self):
        ...
    ```

diff --git a/conftest.py b/conftest.py
@@ -0,0 +1,46 @@
+import pytest
+import numpy as np
+import pybamm
+
+
+def pytest_addoption(parser):
+    parser.addoption(
+        "--scripts",
+        action="store_true",
+        default=False,
+        help="execute the example scripts",
+    )
+    parser.addoption(
+        "--unit", action="store_true", default=False, help="run unit tests"
+    )
+    parser.addoption(
+        "--integration",
+        action="store_true",
+        default=False,
+        help="run integration tests",
+    )
+
+
+def pytest_configure(config):
+    config.addinivalue_line("markers", "scripts: mark test as an example script")
+    config.addinivalue_line("markers", "unit: mark test as a unit test")
+    config.addinivalue_line("markers", "integration: mark test as an integration test")
+
+
+def pytest_collection_modifyitems(items):
+    for item in items:
+        if "unit" in item.nodeid:
+            item.add_marker(pytest.mark.unit)
+        elif "integration" in item.nodeid:
+            item.add_marker(pytest.mark.integration)
+
+
+@pytest.fixture(autouse=True)
+# Set the random seed to 42 for all tests
+def set_random_seed():
+    np.random.seed(42)
+
+
+@pytest.fixture(autouse=True)
+def set_debug_value():
+    pybamm.settings.debug_mode = True
diff --git a/noxfile.py b/noxfile.py
@@ -54,9 +54,9 @@ def set_iree_state():
 
 homedir = os.getenv("HOME")
 PYBAMM_ENV = {
-    "SUNDIALS_INST": f"{homedir}/.local",
     "LD_LIBRARY_PATH": f"{homedir}/.local/lib",
     "PYTHONIOENCODING": "utf-8",
+    "MPLBACKEND": "Agg",
     # Expression evaluators (...EXPR_CASADI cannot be fully disabled at this time)
     "PYBAMM_IDAKLU_EXPR_CASADI": os.getenv("PYBAMM_IDAKLU_EXPR_CASADI", "ON"),
     "PYBAMM_IDAKLU_EXPR_IREE": set_iree_state(),
@@ -156,7 +156,7 @@ def run_integration(session):
     set_environment_variables(PYBAMM_ENV, session=session)
     session.install("setuptools", silent=False)
     session.install("-e", ".[all,dev,jax]", silent=False)
-    session.run("python", "run-tests.py", "--integration")
+    session.run("python", "-m", "pytest", "-m", "integration")
 
 
 @nox.session(name="doctests")
@@ -166,7 +166,13 @@ def run_doctests(session):
     # See: https://bitbucket.org/pybtex-devs/pybtex/issues/169/
     session.install("setuptools", silent=False)
     session.install("-e", ".[all,dev,docs]", silent=False)
-    session.run("python", "run-tests.py", "--doctest")
+    session.run(
+        "python",
+        "-m",
+        "pytest",
+        "--doctest-plus",
+        "pybamm",
+    )
 
 
 @nox.session(name="unit")
@@ -184,7 +190,7 @@ def run_unit(session):
             PYBAMM_ENV.get("IREE_INDEX_URL"),
             silent=False,
         )
-    session.run("python", "run-tests.py", "--unit")
+    session.run("python", "-m", "pytest", "-m", "unit")
 
 
 @nox.session(name="examples")
@@ -194,7 +200,9 @@ def run_examples(session):
     session.install("setuptools", silent=False)
     session.install("-e", ".[all,dev]", silent=False)
     notebooks_to_test = session.posargs if session.posargs else []
-    session.run("pytest", "--nbmake", *notebooks_to_test, external=True)
+    session.run(
+        "pytest", "--nbmake", *notebooks_to_test, "docs/source/examples/", external=True
+    )
 
 
 @nox.session(name="scripts")
@@ -206,7 +214,7 @@ def run_scripts(session):
     # is fixed
     session.install("setuptools", silent=False)
     session.install("-e", ".[all,dev]", silent=False)
-    session.run("python", "run-tests.py", "--scripts")
+    session.run("python", "-m", "pytest", "-m", "scripts")
 
 
 @nox.session(name="dev")
@@ -249,7 +257,7 @@ def run_tests(session):
     set_environment_variables(PYBAMM_ENV, session=session)
     session.install("setuptools", silent=False)
     session.install("-e", ".[all,dev,jax]", silent=False)
-    session.run("python", "run-tests.py", "--all")
+    session.run("python", "-m", "pytest", "-m", "unit or integration")
 
 
 @nox.session(name="docs")

diff --git a/pyproject.toml b/pyproject.toml
@@ -228,20 +228,21 @@ ignore = [
 "docs/source/examples/notebooks/models/lithium-plating.ipynb" = ["F821"]
 
 [tool.pytest.ini_options]
-minversion = "6"
+minversion = "8"
+# Use pytest-xdist to run tests in parallel by default, exit with
+# error if not installed
 required_plugins = [
     "pytest-xdist",
     "pytest-mock",
 ]
 addopts = [
     "-nauto",
-    "-v",
-    "-ra",
+    "-vra",
     "--strict-config",
     "--strict-markers",
 ]
 testpaths = [
-    "docs/source/examples/",
+    "tests",
 ]
 console_output_style = "progress"
 xfail_strict = true