Feat: tox implementation

[moe-ad]

This PR is related to #296.
This is not a catch all PR, the idea is to first migrate some CI workflows to tox and gradually extend to other CI workflows via other PRs.

[moe-ad]

PyDPF uses a custom wheel building step in CI. The first commit implements a way of integrating that step with tox.

[PProfizi]

Hi @moe-ad, thanks for this work.
Could we get a quick reminder of the advantages of switching to Tox?
I do not see it described here https://dev.docs.pyansys.com/how-to/testing.html

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.37%. Comparing base (7ba086b) to head (cf74a02).
Report is 3 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1989      +/-   ##
==========================================
- Coverage   88.45%   88.37%   -0.09%     
==========================================
  Files          89       89              
  Lines       10251    10251              
==========================================
- Hits         9068     9059       -9     
- Misses       1183     1192       +9     

[moe-ad]

@PProfizi
The immediately advantages would be:

• Having workflows that can run locally similarly to CI runs.
• Centralizing CI patch scripts. If the logic in these scripts can be specified entirely within tox.ini, they can be removed altogether.
• Removing friction for contributors: easier to specify instructions like "install tox, then run tox xx to build documentation or tox yy to run tests" than requiring understanding of the entire repo layout to get testing/doc-build tasks done.
• Isolate test environments: this is particularly useful for PyDPF's use case. Per test set in CI, a different environment will be spin up without any interference with/from the environment of other test sets.
• Tox's invocation syntax is very flexible which is especially useful for local runs. Tests can be chosen to run sequentially or in parallel, or even running one specific test / group of tests. The same configuration allows this flexibility.
• Of course being able to run in parallel should ideally translate to faster test runs.

The long term advantages would be:

• Easy invocation of workflows across multiple python versions: currently most testing/doc-build is on py39. If we do decide to extend these workflows to more python versions, tox makes this easy to do.
• The point above doesn't apply to python version alone, we can spin up test environments for any combination of dependencies we wish to execute a workflow against.

Those are the ones I can think of at the moment. I remember @jorgepiloto also shared many advantages. He will be back by tomorrow and he can also provide his inputs.

[moe-ad]

@PProfizi
Now simply running locally:

• tox -e runtests-parallel should run all tests in parallel
• tox -e runtests-sequential should run all tests sequentially

With automated test target organization and server process(es) shutdown before and after all tests have run.

Per test (irrespective of the running mode), a fresh environment with individual ansys.dpf.core package installation is used for test runs. The sacrifice with running in parallel is not having detailed output to stdout, only success/failure summary is shown for each test environment. Possible to force detailed output to stdout but that will result in an unstructured report. Maybe there is some log file being generated that can be inspected at the end of all runs?

Next steps:

• Confirm all environment variables we need to be passing to the test environment since tox won't automatically pass them.
• The package building step is done for each test environment. Will be nice if it can be done just once, and then installed into each environment as needed. Tox is smart enough to do this if package building were via a pyproject.toml or something similar.

[jorgepiloto]

Thanks for working on this, @moe-ad. I know it is still in draft mode, but wanted to raise a point about auto-detecting the platform.

[moe-ad]

@jorgepiloto thanks for our discussion about this, I have improved the configuration significantly by taking advantage of the depends feature in tox in a way that makes test invocation more intuitive:

• tox / tox --parallel will run all tests in sequential/parallel mode.
• tox -e pretest,<test-list>,posttest will run specific tests. This invocation can also be used with the parallel flag.

@PProfizi we will merge this PR as Jorge suggested that we should make this available for users to start taking advantage of tox locally immediately. We will proceed with next steps in other PRs.

[jorgepiloto]

Maybe this is no longer required if we are calling a Python script in the commands section, right?

[moe-ad]

Removed. Thanks for catching that.

[jorgepiloto]

Thanks for the big effort here, @moe-ad. From my point of view, we can proceed. In the future, we can keep cleaning things to avoid the custom build and the pre-post hooks.

[PProfizi]

Hi @moe-ad, thanks for this. Do you have an example of how we should change the contribute guide or the getting started on how to test or build the package?