Skip to content

Commit

Permalink
tox.ini/stestr: remove "--serial" to enable test parallelization (#880)
Browse files Browse the repository at this point in the history
Having "--serial" impedes the parallelization and thus affects the
unit test execution speed greatly. We have no real reason to have
"--serial" in place, it both makes the test execution slower and
possibly hides hidden dependencies between the test cases. The unit
tests must be able to run in parallel and should be indifferent to
the ordering.

This patch removes the "--serial" argument, adds "--random" and
"--slowest" arguments, which randomizes the order of the test cases
and reports the slowest test cases respectively.

Below are the run time differences between the baseline(main) and
this patch(stestr-parallel):

Command `tox`:
---------------------------------------------------
main:
```
py3: OK (66.73=setup[0.03]+cmd[0.06,66.64] seconds)
...
congratulations :) (111.38 seconds)
```

stestr-parallel:
```
py3: OK (10.21=setup[0.03]+cmd[0.05,10.12] seconds)
...
congratulations :) (53.62 seconds)

```

results:
6.53x improvement in "py3" runtime
2.07x improvement in total runtime

Command `tox --parallel`:
---------------------------------------------------
main:
```
py3: OK (68.94=setup[0.17]+cmd[0.05,68.71] seconds)
...
congratulations :) (71.23 seconds)

```

stestr-parallel:

```
py3: OK (12.12=setup[0.21]+cmd[0.06,11.86] seconds)
...
congratulations :) (41.61 seconds)
```

results:
5.68x improvement in "py3" runtime
1.71x improvement in total runtime

Signed-off-by: Mustafa Kemal Gilor <[email protected]>
  • Loading branch information
xmkg authored May 22, 2024
1 parent f83347b commit f3f140f
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ deps =
-r{toxinidir}/test-requirements.txt
commands =
coverage erase
stestr run --serial --test-path {[testenv]unit_tests} {posargs}
stestr run --random --slowest --test-path {[testenv]unit_tests} {posargs}

[testenv:pep8]
allowlist_externals = flake8
Expand Down

0 comments on commit f3f140f

Please sign in to comment.