Skip to content

Commit

Permalink
test: do not count skipped tests as executed
Browse files Browse the repository at this point in the history
[ upstream commit a620df6df6d61660661afade09760b2dfba4eb42 ]

The logic around skipped tests is a little confusing in the unit test
runner.
* Any explicitly disabled tests are counted as skipped but not
  executed.
* Any tests that return TEST_SKIPPED are counted as both skipped and
  executed, using the same statistics counters.

This makes the stats very strange and hard to correlate, since the
totals don't add up.  One would expect that SKIPPED + EXECUTED +
UNSUPPORTED == TOTAL, and that PASSED + FAILED == EXECUTED.

To achieve this, mark any tests returning TEST_SKIPPED, or ENOTSUP as
not having executed.

Signed-off-by: Bruce Richardson <[email protected]>
Acked-by: Akhil Goyal <[email protected]>
Acked-by: Ciara Power <[email protected]>
Acked-by: Tyler Retzlaff <[email protected]>
  • Loading branch information
bruce-richardson authored and kevintraynor committed Mar 8, 2024
1 parent 3096a56 commit 3304b1f
Showing 1 changed file with 5 additions and 3 deletions.
8 changes: 5 additions & 3 deletions app/test/test.c
Original file line number Diff line number Diff line change
Expand Up @@ -350,11 +350,13 @@ unit_test_suite_runner(struct unit_test_suite *suite)

if (test_success == TEST_SUCCESS)
suite->succeeded++;
else if (test_success == TEST_SKIPPED)
else if (test_success == TEST_SKIPPED) {
suite->skipped++;
else if (test_success == -ENOTSUP)
suite->executed--;
} else if (test_success == -ENOTSUP) {
suite->unsupported++;
else
suite->executed--;
} else
suite->failed++;
} else if (test_success == -ENOTSUP) {
suite->unsupported++;
Expand Down

0 comments on commit 3304b1f

Please sign in to comment.