Skip to content

Commit

Permalink
Merge branch 'topic/machu/fix-test-race-condition' into 'master'
Browse files Browse the repository at this point in the history
Fix race condition in testsuite

Closes #218

See merge request eng/ide/libadalang-tools!270
  • Loading branch information
Volham22 committed Dec 3, 2024
2 parents 0aa551f + 7d0efea commit 1363ebc
Show file tree
Hide file tree
Showing 70 changed files with 117 additions and 68 deletions.
47 changes: 47 additions & 0 deletions testsuite/drivers/gnattest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
from pathlib import Path
import os
import shutil
import tempfile

from drivers.base_driver import BaseDriver


class GNATtestDriver(BaseDriver):
"""
GNATtest and TGen Test Driver.
Interface:
Same as `ShellScriptDriver`.
This driver is specific to GNATtest and behaves like `ShellScriptDriver`,
except that it copies the TGen runtime support project to a temporary
directory and adds this copy to `GPR_PROJECT_PATH`.
This approach prevents race conditions when GNATtest builds harnesses
that depend on the TGen runtime support library. During the build process,
`gprbuild` overwrites the static library artifact (`libtgen.a`) each time
it is invoked. To avoid these conflicts, each test run operates on its own
isolated copy of the `tgen_rts` project.
"""

DEFAULT_TGEN_RTS_LOCATION: Path = Path("src/tgen/tgen_rts").absolute()

def run(self) -> None:
tgen_rts_project_path = os.environ.get(
"TGEN_RTS_LOCATION", str(GNATtestDriver.DEFAULT_TGEN_RTS_LOCATION)
)
gpr_project_path = os.environ.get("GPR_PROJECT_PATH", "")

temp_dir = Path(tempfile.mkdtemp()) / "tgen_rts"
shutil.copytree(tgen_rts_project_path, temp_dir)

process_result = self.shell(
["bash", "test.sh"],
catch_error=False,
env={**os.environ, "GPR_PROJECT_PATH": f"{temp_dir}:{gpr_project_path}"},
)

if process_result.status:
self.output += ">>>program returned status code {}\n".format(
process_result.status
)
2 changes: 1 addition & 1 deletion testsuite/tests/test/135_tgen_wrapper_conflict/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@ description: Test that TGen generates precondition wrappers that can be
compiled when a parameter of the tested subprogram has the same name as
the package in which it is defined.

driver: shell_script
driver: gnattest_driver
control:
- [XFAIL, 'x86', 'Marshalling not working for 32bits (UB03-008)']
2 changes: 1 addition & 1 deletion testsuite/tests/test/136_tgen_generics/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@ description:
Test that TGen generates valid code when including a subprogram that is
in a generic nested instantiation.

driver: shell_script
driver: gnattest_driver
control:
- [XFAIL, 'x86', 'Marshalling not working for 32bits (UB03-008)']
2 changes: 1 addition & 1 deletion testsuite/tests/test/146-gen_full_glob/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@ description:
Test support of global variables in TGen, for subprograms with no
parameters.

driver: shell_script
driver: gnattest_driver
control:
- [XFAIL, 'x86', 'Marshalling not working for 32bits (UB03-008)']
2 changes: 1 addition & 1 deletion testsuite/tests/test/147-clean_tgen_dir/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@ description:
Test that the TGen support library is always generated from a clean
directory, removing stale sources.

driver: shell_script
driver: gnattest_driver
control:
- [XFAIL, 'x86', 'Marshalling not working for 32bits (UB03-008)']
2 changes: 1 addition & 1 deletion testsuite/tests/test/151_arr_limit/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@ description:
that the limit set through the TGEN_ARRAY_LIMIT environment variable
does raise an exception.

driver: shell_script
driver: gnattest_driver
control:
- [XFAIL, 'x86', 'Marshalling not working for 32bits (UB03-008)']
2 changes: 1 addition & 1 deletion testsuite/tests/test/153-floating-point-tgen/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@ description:
point values in the range of a floating point type with a range
constraint.

driver: shell_script
driver: gnattest_driver
control:
- [XFAIL, 'x86', 'Marshalling not working for 32bits (UB03-008)']
2 changes: 1 addition & 1 deletion testsuite/tests/test/154-tgen-array-bounds/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@ description:
generates values for array types that have a constrained subtype as
an index type. It used to crash when trying to generate values.

driver: shell_script
driver: gnattest_driver
control:
- [XFAIL, 'x86', 'Marshalling not working for 32bits (UB03-008)']
2 changes: 1 addition & 1 deletion testsuite/tests/test/157_ops/test.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
description: Check that the support library and gnattest harness are legal for
custom operators.

driver: shell_script
driver: gnattest_driver

control:
- [XFAIL, 'x86', 'Marshalling not working for 32bits (UB03-008)']
2 changes: 1 addition & 1 deletion testsuite/tests/test/157_std_shadowing/test.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
description: Check that the support library and gnattest harness are legal when
a user project package is named "standard".

driver: shell_script
driver: gnattest_driver

control:
- [XFAIL, 'True', 'LAL and nameres on a custom standard package, see eng/libadalang/libadalang#1453']
Expand Down
2 changes: 1 addition & 1 deletion testsuite/tests/test/157_test_subp/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ description: Check that the support library and gnattest harness are legal when
either the subprogram under test is called "test" or the unit in which it is
defined is called test.

driver: shell_script
driver: gnattest_driver

control:
- [XFAIL, 'x86', 'Marshalling not working for 32bits (UB03-008)']
2 changes: 1 addition & 1 deletion testsuite/tests/test/177_protected_gen/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@ description: Check that the test generation part of gnattest does not
attempt to generate values for protected subprograms and entries,
as this is not supported by the harness generation part of gnattest.

driver: shell_script
driver: gnattest_driver
control:
- [XFAIL, 'x86', 'Marshalling not working for 32bits (UB03-008)']
2 changes: 1 addition & 1 deletion testsuite/tests/test/177_tasking/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@ description: Check that the test generation part of gnattest does not
attempt to generate values for task entries, as this is not supported
by the harness generation part of gnattest.

driver: shell_script
driver: gnattest_driver
control:
- [XFAIL, 'x86', 'Marshalling not working for 32bits (UB03-008)']
2 changes: 1 addition & 1 deletion testsuite/tests/test/183-formal_params/test.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
description: Test that gnattest does not attempt to generate test inputs for
subprograms with generic formal parameters or return types.

driver: shell_script
driver: gnattest_driver

canonicalize_backslashes: True

Expand Down
2 changes: 1 addition & 1 deletion testsuite/tests/test/184-generic-lib-item/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@ description:
Check that TGen properly rejects types declared in generic instantiations
that are library items.

driver: shell_script
driver: gnattest_driver
control:
- [XFAIL, 'x86', 'Marshalling not working for 32bits (UB03-008)']
2 changes: 1 addition & 1 deletion testsuite/tests/test/185_tgen_wrapper_paren_expr/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@ description: Test that the precondition wrappers generated by TGen incorporate
the full expression, including the parenthesis when it is required, such as in
quantified expressions or if-expressions.

driver: shell_script
driver: gnattest_driver
control:
- [XFAIL, 'x86', 'Marshalling not working for 32bits (UB03-008)']
2 changes: 1 addition & 1 deletion testsuite/tests/test/193-anonymous-type/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@ description:
with an anonymous type parameter. It used to call P_Fully_Qualified_Name on
it, which resulted in a crash.

driver: shell_script
driver: gnattest_driver
control:
- [XFAIL, 'x86', 'Marshalling not working for 32bits (UB03-008)']
2 changes: 1 addition & 1 deletion testsuite/tests/test/194_const_array_limit/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description:
We do this by attempting to generate tests with gnattest for a big array type,
and checking the emitted warnings.

driver: shell_script
driver: gnattest_driver

control:
- [XFAIL, 'x86', 'Marshalling not working for 32bits (UB03-008)']
2 changes: 1 addition & 1 deletion testsuite/tests/test/195-repeated-pragmas/test.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
description:
gnattest test

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/208-gpr-warnings/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ description:
loading the project file. Instead, only output those warnings and still
proceed with the rest of the tool.

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/208-stub_extension/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ description:
for a project tree in which an extending project is present. This used to
crash due to a mis-management of the project table.

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/212-default-encoding/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@ description:
characters from being recognized. This is materialized in zdep.*, which
contain a 'é' character, encoded in latin-1.

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/32-basic_enum/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@ description:
Test basic enumerated generation for scalar types as well as non
discriminated record types.

driver: shell_script
driver: gnattest_driver
control:
- [XFAIL, 'x86', 'Marshalling not working for 32bits (UB03-008)']
2 changes: 1 addition & 1 deletion testsuite/tests/test/36-stub_dangling_setters/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ description:
issues. They used to not be commented out, which could result in lingering
references to types or objects no longer present in the codebase.

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/37-stub-ada-version/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ description:
This used not to be the case because gnattest added a pragma Ada_2005 in
some of the stub data units.

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/37-stub-anon-subp-acc/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ description:
This used not to be the case because gnattest added a pragma Ada_2005 in
some of the stub data units.

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/98-long-json/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,6 @@ description:
longer than 32766 characters, which is the maximum allowed length in
gprbuild.

driver: shell_script
driver: gnattest_driver
control:
- [XFAIL, 'x86', 'Marshalling not working for 32bits (UB03-008)']
2 changes: 1 addition & 1 deletion testsuite/tests/test/T206-015/test.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
description:
gnattest test

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/T213-041/test.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
description:
gnattest test

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/T217-072/test.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
description:
gnattest test

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/T629-003/test.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
description:
gnattest test

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/T715-010/test.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
description:
gnattest test

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/TA07-010/test.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
description:
gnattest test

driver: shell_script
driver: gnattest_driver
control:
- [SKIP, 'windows', 'No symbolic links on Windows']
2 changes: 1 addition & 1 deletion testsuite/tests/test/TA19-048/test.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
description:
gnattest test

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/TA19-051/test.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
description:
gnattest test

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/TB16-055/test.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
description:
gnattest test

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/UA15-049/test.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
description:
gnattest test

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/UA15-080/test.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
description:
gnattest test

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/UB09-001/test.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
description:
gnattest test

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/V406-023/test.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
description:
gnattest test

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/VA31-011/test.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
description:
gnattest test

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/VB28-016/test.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
description:
gnattest test

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/dump_test_inputs/test.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
description:
Test checking test input dumping

driver: shell_script
driver: gnattest_driver

2 changes: 1 addition & 1 deletion testsuite/tests/test/error_recovery/test.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
description:
Test error recovery

driver: shell_script
driver: gnattest_driver

Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
description:
gnattest test

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/instrument_bodies/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@ description:
Check instrumentation of subprograms declared in spec and having
renaming, expression function and a null body in the package body.

driver: shell_script
driver: gnattest_driver

2 changes: 1 addition & 1 deletion testsuite/tests/test/instrument_expr_func/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,5 @@ description:
of artificial package bodies for containing instrumented
bodies if necessary, including library level package body.

driver: shell_script
driver: gnattest_driver

2 changes: 1 addition & 1 deletion testsuite/tests/test/latin_1_char/test.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
description:
gnattest test

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/limited_access_stub/test.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
description: Check that gnattest does not consider in-mode access types
as parameters for which stub setters must be generated.

driver: shell_script
driver: gnattest_driver
2 changes: 1 addition & 1 deletion testsuite/tests/test/marshalling_full_private/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@ description:
with no visible part (i.e. only declared in the private part of the
package).

driver: shell_script
driver: gnattest_driver
control:
- [XFAIL, 'x86', 'Marshalling not working for 32bits (UB03-008)']
Loading

0 comments on commit 1363ebc

Please sign in to comment.