You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Currently, the random seed in integration tests is set to 0 by default, and all cases can pass with this seed. However, if we change the seed to other values, some cases will fail.
I wish we could improve the integration tests to avoid this, e.g. use a dynamic seed for each run, or test with more data for those cases that are easy to fail.
Describe the solution you'd like
Set random seeds for each test run and log the seed for reproducibility.
Describe alternatives you've considered
Or we can make the easy to fail cases stronger, e.g. more data for aggregate tests and tests with long generated strings, more precise data range for fallback/exception tests.
Additional context
Some similar issues that has been fixed: #8716 and #8607
The text was updated successfully, but these errors were encountered:
thirtiseven
changed the title
[FEA] Enhance integration tests to avoid tests failures with different seeds
[FEA] Enhance integration test to avoid test failures with different seeds
Sep 18, 2023
Is your feature request related to a problem? Please describe.
Currently, the random seed in integration tests is set to 0 by default, and all cases can pass with this seed. However, if we change the seed to other values, some cases will fail.
I wish we could improve the integration tests to avoid this, e.g. use a dynamic seed for each run, or test with more data for those cases that are easy to fail.
Some failed cases:
test_cast_fallback_not_UTC
failed with seed 4 #9242test_groupby_first_last
failed with seed 4 #9243test_str_to_map_expr_fixed_pattern_input
failed with seed 4 #9244test_hash_groupby_collect_list
failed with seed 4 #9245test_predefined_character_classes
failed with seed 4 #9246test_initcap
failed with seed 14 #9247We should fix them or mark them as XFAIL first.
Describe the solution you'd like
Set random seeds for each test run and log the seed for reproducibility.
Describe alternatives you've considered
Or we can make the easy to fail cases stronger, e.g. more data for aggregate tests and tests with long generated strings, more precise data range for fallback/exception tests.
Additional context
Some similar issues that has been fixed: #8716 and #8607
The text was updated successfully, but these errors were encountered: