Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Download fewer datasets for C/C++ unit tests #4624

Merged
merged 5 commits into from
Sep 3, 2024
Merged

Download fewer datasets for C/C++ unit tests #4624

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
fd99e94
skip downloading test data for cpp tests, everything required is alre…
ChuckHastings Aug 20, 2024
b853029
debugging
ChuckHastings Aug 20, 2024
310ec3a
Merge branch 'branch-24.10' into skip_downloading_some_data
ChuckHastings Aug 29, 2024
0a5eba1
use new test tarball in s3 when running C++ unit tests
ChuckHastings Aug 29, 2024
b59a60b
Update ci/test_cpp.sh
ChuckHastings Aug 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ci/test_cpp.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ nvidia-smi
# RAPIDS_DATASET_ROOT_DIR is used by test scripts
export RAPIDS_DATASET_ROOT_DIR="$(realpath datasets)"
pushd "${RAPIDS_DATASET_ROOT_DIR}"
./get_test_data.sh --subset
./get_test_data.sh --cpp_ci_subset
ChuckHastings marked this conversation as resolved.
Show resolved Hide resolved
popd

export GTEST_OUTPUT=xml:${RAPIDS_TESTS_DIR}/
Expand Down
8 changes: 8 additions & 0 deletions datasets/get_test_data.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,12 @@ cd "$( cd "$( dirname "$(realpath -m "${BASH_SOURCE[0]}")" )" && pwd )";
#
# FIXME: some test data needs to be extracted to "benchmarks", which is
# confusing now that there's dedicated datasets for benchmarks.
CPP_CI_DATASET_DATA="
# ~10s download
https://data.rapids.ai/cugraph/test/cpp_ci_datasets.tgz
test
"

BASE_DATASET_DATA="
# ~22s download
https://data.rapids.ai/cugraph/test/datasets.tgz
Expand Down Expand Up @@ -89,6 +95,8 @@ if hasArg "--benchmark"; then
DATASET_DATA="${BENCHMARK_DATASET_DATA}"
elif hasArg "--subset"; then
DATASET_DATA="${BASE_DATASET_DATA}"
elif hasArg "--cpp_ci_subset"; then
DATASET_DATA="${CPP_CI_DATASET_DATA}"
elif hasArg "--self_loops"; then
DATASET_DATA="${SELF_LOOPS_DATASET_DATA}"
# Do not include benchmark datasets by default - too big
Expand Down
Loading