Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check return values of raft-ann-bench subprocess calls #1897

Merged
merged 1 commit into from
Oct 12, 2023

Conversation

benfred
Copy link
Member

@benfred benfred commented Oct 12, 2023

The python raft-ann-bench code launches subprocesses to execute C++ code. These scripts weren't checking the return values of the C++ programs though, and just waiting for them to finish. Fix this to propogate the failures up as exceptions by checking the subprocess return value.

After this change, having a failing subprocess looks something like:

2023-10-12 10:54:55 [info] Using the dataset file '/home/ben/code/raft/python/raft-ann-bench/datasets/glove-100-inner/base.fbin'
terminate called after throwing an instance of 'std::runtime_error'
  what():  read header of BinFile failed: /home/ben/code/raft/python/raft-ann-bench/datasets/glove-100-inner/base.fbin
Traceback (most recent call last):
  File "/home/ben/code/raft/python/raft-ann-bench/src/raft-ann-bench/run/__main__.py", line 324, in <module>
    main()
  File "/home/ben/code/raft/python/raft-ann-bench/src/raft-ann-bench/run/__main__.py", line 309, in main
    run_build_and_search(
  File "/home/ben/code/raft/python/raft-ann-bench/src/raft-ann-bench/run/__main__.py", line 113, in run_build_and_search
    subprocess.run(cmd, check=True)
  File "/home/ben/miniconda3/envs/raft/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/ben/miniconda3/envs/raft/bin/ann/RAFT_CAGRA_ANN_BENCH', '--build', '--data_prefix=/home/ben/code/raft/python/raft-ann-bench/datasets/', '--benchmark_out_format=json', '--benchmark_out=/home/ben/code/raft/python/raft-ann-bench/datasets/glove-100-inner/result/build/raft_cagra-10-10000.json', '/home/ben/code/raft/python/raft-ann-bench/src/raft-ann-bench/run/conf/temporary_glove-100-inner.json']' died with <Signals.SIGABRT: 6>.

The python raft-ann-bench code launches subprocesses to execute C++ code.
These scripts weren't checking the return values of the C++ programs though,
and just waiting for them to finish.  Fix this to propogate the failures up
as exceptions by checking the subprocess return value.

After this change, having a failing subprocess looks something like:

```
2023-10-12 10:54:55 [info] Using the dataset file '/home/ben/code/raft/python/raft-ann-bench/datasets/glove-100-inner/base.fbin'
terminate called after throwing an instance of 'std::runtime_error'
  what():  read header of BinFile failed: /home/ben/code/raft/python/raft-ann-bench/datasets/glove-100-inner/base.fbin
Traceback (most recent call last):
  File "/home/ben/code/raft/python/raft-ann-bench/src/raft-ann-bench/run/__main__.py", line 324, in <module>
    main()
  File "/home/ben/code/raft/python/raft-ann-bench/src/raft-ann-bench/run/__main__.py", line 309, in main
    run_build_and_search(
  File "/home/ben/code/raft/python/raft-ann-bench/src/raft-ann-bench/run/__main__.py", line 113, in run_build_and_search
    subprocess.run(cmd, check=True)
  File "/home/ben/miniconda3/envs/raft/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/ben/miniconda3/envs/raft/bin/ann/RAFT_CAGRA_ANN_BENCH', '--build', '--data_prefix=/home/ben/code/raft/python/raft-ann-bench/datasets/', '--benchmark_out_format=json', '--benchmark_out=/home/ben/code/raft/python/raft-ann-bench/datasets/glove-100-inner/result/build/raft_cagra-10-10000.json', '/home/ben/code/raft/python/raft-ann-bench/src/raft-ann-bench/run/conf/temporary_glove-100-inner.json']' died with <Signals.SIGABRT: 6>.
```
@benfred benfred requested a review from a team as a code owner October 12, 2023 18:02
@benfred benfred added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change and removed python labels Oct 12, 2023
@benfred benfred added the python label Oct 12, 2023
@cjnolet
Copy link
Member

cjnolet commented Oct 12, 2023

/merge

@rapids-bot rapids-bot bot merged commit 7d66266 into rapidsai:branch-23.12 Oct 12, 2023
59 checks passed
@benfred benfred deleted the check_ret_vals branch October 12, 2023 21:08
divyegala pushed a commit to divyegala/raft that referenced this pull request Oct 17, 2023
The python raft-ann-bench code launches subprocesses to execute C++ code. These scripts weren't checking the return values of the C++ programs though, and just waiting for them to finish.  Fix this to propogate the failures up as exceptions by checking the subprocess return value.

After this change, having a failing subprocess looks something like:

```
2023-10-12 10:54:55 [info] Using the dataset file '/home/ben/code/raft/python/raft-ann-bench/datasets/glove-100-inner/base.fbin'
terminate called after throwing an instance of 'std::runtime_error'
  what():  read header of BinFile failed: /home/ben/code/raft/python/raft-ann-bench/datasets/glove-100-inner/base.fbin
Traceback (most recent call last):
  File "/home/ben/code/raft/python/raft-ann-bench/src/raft-ann-bench/run/__main__.py", line 324, in <module>
    main()
  File "/home/ben/code/raft/python/raft-ann-bench/src/raft-ann-bench/run/__main__.py", line 309, in main
    run_build_and_search(
  File "/home/ben/code/raft/python/raft-ann-bench/src/raft-ann-bench/run/__main__.py", line 113, in run_build_and_search
    subprocess.run(cmd, check=True)
  File "/home/ben/miniconda3/envs/raft/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/ben/miniconda3/envs/raft/bin/ann/RAFT_CAGRA_ANN_BENCH', '--build', '--data_prefix=/home/ben/code/raft/python/raft-ann-bench/datasets/', '--benchmark_out_format=json', '--benchmark_out=/home/ben/code/raft/python/raft-ann-bench/datasets/glove-100-inner/result/build/raft_cagra-10-10000.json', '/home/ben/code/raft/python/raft-ann-bench/src/raft-ann-bench/run/conf/temporary_glove-100-inner.json']' died with <Signals.SIGABRT: 6>.
```

Authors:
  - Ben Frederickson (https://github.com/benfred)

Approvers:
  - Divye Gala (https://github.com/divyegala)

URL: rapidsai#1897
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improvement / enhancement to an existing function non-breaking Non-breaking change python
Projects
Development

Successfully merging this pull request may close these issues.

3 participants