Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: Skip invalid log lines in pytest_v2 log parser. #270

Merged

Conversation

buaabarty
Copy link
Contributor

Description

This pull request addresses an IndexError encountered in the parse_log_pytest_v2 function within the log_parsers.py file. The error occurred when the function attempted to access an index in the test_case list that did not exist, specifically when processing invalid log lines.

Changes Made

  • Added a conditional check to ensure that the test_case list has the expected length before attempting to access its elements. This prevents the IndexError by skipping over log lines that do not conform to the expected format.

Error Log

The error was observed with the following log entry:

2024-12-04 23:58:13,842 - ERROR - Error in evaluating model for astropy__astropy-14995: list index out of range
Traceback (most recent call last):
  ...
  File "/home/barty/miniconda3/lib/python3.12/site-packages/swebench/harness/log_parsers.py", line 158, in parse_log_pytest_v2
    test_status_map[test_case[0]] = test_case[1]
                                    ~~~~~~~~~^^^
IndexError: list index out of range

Impact

This fix improves the robustness of the log parser by ensuring it can handle unexpected log formats without crashing, thereby enhancing the overall stability of the evaluation process.

@buaabarty buaabarty force-pushed the buaabarty/fix-pytestv2-logparse branch from 6ec39dd to 7afd86f Compare December 5, 2024 08:21
@john-b-yang
Copy link
Member

Thanks for this fix @buaabarty, merging!

@john-b-yang john-b-yang merged commit 62cd3d2 into swe-bench:main Dec 10, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants