Skip to content

Commit

Permalink
Fixed issue in judge prompt.
Browse files Browse the repository at this point in the history
  • Loading branch information
JoelNiklaus committed Dec 23, 2024
1 parent c49e1e2 commit 4418e82
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion community_tasks/swiss_legal_evals.py
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ def compute(
**kwargs,
) -> dict[str, float]:
logger.info(f"Judging {len(formatted_docs)} samples with {self.short_judge_name}...")
questions = [formatted_doc.specific["question"] for formatted_doc in formatted_docs]
questions = [formatted_doc.specific["source"] for formatted_doc in formatted_docs]
options = [formatted_doc.choices for formatted_doc in formatted_docs]
golds = [formatted_doc.get_golds()[0] for formatted_doc in formatted_docs]
predictions = [response[0].result for response in responses]
Expand Down

0 comments on commit 4418e82

Please sign in to comment.