Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

System submission disregards user-selected metrics #324

Closed
OscarWang114 opened this issue Sep 14, 2022 · 2 comments
Closed

System submission disregards user-selected metrics #324

OscarWang114 opened this issue Sep 14, 2022 · 2 comments
Assignees
@OscarWang114

Comments

@OscarWang114
Copy link
Collaborator

In summarization, the submission process disregards user-selected metrics and only evaluates rouge1, rouge2, rougeL, and length_ratio every time.
@OscarWang114 OscarWang114 self-assigned this Sep 14, 2022
@pfliu-nlp
Copy link
Collaborator

Hi @OscarWang114 this issue has been fixed in SDK: neulab/ExplainaBoard#454
Maybe when all existing PRs in this repo are merged, we can keep explainaboard web updated with explainaboard sdk.

@OscarWang114
Copy link
Collaborator Author

Yep! Confirmed it's working after merging #365

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@OscarWang114 OscarWang114

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@OscarWang114 @pfliu-nlp