You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Extend the code in utils.py to include validation for inputs, and potentially handling containers of inputs (mapping a TREC run file to either a list of lists, or a dictionary of lists, etc).
Validation needs to ensure that:
Document ranks are obeyed -- ranks are assumed to be strictly increasing, but gaps are allowed for representing tied elements. For example, having a sequence of ranks like [1, 1, 1, 4, 5] should be valid,
Scores are used only as a diagnostic against provided ranks. If the ranks and scores do not agree, either warn or error.
The text was updated successfully, but these errors were encountered:
Extend the code in
utils.py
to include validation for inputs, and potentially handling containers of inputs (mapping a TREC run file to either a list of lists, or a dictionary of lists, etc).Validation needs to ensure that:
[1, 1, 1, 4, 5]
should be valid,The text was updated successfully, but these errors were encountered: